Making use of LightningDataModule and simplification of data loading

This merge request adds LightningDataModule to better organize the code and make better use of lightning's features. This centralizes common tasks such as DataLoader creation and application of transforms into a base class to be inherited from.

Data loading was also simplified by removing custom Sample classes and maker functions, and the addition of RuntimeDataset and CachedDataset.

Remaining tasks:

• (@dcarron) Create a common/default DataModule for the shenzhen dataset that takes protocols and transforms as parameters to avoid copying code for each protocol configuration
• (@dcarron) Add typehints to ShenzenDataModule
• (@dcarron) Investigate issue where training a new model with ElasticDeformation as a transform converges more slowly if the data is not cached.
• (@andre.anjos) Update documentation on ShenzenDataModule
• (@biosignal) Update all datasets, using Shenzhen as a reference
  • (@mdelitroz) Montgomery
  • (@mdelitroz) Hivtb
  • (@andre.anjos) Indian
  • (@andre.anjos) Padchest
  • (@mdelitroz) Tbpoc
  • (@andre.anjos) tbx11_simplified -> renamed as tbx11k, protocol v1 (uses original dataset organisation)
  • (@andre.anjos) tbx11_simplified_v2 -> renamed as tbx11k, protocol v2 (uses original dataset organisation)
  • (@andre.anjos) mc_ch -> renamed as montgomery-shenzhen
  • (@andre.anjos) mc_ch_in -> renamed as montgomery-shenzhen-indian
  • (@andre.anjos) mc_ch_in_11k -> renamed as montgomery-shenzhen-indian-tbx11k-v1
  • (@andre.anjos) mc_ch_in_11k_v2 -> renamed as montgomery-shenzhen-indian-tbx11k-v2
  • (@andre.anjos) mc_ch_in_pc -> renamed as montgomery-shenzhen-indian-padchest
  • (@andre.anjos) nih_cxr14_re -> renamed as nih-cxr14 (n.b.: multi-class dataset, radiological findings)
  • (@andre.anjos) nih_cxr14_re_pc -> renamed as nih-cxr14-padchest
• (@dcarron) Update models
• (@dcarron) Update evaluation scripts
• (@dcarron) Update support for extra_validation datasets
• (@biosignal) Update full documentation
• (@biosignal) Update unit tests

Addresses the following issues:

• Closes #14 (closed)
• Closes #18 (closed)
• Closes #19 (closed)
• Closes #21 (closed)
• Closes #29 (closed)

doc/config.rst

@@ -8,7 +8,7 @@ Preset Configurations
 ---------------------
 
 This module contains preset configurations for baseline CNN architectures and
-datasets.
+datamodules.
 
 
 Models
@@ -22,114 +22,79 @@ Models
    ptbench.configs.models.alexnet_pretrained
    ptbench.configs.models.densenet
    ptbench.configs.models.densenet_pretrained
+   ptbench.configs.models.densenet_rs
    ptbench.configs.models.logistic_regression
    ptbench.configs.models.pasa
    ptbench.configs.models.signs_to_tb
-   ptbench.configs.models_datasets.densenet_rs
 
 
-.. _ptbench.configs.datasets:
+.. _ptbench.configs.datamodules:
 
-Datasets
-========
+Preset Datamodules
+==================
 
-Datasets include iterative accessors to raw data
-(:ref:`ptbench.setup.datasets`) including data pre-processing and augmentation,
-if applicable.  Use these datasets for training and evaluating your models.
+Datamodules include iterative accessors to raw data
+(:ref:`ptbench.setup.databases`) including data pre-processing to a minimum
+input standard, if applicable.  Use these datamodules for training and evaluating
+your models.
 
 .. autosummary::
-   :toctree: api/configs/datasets
+   :toctree: api/configs/datamodules
    :template: config.rst
 
-   ptbench.configs.datasets.indian.default
-   ptbench.configs.datasets.indian.rgb
-   ptbench.configs.datasets.indian_RS.default
-   ptbench.configs.datasets.mc_ch.default
-   ptbench.configs.datasets.mc_ch.rgb
-   ptbench.configs.datasets.mc_ch_RS.default
-   ptbench.configs.datasets.mc_ch_in.default
-   ptbench.configs.datasets.mc_ch_in.rgb
-   ptbench.configs.datasets.mc_ch_in_RS.default
-   ptbench.configs.datasets.mc_ch_in_11k.default
-   ptbench.configs.datasets.mc_ch_in_11k.rgb
-   ptbench.configs.datasets.mc_ch_in_11k_RS.default
-   ptbench.configs.datasets.mc_ch_in_11kv2.default
-   ptbench.configs.datasets.mc_ch_in_11kv2.rgb
-   ptbench.configs.datasets.mc_ch_in_11kv2_RS.default
-   ptbench.configs.datasets.mc_ch_in_pc.default
-   ptbench.configs.datasets.mc_ch_in_pc.rgb
-   ptbench.configs.datasets.mc_ch_in_pc_RS.default
-   ptbench.configs.datasets.montgomery.default
-   ptbench.configs.datasets.montgomery.rgb
-   ptbench.configs.datasets.montgomery_RS.default
-   ptbench.configs.datasets.nih_cxr14_re.cardiomegaly
-   ptbench.configs.datasets.nih_cxr14_re.default
-   ptbench.configs.datasets.nih_cxr14_re_pc.idiap
-   ptbench.configs.datasets.padchest.cardiomegaly_idiap
-   ptbench.configs.datasets.padchest.idiap
-   ptbench.configs.datasets.padchest.no_tb_idiap
-   ptbench.configs.datasets.padchest.tb_idiap
-   ptbench.configs.datasets.padchest.tb_idiap_rgb
-   ptbench.configs.datasets.padchest_RS.tb_idiap
-   ptbench.configs.datasets.shenzhen.default
-   ptbench.configs.datasets.shenzhen.rgb
-   ptbench.configs.datasets.shenzhen_RS.default
-   ptbench.configs.datasets.tbx11k_simplified.default
-   ptbench.configs.datasets.tbx11k_simplified.rgb
-   ptbench.configs.datasets.tbx11k_simplified_RS.default
-   ptbench.configs.datasets.tbx11k_simplified_v2.default
-   ptbench.configs.datasets.tbx11k_simplified_v2.rgb
-   ptbench.configs.datasets.tbx11k_simplified_v2_RS.default
-
-
-.. _ptbench.configs.datasets.folds:
-
-Cross-Validation Datasets
-=========================
+   ptbench.data.indian.default
+   ptbench.data.montgomery.default
+   ptbench.data.nih_cxr14.default
+   ptbench.data.padchest.idiap
+   ptbench.data.shenzhen.default
+   ptbench.data.tbx11k.v1_healthy_vs_atb
+   ptbench.data.tbx11k.v2_others_vs_atb
+
+
+.. _ptbench.configs.datamodules.remix:
+
+Remix Datamodules
+=================
+
+We provide some aggregated datamodules to facilitate cross-database development.
+
+.. autosummary::
+   :toctree: api/configs/datamodules
+   :template: config.rst
+
+   ptbench.data.montgomery_shenzhen.default
+   ptbench.data.montgomery_shenzhen_indian.default
+   ptbench.data.montgomery_shenzhen_indian_padchest.default
+   ptbench.data.montgomery_shenzhen_indian_tbx11k.v1_healthy_vs_atb
+   ptbench.data.montgomery_shenzhen_indian_tbx11k.v2_others_vs_atb
+   ptbench.data.nih_cxr14_padchest.idiap
+
+
+.. _ptbench.configs.datamodules.folds:
+
+Cross-validation Datamodules
+============================
 
 We support cross-validation with precise preset folds.  In this section, you
 will find the configuration for the first fold (fold-0) for all supported
-datasets.  Nine other folds are available for every configuration (from 1 to
-9), making up 10 folds per supported dataset.
+datamodules.  Nine other folds are available for every configuration (from 1 to
+9), making up 10 folds per supported datamodule.
 
 
 .. autosummary::
-   :toctree: api/configs/datasets
+   :toctree: api/configs/datamodules
    :template: config.rst
 
-   ptbench.configs.datasets.hivtb.fold_0
-   ptbench.configs.datasets.hivtb.fold_0_rgb
-   ptbench.configs.datasets.hivtb_RS.fold_0
-   ptbench.configs.datasets.indian.fold_0
-   ptbench.configs.datasets.indian.fold_0_rgb
-   ptbench.configs.datasets.indian_RS.fold_0
-   ptbench.configs.datasets.mc_ch.fold_0
-   ptbench.configs.datasets.mc_ch.fold_0_rgb
-   ptbench.configs.datasets.mc_ch_RS.fold_0
-   ptbench.configs.datasets.mc_ch_in.fold_0
-   ptbench.configs.datasets.mc_ch_in.fold_0_rgb
-   ptbench.configs.datasets.mc_ch_in_RS.fold_0
-   ptbench.configs.datasets.mc_ch_in_11k.fold_0
-   ptbench.configs.datasets.mc_ch_in_11k.fold_0_rgb
-   ptbench.configs.datasets.mc_ch_in_11k_RS.fold_0
-   ptbench.configs.datasets.mc_ch_in_11kv2.fold_0
-   ptbench.configs.datasets.mc_ch_in_11kv2.fold_0_rgb
-   ptbench.configs.datasets.mc_ch_in_11kv2_RS.fold_0
-   ptbench.configs.datasets.montgomery.fold_0
-   ptbench.configs.datasets.montgomery.fold_0_rgb
-   ptbench.configs.datasets.montgomery_RS.fold_0
-   ptbench.configs.datasets.shenzhen.fold_0
-   ptbench.configs.datasets.shenzhen.fold_0_rgb
-   ptbench.configs.datasets.shenzhen_RS.fold_0
-   ptbench.configs.datasets.tbpoc.fold_0
-   ptbench.configs.datasets.tbpoc.fold_0_rgb
-   ptbench.configs.datasets.tbpoc_RS.fold_0
-   ptbench.configs.datasets.tbx11k_simplified.fold_0
-   ptbench.configs.datasets.tbx11k_simplified.fold_0_rgb
-   ptbench.configs.datasets.tbx11k_simplified_RS.fold_0
-   ptbench.configs.datasets.tbx11k_simplified_v2.fold_0
-   ptbench.configs.datasets.tbx11k_simplified_v2.fold_0_rgb
-   ptbench.configs.datasets.tbx11k_simplified_v2_RS.fold_0
-
+   ptbench.data.hivtb.fold_0
+   ptbench.data.indian.fold_0
+   ptbench.data.montgomery.fold_0
+   ptbench.data.montgomery_shenzhen.fold_0
+   ptbench.data.montgomery_shenzhen_indian.fold_0
+   ptbench.data.montgomery_shenzhen_indian_tbx11k.v1_fold_0
+   ptbench.data.montgomery_shenzhen_indian_tbx11k.v2_fold_0
+   ptbench.data.shenzhen.fold_0
+   ptbench.data.tbpoc.fold_0
+   ptbench.data.tbx11k.v1_fold_0
+   ptbench.data.tbx11k.v2_fold_0
 
 .. include:: links.rst