Configs

class InnerEye.ML.config.EnsembleAggregationType(value)

An enumeration.

Average = 'Average'

class InnerEye.ML.config.MixtureLossComponent(weight: float, loss_type: SegmentationLoss, class_weight_power: float)

A member of the value of the mixture_loss_components parameter.

Parameters for the loss function will be pulled from the model config,

except SegmentationModelBase.loss_class_weight_power which is ignored.

class_weight_power: float

For weighted loss, power to which to raise the weights per class.

loss_type: SegmentationLoss

weight: float

class InnerEye.ML.config.ModelArchitectureConfig

Supported model architecture types

Basic = 'Basic'

UNet2D = 'UNet2D'

UNet3D = 'UNet3D'

class InnerEye.ML.config.PaddingMode(value)

Supported padding modes for numpy and torch image padding.

Edge = 'edge'

Pads with the edge values of array.

LinearRamp = 'linear_ramp'

Pads with the linear ramp between end_value and the array edge value.

Maximum = 'maximum'

Pads with the maximum value of all or part of the vector along each axis.

Mean = 'mean'

Pads with the mean value of all or part of the vector along each axis.

Median = 'median'

Pads with the median value of all or part of the vector along each axis.

Minimum = 'minimum'

Pads with the minimum value of all or part of the vector along each axis.

NoPadding = 'no_padding'

No padding is performed

Reflect = 'reflect'

Pads with the reflection of the vector mirrored on the first and last values of the vector along each axis.

Symmetric = 'symmetric'

Pads with the reflection of the vector mirrored along the edge of the array.

Wrap = 'wrap'

Pads with the wrap of the vector along the axis. The first values are used to pad the end and the end values are used to pad the beginning.

Zero = 'constant'

Zero padding scheme.

class InnerEye.ML.config.PhotometricNormalizationMethod(value)

Contains the valid methods that can be used to perform photometric normalization of a medical image.

CtWindow = 'CT Window'

MriWindow = 'MRI Window'

SimpleNorm = 'Simple Norm'

TrimmedNorm = 'Trimmed Norm'

Unchanged = 'None'

class InnerEye.ML.config.SegmentationLoss(value)

The types of training loss that are supported for segmentation models. Parameters that can be set in the segmentation configs related to loss functions:

SoftDice: SegmentationModelBase.loss_class_weight_power

CrossEntropy: SegmentationModelBase.loss_class_weight_power, DeepLearningConfig.label_smoothing_eps

Focal: SegmentationModelBase.loss_class_weight_power, DeepLearningConfig.label_smoothing_eps, SegmentationModelBase.focal_loss_gamma

Mixture: SegmentationModelBase.mixture_loss_components. See MixtureLossComponent for component parameters.

CrossEntropy = 'CrossEntropy'

Focal = 'Focal'

Mixture = 'Mixture'

SoftDice = 'SoftDice'

class InnerEye.ML.config.SegmentationModelBase(center_size: Optional[Tuple[int, int, int]] = None, inference_stride_size: Optional[Tuple[int, int, int]] = None, min_l_rate: float = 0, largest_connected_component_foreground_classes: Optional[Sequence[Union[str, Tuple[str, Optional[float]]]]] = None, **params: Any)

A class that holds all settings that are specific to segmentation models.

activation_map_layers: Optional[List[str]] = None

Layer name hierarchy (parent, child recursive) as by model definition. If None, no activation maps will be saved

allow_incomplete_labels: bool = False

architecture: str = 'Basic'

The segmentation model architecture to use. Valid options are defined at ModelArchitectureConfig: ‘Basic (DeepMedic)’, ‘UNet3D’, ‘UNet2D’

property center_size: Tuple[int, int, int]

Gets the size of the center crop that the model predicts.

check_exclusive: bool = True

FocalLoss) will fail with a cryptic error message. Set to false if you are sure that you want to use labels that are not mutually exclusive.

class_and_index_with_background() → Dict[str, int]

Returns a dict of class names to indices, including the background class. The class index assumes that background is class 0, foreground starts at 1. For example, if the ground_truth_ids are [“foo”, “bar”], the result is {“background”: 0, “foo”: 1, “bar”: 2}

Returns:

A dict, one entry for each entry in ground_truth_ids + 1 for the background class.

class_weights: Optional[List[float]] = None

The per-class probabilities for picking a center point of a crop.

col_type_converters: Optional[Dict[str, Any]] = None

Dictionary of types to enforce for certain DataFrame columns, where key is column name and value is desired type.

colours: List[TupleInt3] = None

Colours in (R, G, B) for the structures, same order as in ground_truth_ids_display_names

comparison_blob_storage_paths: List[Tuple[str, str]] = None

List of (name, container) pairs, where name is a descriptive name and container is a Azure ML storage account container name to be used for statistical comparisons

create_and_set_torch_datasets(for_training: bool = True, for_inference: bool = True) → None

Creates torch datasets for all model execution modes, and stores them in the object.

create_model() → Any

Creates a PyTorch model from the settings stored in the present object. :return: The network model as a torch.nn.Module object

crop_size: TupleInt3 = (1, 1, 1)

The size of the random crops that will be drawn from the input images during training. This is also the input size of the model.

dataset_expected_spacing_xyz: Optional[TupleFloat3] = None

The spacing X, Y, Z expected for all images in the dataset

debug_mode: bool = False

If true, create additional plots during image normalization.

disable_extra_postprocessing: bool = False

Whether to ignore slice_exclusion_rules and summed_probability_rules even if defined

ensemble_aggregation_type: EnsembleAggregationType = 'Average'

The aggregation method to use when testing ensemble models. See :attr: EnsembleAggregationType for options.

property example_images_folder: Path

Gets the full path in which the example images should be stored during training.

feature_channels: List[int] = None

The number of feature channels at different stages of the model.

fill_holes: List[bool] = None

List of bool specifiying if structures need filling holes. If True, the output of the model for that class will include postprocessing to fill holes, in the same order as in ground_truth_ids_display_names

focal_loss_gamma: float = 1.0

Gamma value for focal loss: weight for each pixel is posterior likelihood to the power -focal_loss_gamma.

get_cropped_image_sample_transforms() → ModelTransformsPerExecutionMode

Get transforms to perform on cropped samples for each model execution mode. By default no transformation is performed.

get_full_image_sample_transforms() → ModelTransformsPerExecutionMode

Get transforms to perform on full image samples for each model execution mode. By default only PhotometricNormalization is performed.

get_model_train_test_dataset_splits(dataset_df: DataFrame) → DatasetSplits

Computes the training, validation and test splits for the model, from a dataframe that contains the full dataset.

Parameters:

dataset_df – A dataframe that contains the full dataset that the model is using.

Returns:

An instance of DatasetSplits with dataframes for training, validation and testing.

get_output_size(execution_mode: ModelExecutionMode = ModelExecutionMode.TRAIN) → Optional[Tuple[int, int, int]]

Returns shape of model’s output tensor for training, validation and testing inference modes

get_parameter_search_hyperdrive_config(run_config: ScriptRunConfig) → HyperDriveConfig

Turns the given AzureML estimator (settings for running a job in AzureML) into a configuration object for doing hyperparameter searches.

Parameters:

run_config – The settings for running a single AzureML job.

Returns:

A HyperDriveConfig object for running multiple AzureML jobs.

ground_truth_ids: List[str] = None

The names of the ground truth channels that the model consumes. These channels must be present in the dataset.csv file

ground_truth_ids_display_names: List[str] = None

User friendly display names to be used for each of the predicted GT classes. Default is ground_truth_ids if None provided

image_channels: List[str] = None

The names of the image input channels that the model consumes. These channels must be present in the dataset.csv file.

inference_batch_size: int = 8

The batch size to use for inference forward pass.

property inference_stride_size: Optional[Tuple[int, int, int]]

Gets the stride size that should be used when stitching patches at inference time.

interpreter: str = 'Default_Interpreter'

is_plotting_enabled: bool = True

If true, various overview plots with results are generated during model evaluation. Set to False if you see non-deterministic pull request build failures.

kernel_size: int = 3

The size of the convolution kernels.

property largest_connected_component_foreground_classes: Optional[Sequence[Union[str, Tuple[str, Optional[float]]]]]

Gets the list of classes for which the largest connected components should be computed when predicting.

level: int = 50

The level setting for the PhotometricNormalizationMethod.CtWindow normalization.

loss_class_weight_power: Optional[float] = None

For weighted loss, power to which to raise the weights per class. If this is None, loss is not weighted.

loss_type: SegmentationLoss = 'SoftDice'

The loss type to use during training. Valid options are defined at SegmentationLoss: “SoftDice”, “CrossEntropy”, “Focal”, “Mixture”

manufacturer: str = 'Default_Manufacturer'

mask_id: Optional[str] = None

The name of the channel that contains the inside/outside body information (to mask out the background). This channel must be present in the dataset

mixture_loss_components: Optional[List[MixtureLossComponent]] = None

List of pairs of weights, loss types and class-weight-power values for use when loss_type is SegmentationLoss.MixtureLoss”.

name = 'SegmentationModelBase'

norm_method: PhotometricNormalizationMethod = 'CT Window'

num_downsampling_paths: int = 4

The number of image levels used in Unet (in encoding and decoding paths).

property number_of_classes: int

Returns the number of ground truth ids, including the background class.

property number_of_image_channels: int

Gets the number of image input channels that the model has (usually 1 CT channel, or multiple MR).

output_range: TupleFloat2 = (-1.0, 1.0)

The value range that image normalization should produce. This is the input range to the network.

padding_mode: PaddingMode = 'edge'

Padding mode to use for training and inference. See PaddingMode for valid options.

param = <param.parameterized.Parameters object>

posterior_smoothing_mm: Optional[TupleInt3] = None

The size of the smoothing kernel in mm to be used for smoothing posteriors before computing the final segmentations. No smoothing is performed if set to None.

read_dataset_into_dataframe_and_pre_process() → None

Loads a dataset from the dataset_csv file, and stores it in the present object.

roi_interpreted_types: List[str] = None

set_derived_model_properties(model: Any) → None

Updates the model config parameters that depend on how the segmentation model is structured. In particular, this computes the model’s output size for the training and the inference crops. If the inference stride size is not set, then set it to be equal to the size of the inference output patches.

sharpen: float = 0.9

Sharpen parameter specifies number of standard deviations from mean to be included in window range. Used in PhotometricNormalizationMethod.MriWindow

show_patch_sampling: int = 1

slice_exclusion_rules: List[SliceExclusionRule] = []

List of rules for structures that should be prevented from sharing the same slice. These are not applied if disable_extra_postprocessing is True. Parameter should be a list of SliceExclusionRule objects.

store_dataset_sample: bool = False

If True save image and segmentations for one image in a batch for each training epoch

summed_probability_rules: List[SummedProbabilityRule] = []

List of rules for class pairs whose summed probability is used to create the segmentation map from predicted posterior probabilities. These are not applied if disable_extra_postprocessing is True. Parameter should be a list of SummedProbabilityRule objects.

tail: List[float] = None

Tail parameter allows window range to be extended to right, used in PhotometricNormalizationMethod.MriWindow. The value must be a list with one entry per input channel if the model has multiple input channels

test_crop_size: Optional[TupleInt3] = None

The crop size to use for model testing. If nothing is specified, crop_size parameter is used instead, i.e. training and testing crop size will be the same.

trim_percentiles: TupleFloat2 = (1.0, 99.0)

Percentile at which to trim input distribution prior to normalization. Used in PhotometricNormalizationMethod.TrimmedNorm

validate() → None

Validates the parameters stored in the present object.

static validate_class_weights(class_weights: List[float]) → None

Checks that the given list of class weights is valid: The weights must be positive and add up to 1.0. Raises a ValueError if that is not the case.

static validate_inference_stride_size(inference_stride_size: Optional[Tuple[int, int, int]], output_size: Optional[Tuple[int, int, int]]) → None

Checks that patch stride size is positive and smaller than output patch size to ensure that posterior predictions are obtained for all pixels

window: int = 600

The Window setting for the PhotometricNormalizationMethod.CtWindow normalization.

class InnerEye.ML.config.SliceExclusionRule(higher_class: str, lower_class: str, higher_dominates: bool)

Rule mandating that voxels of higher_class must always be in strictly higher slices than those of lower_class (slices are along the z-axis). If this is not the case, then if higher_dominates is True, any lower_class voxels in a higher or equal slice to any higher_class voxels are converted to higher_class. If higher_dominates is False, any higher_class voxels in a lower or equal slice to any lower_class voxels are converted to lower_class.

higher_class: str

higher_dominates: bool

lower_class: str

validate(ground_truth_ids: List[str]) → None

Check this rule is valid for the given set of ground_truth_ids.

class InnerEye.ML.config.SummedProbabilityRule(first_class: str, second_class: str, external_class: str)

At the boundary between two classes, the predicted class probability for both classes may be low. To avoid these voxels being categorized as external voxels, the summed probability of first_class and second_class will be used to create the segmentation map. If the summed probability of first_class and second_class is greater than external_class, we will label the voxel with first_class or second_class (whichever has the higher probability) instead of external_class.

external_class: str

first_class: str

second_class: str

validate(ground_truth_ids: List[str]) → None

Check this rule is valid for the given set of ground_truth_ids.

InnerEye.ML.config.equally_weighted_classes(foreground_classes: List[str], background_weight: Optional[float] = None) → List

Computes a list of weights for the background class and all foreground classes. If no background_weight is given, all foreground classes and the background class (class index 0) are given equal weight. If a background_weight is given explicitly, that weight is assigned to class index 0, and the rest of the weight is equally distributed across all foreground classes. All weights will sum to 1.0

Parameters:

• foreground_classes – The list of foreground classes that the model uses.

• background_weight – The weight that should be given to the background class (index 0). This can be None.

Returns:

A list of length len(foreground_classes) + 1, with weights for all classes including the background class. The weights will sum to 1.0

InnerEye.ML.config.get_center_size(arch: str, crop_size: Tuple[int, int, int]) → Tuple[int, int, int]

Computes the size of the output tensor, if the model is fed with an input tensor of the given crop_size. This makes a lot of assumptions about the architectures that are hardcoded, this method should be used with care.

Parameters:

• arch – The model architecture that is used.

• crop_size – The size of the model’s input tensor.

Returns:

The size of the model’s output tensor.

class InnerEye.ML.model_config_base.ModelConfigBase(**params: Any)

create_and_set_torch_datasets(for_training: bool = True, for_inference: bool = True) → None

Creats and sets torch datasets for training and validation, and stores them in the self._datasets_for_training field. Similarly, create torch datasets in the form required for model inference, for all of the 3 splits of the full data, and stored them in the self._datasets_for_training and/or self._datasets_for_inference fields. This is an abstract method that each specific model should override. If for_training and/or for_inference are False, the derived method may still create the corresponding datasets, but should not assume that the relevant splits (train/test/val) are non-empty. If either or both is True, they must create the corresponding datasets, and should be able to make the assumption.

Parameters:

• for_training – whether to create the datasets required for training.

• for_inference – whether to create the datasets required for inference.

create_data_loaders() → Dict[ModelExecutionMode, Any]

Creates the torch DataLoaders that supply the training and the validation set during training only. :return: A dictionary, with keys ModelExecutionMode.TRAIN and ModelExecutionMode.VAL, and their respective

data loaders.

create_model() → Any

Creates a PyTorch model from the settings stored in the present object. This is an abstract method that each model class (segmentation, regression) should override. Return type is LightningModule, not Any - but we want to avoid importing torch at this point.

generate_custom_report(report_dir: Path, model_proc: ModelProcessing) → Path

Enables creating a custom results report, given the metrics files written during model training and inference. By default, this method is a no-op.

Parameters:

• report_dir – The output directory where the generated report should be saved.

• model_proc – The type of model that is registered (single or ensemble)

Returns:

The path to the generated report file.

get_cross_validation_dataset_splits(dataset_split: DatasetSplits) → DatasetSplits

When running cross validation, this method returns the dataset split that should be used for the currently executed cross validation split.

Parameters:

dataset_split – The full dataset, split into training, validation and test section.

Returns:

The dataset split with training and validation sections shuffled according to the current cross validation index.

get_cross_validation_hyperdrive_config(run_config: ScriptRunConfig) → HyperDriveConfig

Returns a configuration for AzureML Hyperdrive that varies the cross validation split index.

Parameters:

run_config – The AzureML run configuration object that training for an individual model.

Returns:

A hyperdrive configuration object.

get_cross_validation_hyperdrive_sampler() → GridParameterSampling

Returns the cross validation sampler, required to sample the entire parameter space for cross validation.

get_dataset_splits() → DatasetSplits

The Train/Val/Test dataset splits. If number_of_cross_validation_splits > 1 then the corresponding cross_validation_split_index will be used to get the cross validation split for Train/Val/Test. :return: DatasetSplits

get_hyperdrive_config(run_config: ScriptRunConfig) → HyperDriveConfig

Returns the HyperDrive config for either parameter search or cross validation (if number_of_cross_validation_splits > 1).

Parameters:

run_config – AzureML estimator

Returns:

HyperDriveConfigs

get_model_train_test_dataset_splits(dataset_df: DataFrame) → DatasetSplits

Computes the training, validation and test splits for the model, from a dataframe that contains the full dataset.

Parameters:

dataset_df – A dataframe that contains the full dataset that the model is using.

Returns:

An instance of DatasetSplits with dataframes for training, validation and testing.

get_parameter_search_hyperdrive_config(run_config: ScriptRunConfig) → HyperDriveConfig

Returns a configuration for AzureML Hyperdrive that should be used when running hyperparameter tuning. This is an abstract method that each specific model should override.

Parameters:

run_config – The AzureML estimator object that runs model training.

Returns:

A hyperdrive configuration object.

get_torch_dataset_for_inference(mode: ModelExecutionMode) → Any

Returns a torch Dataset for running the model in inference mode, on the given split of the full dataset. The torch dataset must return data in the format required for running the model in inference mode.

Parameters:

mode – The mode of the model, either test, train or val.

Returns:

A torch Dataset object.

name = 'ModelConfigBase'

param = <param.parameterized.Parameters object>

pre_process_dataset_dataframe() → None

Performs any dataframe pre-processing functions, default is identity :return:

read_dataset_if_needed() → DataFrame

If the present object already stores a data frame, return it. Otherwise, read it from file. :return: The data frame that the model uses.

read_dataset_into_dataframe_and_pre_process() → None

Loads a dataset from a file or other source, and saves it into the model’s data_frame property. Applying any pre-processing functions defined in pre_process_dataset_dataframe The data frame should contain all of the training, test, and validation data.

set_derived_model_properties(model: Any) → None

A hook to adjust the model configuration that is stored in the present object to match the torch model given in the argument. This hook is called after adjusting the model for mixed precision and parallel training.

Parameters:

model – The torch model.

write_dataset_files(root: Optional[Path] = None) → None

Writes to disk the dataset.csv and the train and test files :return: None

class InnerEye.ML.model_config_base.ModelConfigBaseMeta(name, bases, namespace, **kwargs)

Metaclass to make the hierarchy explicit for ModelConfigBase

class InnerEye.ML.model_config_base.ModelTransformsPerExecutionMode(train: Optional[Callable] = None, val: Optional[Callable] = None, test: Optional[Callable] = None)

This is a container class used to store transformations to apply to each sample depending on each execution mode (train, validation and test)

class InnerEye.ML.deep_learning_config.DatasetParams(**params)

all_azure_dataset_ids() → List[str]

Returns a list with all azure dataset IDs that are specified in self.azure_dataset_id and self.extra_azure_dataset_ids

all_dataset_mountpoints() → List[str]

Returns a list with all dataset mount points that are specified in self.dataset_mountpoint and self.extra_dataset_mountpoints

all_local_dataset_paths() → List[Path]

Returns a list with all dataset mount points that are specified in self.local_dataset and self.extra_local_dataset_paths

azure_dataset_id: str = ''

dataset_mountpoint: str = ''

extra_azure_dataset_ids: List[str] = []

extra_dataset_mountpoints: List[str] = []

extra_local_dataset_paths: List[Optional[Path]] = []

local_dataset: Optional[Path] = None

name = 'DatasetParams'

param = <param.parameterized.Parameters object>

validate() → None

class InnerEye.ML.deep_learning_config.DeepLearningConfig(**params: Any)

A class that holds all settings that are shared across segmentation models and regression/classification models.

avoid_process_spawn_in_data_loaders: bool = False

property compute_grad_cam: bool

property compute_mean_teacher_model: bool

Returns True if the mean teacher model should be computed.

dataset_csv: str = 'dataset.csv'

Name of the csv file providing information on the dataset to be used.

property dataset_data_frame: Optional[DataFrame]

Gets the pandas data frame that the model uses. :return:

drop_last_batch_in_training: bool = False

get_total_number_of_training_epochs() → int

Returns the number of epochs for which a model will be trained. :return:

get_total_number_of_validation_epochs() → int

Returns the number of epochs for which a model will be validated. :return:

get_train_epochs() → List[int]

Returns the epochs for which training will be performed. :return:

property is_scalar_model: bool

Returns True if the present model configuration belongs to the high-level category ModelCategory.Scalar i.e. for Classification or Regression models.

property is_segmentation_model: bool

Returns True if the present model configuration belongs to the high-level category ModelCategory.Segmentation.

label_smoothing_eps: float = 0.0

load_checkpoint_and_modify(path_to_checkpoint: Path) → Dict[str, Any]

By default, uses torch.load to read and return the state dict from the checkpoint file, and does no modification of the checkpoint file.

Overloading this function: When weights_url or local_weights_path is set, the file downloaded may not be in the exact format expected by the model’s load_state_dict() - for example, pretrained Imagenet weights for networks may have mismatched layer names in different implementations. In such cases, you can overload this function to extract the state dict from the checkpoint.

NOTE: The model checkpoint will be loaded using the torch function load_state_dict() with argument strict=False, so extra care needs to be taken to check that the state dict is valid. Check the logs for warnings related to missing and unexpected keys. See https://pytorch.org/tutorials/beginner/saving_loading_models.html#warmstarting-model-using-parameters -from-a-different-model for an explanation on why strict=False is useful when loading parameters from other models.

Parameters:

path_to_checkpoint – Path to the checkpoint file.

Returns:

Dictionary with model and optimizer state dicts. The dict should have at least the following keys: 1. Key ModelAndInfo.MODEL_STATE_DICT_KEY and value set to the model state dict. 2. Key ModelAndInfo.EPOCH_KEY and value set to the checkpoint epoch. Other (optional) entries corresponding to keys ModelAndInfo.OPTIMIZER_STATE_DICT_KEY and ModelAndInfo.MEAN_TEACHER_STATE_DICT_KEY are also supported.

log_summaries_to_files: bool = True

log_to_parent_run: bool = False

max_batch_grad_cam: int = 0

mean_teacher_alpha: float = None

property model_category: ModelCategory

Gets the high-level model category that this configuration objects represents (segmentation or scalar output).

name = 'DeepLearningConfig'

num_dataload_workers: int = 2

param = <param.parameterized.Parameters object>

pin_memory: bool = True

restrict_subjects: Optional[str] = ''

shuffle: bool = True

train_batch_size: int = 4

use_imbalanced_sampler_for_training: bool = False

use_model_parallel: bool = False

validate() → None

Validates the parameters stored in the present object.

class InnerEye.ML.deep_learning_config.DeepLearningFileSystemConfig(**params)

High level config to abstract the file system related configs for deep learning models

add_subfolder(subfolder: str) → DeepLearningFileSystemConfig

Creates a new output folder configuration, where both outputs and logs go into the given subfolder inside the present outputs folder.

Parameters:

subfolder – The subfolder that should be created.

Returns:

static create(project_root: Path, is_offline_run: bool, model_name: str, output_to: Optional[str] = None) → DeepLearningFileSystemConfig

Creates a new object that holds output folder configurations. When running inside of AzureML, the output folders will be directly under the project root. If not running inside AzureML, a folder with a timestamp will be created for all outputs and logs.

Parameters:

• project_root – The root folder that contains the code that submitted the present training run. When running inside the InnerEye repository, it is the git repo root. When consuming InnerEye as a package, this should be the root of the source code that calls the package.

• is_offline_run – If true, this is a run outside AzureML. If False, it is inside AzureML.

• model_name – The name of the model that is trained. This is used to generate a run-specific output folder.

• output_to – If provided, the output folders will be created as a subfolder of this argument. If not given, the output folders will be created inside of the project root.

logs_folder: Path = PosixPath('.')

name = 'DeepLearningFileSystemConfig'

outputs_folder: Path = PosixPath('.')

param = <param.parameterized.Parameters object>

project_root: Path = PosixPath('.')

run_folder: Path = PosixPath('.')

class InnerEye.ML.deep_learning_config.LRSchedulerType(value)

Supported lr scheduler types for model training

Cosine = 'Cosine'

Exponential = 'Exponential'

MultiStep = 'MultiStep'

Polynomial = 'Polynomial'

Step = 'Step'

class InnerEye.ML.deep_learning_config.LRWarmUpType(value)

Supported LR warm up types for model training

Linear = 'Linear'

NoWarmUp = 'NoWarmUp'

class InnerEye.ML.deep_learning_config.ModelCategory(value)

Describes the different high-level model categories that the codebase supports.

Classification = 'Classification'

Regression = 'Regression'

Segmentation = 'Segmentation'

property is_scalar: bool

Return True if the current ModelCategory is either Classification or Regression

class InnerEye.ML.deep_learning_config.MultiprocessingStartMethod(value)

Different methods for starting data loader processes.

fork = 'fork'

forkserver = 'forkserver'

spawn = 'spawn'

class InnerEye.ML.deep_learning_config.OptimizerParams(**params)

adam_betas: Tuple[float, float] = (0.9, 0.999)

l_rate: float = 0.0001

l_rate_exponential_gamma: float = 0.9

l_rate_multi_step_gamma: float = 0.1

l_rate_multi_step_milestones: Optional[List[int]] = None

l_rate_polynomial_gamma: float = 0.0001

l_rate_scheduler: LRSchedulerType = 'Polynomial'

l_rate_step_gamma: float = 0.1

l_rate_step_step_size: int = 50

l_rate_warmup: LRWarmUpType = 'NoWarmUp'

l_rate_warmup_epochs: int = 0

property min_l_rate: float

momentum: float = 0.6

name = 'OptimizerParams'

opt_eps: float = 0.0001

optimizer_type: OptimizerType = 'Adam'

param = <param.parameterized.Parameters object>

rms_alpha: float = 0.9

validate() → None

weight_decay: float = 0.0001

class InnerEye.ML.deep_learning_config.OptimizerType(value)

Supported optimizers for model training

AMSGrad = 'AMSGrad'

Adam = 'Adam'

RMSprop = 'RMSprop'

SGD = 'SGD'

class InnerEye.ML.deep_learning_config.OutputParams(**params)

property checkpoint_folder: Path

Gets the full path in which the model checkpoints should be stored during training.

create_filesystem(project_root: Path = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/innereye-deeplearning/checkouts/latest')) → None

Creates new file system settings (outputs folder, logs folder) based on the information stored in the present object. If any of the folders do not yet exist, they are created.

Parameters:

project_root – The root folder for the codebase that triggers the training run.

file_system_config: DeepLearningFileSystemConfig = DeepLearningFileSystemConfig(logs_folder=PosixPath('.'), name='DeepLearningFileSystemConfig00002', outputs_folder=PosixPath('.'), project_root=PosixPath('.'), run_folder=PosixPath('.'))

get_path_to_best_checkpoint() → Path

Returns the full path to a checkpoint file that was found to be best during training, whatever criterion was applied there.

get_path_to_checkpoint() → Path

Returns the full path to a recovery checkpoint.

property logs_folder: Path

Gets the full path in which the model logs should be stored.

property model_name: str

Gets the human readable name of the model (e.g., Liver). This is usually set from the class name. :return: A model name as a string.

name = 'OutputParams'

output_to: str = ''

property outputs_folder: Path

Gets the full path in which the model outputs should be stored.

param = <param.parameterized.Parameters object>

set_output_to(output_to: Union[Path, str]) → None

Adjusts the file system settings in the present object such that all outputs are written to the given folder.

Parameters:

output_to – The absolute path to a folder that should contain the outputs.

property visualization_folder: Path

Gets the full path in which the visualizations notebooks should be saved during training.

class InnerEye.ML.deep_learning_config.TemperatureScalingConfig(**params)

High level config to encapsulate temperature scaling parameters

ece_num_bins: int = 15

lr: float = 0.002

max_iter: int = 50

name = 'TemperatureScalingConfig'

param = <param.parameterized.Parameters object>

class InnerEye.ML.deep_learning_config.TrainerParams(**params)

autosave_every_n_val_epochs: int = 1

detect_anomaly: bool = False

max_num_gpus: int = -1

monitor_gpu: bool = False

monitor_loading: bool = False

name = 'TrainerParams'

num_epochs: int = 100

num_gpus_per_node() → int

Computes the number of gpus to use for each node: either the number of gpus available on the device or restrict it to max_num_gpu, whichever is smaller. Returns 0 if running on a CPU device.

param = <param.parameterized.Parameters object>

pl_check_val_every_n_epoch: int = 1

pl_deterministic: bool = False

pl_find_unused_parameters: bool = False

pl_limit_train_batches: Optional[int] = None

pl_limit_val_batches: Optional[int] = None

pl_num_sanity_val_steps: int = 0

pl_profiler: Optional[str] = None

pl_progress_bar_refresh_rate: Optional[int] = None

property use_gpu: bool

Returns True if a GPU is available, and the self.max_num_gpus flag allows it to be used. Returns False otherwise (i.e., if there is no GPU available, or self.max_num_gpus==0)

use_mixed_precision: bool = False

class InnerEye.ML.deep_learning_config.WorkflowParams(**params)

This class contains all parameters that affect how the whole training and testing workflow is executed.

cross_validation_split_index: int = -1

ensemble_inference_on_test_set: Optional[bool] = None

ensemble_inference_on_train_set: Optional[bool] = None

ensemble_inference_on_val_set: Optional[bool] = None

generate_report: bool = True

get_effective_random_seed() → int

Returns the random seed set as part of this configuration. If the configuration corresponds to a cross validation split, then the cross validation fold index will be added to the set random seed in order to return the effective random seed. :return:

inference_on_test_set: Optional[bool] = None

inference_on_train_set: Optional[bool] = None

inference_on_val_set: Optional[bool] = None

is_inference_required(model_proc: ModelProcessing, data_split: ModelExecutionMode) → bool

Returns True if inference is required for this model_proc (single or ensemble) and data_split (Train/Val/Test).

Parameters:

• model_proc – Whether we are testing an ensemble or single model.

• data_split – Indicates which of the 3 sets (training, test, or validation) is being processed.

Returns:

True if inference required.

property is_offline_run: bool

Returns True if the run is executing outside AzureML, or False if inside AzureML.

local_weights_path: List[Path] = []

model_id: str = ''

monitoring_interval_seconds: int = 0

multiprocessing_start_method: MultiprocessingStartMethod = 'fork'

name = 'WorkflowParams'

number_of_cross_validation_splits: int = 0

param = <param.parameterized.Parameters object>

property perform_cross_validation: bool

True if cross validation will be be performed as part of the training procedure. :return:

pretraining_run_recovery_id: str = None

random_seed: int = 42

regression_test_csv_tolerance: float = 0.0

regression_test_folder: Optional[Path] = None

validate() → None

weights_url: List[str] = []

InnerEye.ML.deep_learning_config.load_checkpoint(path_to_checkpoint: Path, use_gpu: bool = True) → Dict[str, Any]

Loads a Torch checkpoint from the given file. If use_gpu==False, map all parameters to the GPU, otherwise left the device of all parameters unchanged.

class InnerEye.ML.scalar_config.AggregationType(value)

The type of global pooling aggregation to use between the encoder and the classifier.

Average = 'Average'

GatedPooling = 'Gated'

MaxPooling = 'Max'

MixPooling = 'Mix'

ZAdaptive3dAvg = 'Adaptive3dAverage'

class InnerEye.ML.scalar_config.EnsembleAggregationType(value)

An enumeration.

Average = 'Average'

class InnerEye.ML.scalar_config.LabelTransformation(value)

Defines all possible transformation to apply to the labels.

static difference(labels: List) → Any

Assumes two labels channels [chan_1, chan_2] as input, the returned labels is the difference chan_2 - chan_1.

static get_scaling_transform(max_value: int = 100, min_value: int = 0, last_in_pipeline: bool = True) → Callable

Defines the function to scale labels.

Parameters:

• max_value –

• min_value –

• last_in_pipeline – if the transformation is the last in the pipeline it should expect a single label as an argument.

Else if returns a list of scaled labels for further transforms. :return: The scaling function

static identity(labels: List) → Any

No changes applied, checks that only one channel has been provided.

class InnerEye.ML.scalar_config.ScalarLoss(value)

An enumeration.

BinaryCrossEntropyWithLogits = 'BinaryCrossEntropyWithLogits'

CustomClassification = 'CustomClassification'

CustomRegression = 'CustomRegression'

MeanSquaredError = 'MeanSquaredError'

WeightedCrossEntropyWithLogits = 'WeightedCrossEntropyWithLogits'

is_classification_loss() → bool

is_regression_loss() → bool

class InnerEye.ML.scalar_config.ScalarModelBase(num_dataset_reader_workers: int = 0, **params: Any)

add_differences_for_features: List[str] = []

aggregation_type: AggregationType = 'Average'

categorical_columns: List[str] = []

categorical_feature_encoder: Optional[OneHotEncoderBase] = None

center_crop_size: Optional[Tuple[int, int, int]] = None

channel_column: str = 'channel'

class_names: List[str] = ['Default']

compute_and_log_metrics(logits: Tensor, targets: Tensor, subject_ids: List[str], is_training: bool, metrics: ModuleDict, logger: DataframeLogger, current_epoch: int, data_split: ModelExecutionMode) → None

Computes all the metrics for a given (logits, labels) pair, and writes them to the loggers.

Parameters:

• logits – The model output before normalization.

• targets – The expected model outputs.

• subject_ids – The subject IDs for the present minibatch.

• is_training – If True, write the metrics as training metrics, otherwise as validation metrics.

• metrics – A dictionary mapping from names of prediction targets to a list of metric computers, as returned by create_metric_computers.

• logger – An object of type DataframeLogger which can be be used for logging within this function.

• current_epoch – Current epoch number.

• data_split – ModelExecutionMode object indicating if this is the train or validation split.

create_and_set_torch_datasets(for_training: bool = True, for_inference: bool = True) → None

Creates and sets torch datasets for all model execution modes, and stores them in the self._datasets field. It also calls the hook to compute statistics for the train/val/test datasets.

create_metric_computers() → ModuleDict

Gets a set of objects that compute all the metrics for the type of model that is being trained, across all prediction targets (sequence positions when using a sequence model). :return: A dictionary mapping from names of prediction targets to a list of metric computers.

create_model() → Any

Creates a PyTorch model from the settings stored in the present object. This is an abstract method that each model class (segmentation, regression) should override. Return type is LightningModule, not Any - but we want to avoid importing torch at this point.

create_torch_datasets(dataset_splits: DatasetSplits) → Dict[ModelExecutionMode, Any]

dataset_stats_hook: Optional[Callable[[Dict[ModelExecutionMode, Any]], None]] = None

ensemble_aggregation_type: EnsembleAggregationType = 'Average'

expected_column_values: List[Tuple[str, str]] = None

filter_dataframe(df: DataFrame) → DataFrame

Filter dataframes based on expected values on columns

Parameters:

df – the input dataframe

Returns:

the filtered dataframe

get_image_transform() → ModelTransformsPerExecutionMode

Get transforms to apply to images for each model execution mode. By default only no transformation is performed.

get_label_transform() → Union[Callable, List[Callable]]

Return a transformation or list of transformation to apply to the labels.

get_loss_function() → Callable

Returns a custom loss function to be used with ScalarLoss.CustomClassification or CustomRegression.

get_model_train_test_dataset_splits(dataset_df: DataFrame) → DatasetSplits

Computes the training, validation and test splits for the model, from a dataframe that contains the full dataset.

Parameters:

dataset_df – A dataframe that contains the full dataset that the model is using.

Returns:

An instance of DatasetSplits with dataframes for training, validation and testing.

get_non_image_feature_channels_dict() → Dict

Convert the provided non_image_features_channels from List to Dictionary mapping each feature to its channels. As well as converting default key to each not defined features. Making it a property to avoid doing this conversion several time throughout the code.

get_parameter_search_hyperdrive_config(run_config: ScriptRunConfig) → HyperDriveConfig

Returns a configuration for AzureML Hyperdrive that should be used when running hyperparameter tuning. This is an abstract method that each specific model should override.

Parameters:

run_config – The AzureML estimator object that runs model training.

Returns:

A hyperdrive configuration object.

get_post_loss_logits_normalization_function() → Callable

Post loss normalization function to apply to the logits produced by the model. :return:

get_scalar_item_transform() → ModelTransformsPerExecutionMode

get_segmentation_transform() → ModelTransformsPerExecutionMode

Get transforms to apply on segmentations maps inputs for each model execution mode. By default only no transformation is performed.

get_total_number_of_categorical_non_imaging_features() → int

Returns the total number of categorical non imaging features expected in the input eg for the categorical channels A and B the total number would be: 2 ( feature channels A and B) * 4 (which is the number of bits required to one-hot encode a single channel) A| True, No => [1, 0, 0, 1] B| False, Yes => [0, 1, 1, 0]

get_total_number_of_non_imaging_features() → int

Returns the total number of non imaging features expected in the input

get_total_number_of_numerical_non_imaging_features() → int

Returns the total number of numerical non imaging features expected in the input

get_total_number_of_training_samples() → int

get_training_class_counts() → Dict

image_channels: List[str] = []

image_file_column: Optional[str] = None

image_size: Optional[Tuple[int, int, int]] = None

property is_classification_model: bool

Returns whether the model is a classification model

property is_non_imaging_model: bool

Returns whether the model uses non image features only

property is_regression_model: bool

Returns whether the model is a regression model

label_channels: Optional[List[str]] = None

label_value_column: str = ''

load_segmentation: bool = False

loss_type: ScalarLoss = 'BinaryCrossEntropyWithLogits'

name = 'ScalarModelBase'

non_image_feature_channels: Union[List[str], Dict[str, List[str]]] = None

num_dataset_reader_workers: int = 0

numerical_columns: List[str] = []

param = <param.parameterized.Parameters object>

pre_process_dataset_dataframe() → None

Performs any dataframe pre-processing functions, default is identity :return:

read_dataset_into_dataframe_and_pre_process() → None

Loads a dataset from a file or other source, and saves it into the model’s data_frame property. Applying any pre-processing functions defined in pre_process_dataset_dataframe The data frame should contain all of the training, test, and validation data.

should_generate_multilabel_report() → bool

Determines whether to produce a multilabel report. Override this to implement custom behaviour.

subject_column: str = 'subject'

target_names: List[str] = None

traverse_dirs_when_loading: bool = False

validate() → None

Validates the parameters stored in the present object.

InnerEye.ML.scalar_config.get_non_image_features_dict(default_channels: List[str], specific_channels: Optional[Dict[str, List[str]]] = None) → Dict[str, List[str]]

Returns the channels dictionary for non-imaging features.

Parameters:

• default_channels – the channels to use for all features except the features specified in specific_channels

• specific_channels – a dictionary mapping feature names to channels for all features that do not use the default channels

class InnerEye.ML.model_inference_config.ModelInferenceConfig(model_name: str, checkpoint_paths: List[str], model_configs_namespace: str = '')

Class for configuring a model for inference

checkpoint_paths: List[str]

classmethod from_dict(kvs: Optional[Union[dict, list, str, int, float, bool]], *, infer_missing=False) → A

classmethod from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) → A

model_configs_namespace: str = ''

model_name: str

classmethod schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) → SchemaF[A]

to_dict(encode_json=False) → Dict[str, Optional[Union[dict, list, str, int, float, bool]]]

to_json(*, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Optional[Union[int, str]] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) → str

InnerEye.ML.model_inference_config.read_model_inference_config(path_to_model_inference_config: Path) → ModelInferenceConfig

Read the model inference configuration from a json file, and instantiate a ModelInferenceConfig object using this.