Utils

Data Utils

class InnerEye.ML.utils.csv_util.OutlierType(value)

An enumeration.

HIGH = 'High'

LOW = 'Low'

InnerEye.ML.utils.csv_util.drop_rows_missing_important_values(df: DataFrame, important_cols: List[str]) → DataFrame

Remove rows from the DataFrame in which the columns that have been specified by the user as “important” contain null values or only whitespace.

Parameters:

• df – DataFrame

• important_cols – Columns which must not contain null values

Returns:

df: DataFrame without the dropped rows.

InnerEye.ML.utils.csv_util.extract_outliers(df: DataFrame, outlier_range: float, outlier_col: str = 'Dice', outlier_type: OutlierType = OutlierType.LOW) → DataFrame

Given a DataFrame, extract the subset in which a given value (specified by outlier_col) falls outside of mean +- outlier_range * std.

Parameters:

• df – DataFrame from which to extract the outliers

• outlier_range – The number of standard deviation from the mean which the points have to be apart to be considered an outlier i.e. a point is considered an outlier if its outlier_col value is above

mean + outlier_range * std (if outlier_type is HIGH) or below mean - outlier_range * std (if outlier_type is LOW).

Parameters:

• outlier_col – The column from which to calculate outliers, e.g. Dice

• outlier_type – Either LOW (i.e. below accepted range) or HIGH (above accepted range) outliers.

Returns:

DataFrame containing only the outliers

InnerEye.ML.utils.csv_util.get_worst_performing_outliers(df: DataFrame, outlier_range: float, outlier_col_name: str = 'Dice', max_n_outliers: Optional[int] = None) → List[Tuple[int, str, float, str]]

Returns a sorted list (worst to best) of all the worst performing outliers in the metrics table according to metric provided by outlier_col_name

Parameters:

• df – Metrics DataFrame

• outlier_col_name – The column by which to determine outliers

• outlier_range – The standard deviation from the mean which the points have to be below to be considered an outlier.

• max_n_outliers – the number of (worst performing) outlier IDs to return.

Returns:

a sorted list (worst to best) of all the worst performing outliers

InnerEye.ML.utils.csv_util.load_csv(csv_path: Path, expected_cols: List[str], col_type_converters: Optional[Dict[str, Any]] = None) → DataFrame

Load a pandas dataframe from a csv. If the columns do not contain at least expected_cols, an exception is raised

Parameters:

• csv_path – Path to file

• expected_cols – A list of the columns which must, as a minimum, be present.

• col_type_converters – Dictionary of column: type, which ensures certain DataFrame columns are parsed with specific types

Returns:

Loaded pandas DataFrame

InnerEye.ML.utils.csv_util.mark_outliers(df: DataFrame, outlier_range: float, outlier_col: str, high_values_are_good: bool) → DataFrame

Given a DataFrame, add a column “is_outlier” that contains “Yes” for all rows that are considered outliers. Rows that are not considered outliers have an empty string in the new column. Outliers are taken from the column outlier_col, that have a value that falls outside of mean +- outlier_range * std.

Parameters:

• df – DataFrame from which to extract the outliers

• outlier_range – The number of standard deviation from the mean which the points have to be apart to be considered an outlier i.e. a point is considered an outlier if its outlier_col value is above

mean + outlier_range * std (if outlier_type is HIGH) or below mean - outlier_range * std (if outlier_type is LOW).

Parameters:

• outlier_col – The column from which to calculate outliers, e.g. Dice

• high_values_are_good – If True, high values for the metric are considered good, and hence low values are marked as outliers. If False, low values are considered good, and high values are marked as outliers.

Returns:

DataFrame with an additional column is_outlier

class InnerEye.ML.utils.dataset_util.CategoricalToOneHotEncoder(columns_and_possible_categories: OrderedDict[str, List[str]])

Encoder to handle conversion to one-hot encoding for categorical data.

static create_from_dataframe(dataframe: DataFrame, columns: List[str]) → CategoricalToOneHotEncoder

Create an encoder that handles the conversion of the provided columns from a dataframe.

Parameters:

• dataframe – Dataframe to create the encoder from.

• columns – Supported columns for this encoder.

Returns:

CategoricalToOneHotEncoder that handles the conversion of the columns.

encode(x: Dict[str, List[str]]) → Tensor

Encode a dictonary mapping features to a list of values (one per channel). The values are expected to be string or NaN (if missing).

Example for features “A” and “B” A| True, False B| False, True => {“A”: [‘True’, ‘False’], “B”: [‘False’, ‘True’]} => [1, 0, 0, 1, 0, 1, 1, 0]

In the case of missing values: A| True, False B| False, => {“A”: [‘True’, ‘False’], “B”: [‘False’, nan]} => [1, 0, 0, 1, 0, 1, nan, nan]

Parameters:

x – A dictonary mapping features to their categorical values (one value per channel).

Returns:

A one-hot encoded Tensor of shape: [total feature length,]

get_feature_length(feature_name: str) → int

The expected length of the one-hot encoded feature vector for a given feature. For example, a feature that takes 3 values, will be encoded as a one-hot vector of length 3.

Parameters:

feature_name – the name of the column for which to compute the feature length.

Returns:

the feature length i.e. number of possible values for this feature.

get_supported_dataset_column_names() → List[str]

Returns:

list of categorical columns that are supported by this encoder

class InnerEye.ML.utils.dataset_util.DatasetExample(epoch: int, patient_id: int, header: ImageHeader, image: ndarray, prediction: ndarray, labels: ndarray)

Dataset sample with predictions after being passed through a model.

epoch: int

header: ImageHeader

image: ndarray

labels: ndarray

patient_id: int

prediction: ndarray

InnerEye.ML.utils.dataset_util.add_label_stats_to_dataframe(input_dataframe: DataFrame, dataset_root_directory: Path, target_label_names: List[str]) → DataFrame

Loops through all available subject IDs, generates ground-truth label statistics and updates input dataframe with the computed stats by adding new columns. In particular, it checks the overlapping regions between different structures and volume of labels.

Parameters:

• input_dataframe – Input Pandas dataframe object containing subjectIds and label names

• dataset_root_directory – Path to dataset root directory

• target_label_names – A list of label names that are used in label stat computations

InnerEye.ML.utils.dataset_util.store_and_upload_example(dataset_example: DatasetExample, segmentation_config: SegmentationModelBase) → None

Stores an example input and output of the network to Nifti files.

Parameters:

• dataset_example – The dataset example, with image, label and prediction, that should be written.

• segmentation_config – configuration information to be used for normalization and example_images_folder

class InnerEye.ML.utils.hdf5_util.HDF5Field(value)

An enumeration.

DATE = 'acquisition_date'

PATIENT_ID = 'id'

SEGMENTATION = 'segmentation'

VOLUME = 'volume'

class InnerEye.ML.utils.hdf5_util.HDF5ImageDataType(value)

Data type of medical image data (e.g. masks and labels) Segmentation label maps (LABEL) are one-hot encoded.

IMAGE = <class 'numpy.float32'>

MASK = <class 'numpy.uint8'>

QUANTITY = <class 'numpy.float64'>

SEGMENTATION = <class 'numpy.float32'>

THICKNESS = <class 'numpy.float32'>

VESSELS = <class 'numpy.float32'>

class InnerEye.ML.utils.hdf5_util.HDF5Object(patient_id: str, volume: ndarray, acquisition_date: Union[str, datetime], segmentation: Optional[ndarray])

An HDF5 file. Each of volume (images), segmentation (labels), acquisition date and patient ID must be provided

classmethod from_file(hdf5_path: Path, load_segmentation: bool) → T

Load HDF5 object from file

Parameters:

• hdf5_path – Path to an HDF5 file

• load_segmentation – If True it loads segmentation (if present on the same file as the image).

Returns:

HDF5 object

static parse_acquisition_date(date: str) → Optional[datetime]

Converts a string representing a date to a datetime object

Parameters:

date – string representing a date

Returns:

converted date, None if the string is invalid for date conversion.

class InnerEye.ML.utils.image_util.ImageDataType(value)

Data type for medical image data (e.g. masks and labels) Segmentation maps are one-hot encoded.

CLASSIFICATION_LABEL = <class 'numpy.float32'>

IMAGE = <class 'numpy.float32'>

MASK = <class 'numpy.uint8'>

SEGMENTATION = <class 'numpy.float32'>

class InnerEye.ML.utils.image_util.ImageHeader(spacing: Tuple[float, float, float], origin: Tuple[float, float, float], direction: Tuple[float, float, float, float, float, float, float, float, float])

A 3D image header

direction: Tuple[float, float, float, float, float, float, float, float, float]

origin: Tuple[float, float, float]

spacing: Tuple[float, float, float]

InnerEye.ML.utils.image_util.apply_mask_to_posteriors(posteriors: Union[ndarray, Tensor], mask: Union[ndarray, Tensor]) → Union[ndarray, Tensor]

Apply a binary mask to the provided posteriors such that for all voxels outside of the mask:

1. The background class posterior (index == 0) is set to 1.

2. All other classes posteriors are set to 0.

Parameters:

• posteriors – image tensors in shape: Batches (optional) x Classes x Z x Y x X

• mask – image tensor in shape: Batches (optional) x Z x Y x X

Returns:

posteriors with mask applied

InnerEye.ML.utils.image_util.apply_slice_exclusion_rules(model_config: SegmentationModelBase, segmentation: ndarray) → ndarray

Applies each slice exclusion rule to segmentation, modifying it in place.

InnerEye.ML.utils.image_util.apply_summed_probability_rules(model_config: SegmentationModelBase, posteriors: Union[ndarray, Tensor], segmentation: Union[ndarray, Tensor]) → Union[ndarray, Tensor]

Applies summed probability rules to segmentation, modifying it in place.

Parameters:

• model_config – Model configuration information

• posteriors – Confidences per voxel per class, in format Batch x Classes x Z x Y x X if batched, or Classes x Z x Y x X if not batched.

• segmentation – Class labels per voxel, in format Batch x Z x Y x X if batched, or Z x Y x X if not batched.

Returns:

Modified segmentation, as Batch x Z x Y x X if batched, or Z x Y x X if not batched.

InnerEye.ML.utils.image_util.binaries_from_multi_label_array(array: ndarray, num_classes_including_background: int) → Iterator[ndarray]

Given multimap array containing C classes, yields an iterator with C elements where each item is a binary array of the same shape as the original multimap array. For each binary array, the value is 1 in positions where the value of the multimap array is c, and 0 elsewhere.

InnerEye.ML.utils.image_util.check_array_range(data: ndarray, expected_range: Optional[Tuple[Union[int, float], Union[int, float]]] = None, error_prefix: Optional[str] = None) → None

Checks if all values in the given array fall into the expected range. If not, raises a ValueError, and prints out statistics about the values that fell outside the expected range. If no range is provided, it checks that all values in the array are finite (that is, they are not infinity and not np.nan).

Parameters:

• data – The array to check. It can have any size.

• expected_range – The interval that all array elements must fall into. The first entry is the lower bound, the second entry is the upper bound.

• error_prefix – A string to use as the prefix for the error message.

InnerEye.ML.utils.image_util.check_if_posterior_array(posteriors: ndarray) → None

Checks if the provided input is a valid posteriors array, raises ValueError otherwise

InnerEye.ML.utils.image_util.compute_uncertainty_map_from_posteriors(posteriors: ndarray) → ndarray

Compute voxel wise uncertainty from a given posterior input using Normalized Shannon Entropy: https://en.wiktionary.org/wiki/Shannon_entropy

Parameters:

posteriors – Normalized probability distribution in range [0, 1] for each class, in shape: Class x Z x Y x X

Returns:

Shannon Entropy for each voxel, shape: Z x Y x X expected range is [0,1] where 1 represents low confidence or uniform posterior distribution across classes.

InnerEye.ML.utils.image_util.extract_largest_foreground_connected_component(multi_label_array: ndarray, restrictions: Optional[List[Tuple[int, Optional[float]]]] = None) → ndarray

Extracts the largest foreground connected component per class from a multi-label array.

Parameters:

• multi_label_array – An array of class assignments, i.e. value c at (z, y, x) is a class c.

• restrictions – restrict processing to a subset of the classes (if provided). Each element is a pair (class_index, threshold) where threshold may be None.

Returns:

An array of class assignments

InnerEye.ML.utils.image_util.find_intersection_array_indices(indices1: Union[ndarray, Tuple[ndarray, ...]], indices2: Union[ndarray, Tuple[ndarray, ...]], shape: Union[ndarray, Tuple[int, ...]]) → Tuple[ndarray, ...]

Finds the intersection of two sets of indices for multidimensional arrays

Parameters:

• indices1 – Tuple with n arrays, each containing indices in the ith dimension, as returned by np.where()

• indices2 – Tuple with n arrays, each containing indices in the ith dimension, as returned by np.where()

• shape – The shape of the array that was indexed originally

Returns:

Tuple with n arrays, each containing indices in the ith dimension

InnerEye.ML.utils.image_util.gaussian_smooth_posteriors(posteriors: ndarray, kernel_size_mm: Tuple[float, float, float], voxel_spacing_mm: Tuple[float, float, float]) → ndarray

Performs Gaussian smoothing on posteriors

Parameters:

• posteriors – Normalized probability distribution in range [0, 1] for each class, in shape: Class x Z x Y x X

• kernel_size_mm – The size of the smoothing kernel in mm to be used in each dimension (Z, Y, X)

• voxel_spacing_mm – Voxel spacing to use to map from mm space to pixel space for the Gaussian sigma parameter for each dimension in (Z x Y x X) order.

Returns:

InnerEye.ML.utils.image_util.get_center_crop(image: Union[ndarray, Tensor], crop_shape: Tuple[int, int, int]) → Union[ndarray, Tensor]

Extracts the center region specified by the crop_shape argument from the input image

Parameters:

• image – The original image to extract crop from

• crop_shape – The shape of the center crop to extract

Returns:

the center region as specified by the crop_shape argument.

InnerEye.ML.utils.image_util.get_class_weights(target: Tensor, class_weight_power: float = 1.0) → Tensor

Returns class weights inversely proportional to some power of the number of pixels in each class.

Parameters:

• target – one-hot tensor of shape (B, C, Z, X, Y); thus class dimension (of size C) is dimension 1

• class_weight_power – power to raise 1/c to, for each class count c

InnerEye.ML.utils.image_util.get_largest_z_slice(mask: ndarray) → int

Gets the Z position of the given 3D image that has the largest number of non-zero entries across the whole XY plane. If there are multiple Z positions that attain the maximum, return the lowest one.

Parameters:

mask – A 3D image in Z x Y x X order.

Returns:

The Z index that has the largest number of non-zero elements across mask[z,:,:]

InnerEye.ML.utils.image_util.get_unit_image_header(spacing: Optional[Tuple[float, float, float]] = None) → ImageHeader

Creates an ImageHeader object with the origin at 0, and unit direction. The spacing is set to the argument, defaulting to (1, 1, 1) if not provided.

Parameters:

spacing – The image spacing, as a (Z, Y, X) tuple.

InnerEye.ML.utils.image_util.is_binary_array(array: ndarray) → bool

Checks to see if the array passed has only binary values.

Parameters:

array – the np.ndarray to check

Returns:

True if the Array is binary or False otherwise

InnerEye.ML.utils.image_util.largest_connected_components(img: ndarray, deletion_limit: Optional[float], class_index: Optional[int] = None) → ndarray

Select the largest connected binary components (plural) in an image. If deletion_limit is set in which case a component is only deleted (i.e. its voxels are False in the output) if its voxel count as a proportion of all the True voxels in the input is less than deletion_limit.

Parameters:

• img – np.ndarray

• deletion_limit – if set, a component is deleted only if its voxel count as a proportion of all the True voxels in the input is less than deletion_limit.

• class_index – Optional. Can be used to provide a class index for logging purposes if the image contains only pixels from a specific class.

InnerEye.ML.utils.image_util.merge_masks(masks: ndarray) → ndarray

Merges a one-hot encoded mask tensor (Classes x Z x Y x X) into a multi-label map with labels corresponding to their index in the original tensor of shape (Z x Y x X).

Parameters:

masks – array of shape (Classes x Z x Y x X) containing the mask for each class

Returns:

merged_mask of shape (Z x Y x X).

InnerEye.ML.utils.image_util.multi_label_array_to_binary(array: ndarray, num_classes_including_background: int) → ndarray

Converts a multimap array into a array of binary masks for each class. If the number of classes is 2, the result will contain a binary mask for all entries in the original array where the value was 0, and a binary mask for the entries that were 1.

Parameters:

• array – An array of class assignments.

• num_classes_including_background – The number of class assignments to search for. If 3 classes, the class assignments to search for will be 0, 1, and 2.

Returns:

an array of size (num_classes_including_background, array.shape)

InnerEye.ML.utils.image_util.one_hot_to_class_indices(labels: Tensor) → Tensor

Converts one hot encoded label tensor to a tensor representing class ids.

Parameters:

labels – One-hot encoded label tensor

InnerEye.ML.utils.image_util.pad_images(images: ndarray, output_size: Optional[Tuple[int, int, int]], padding_mode: PaddingMode = PaddingMode.Zero) → ndarray

Pad the original images such that their shape after padding is equal to a fixed output_size, using the provided padding mode.

Parameters:

• images – the image(s) to be padded, in shape: Z x Y x X or batched in shape: Batches x Z x Y x X.

• output_size – the target output shape after padding.

• padding_mode – a valid numpy padding mode

Returns:

padded copy of the original image.

InnerEye.ML.utils.image_util.pad_images_for_inference(images: ndarray, crop_size: Tuple[int, int, int], output_size: Optional[Tuple[int, int, int]], padding_mode: PaddingMode = PaddingMode.Zero) → ndarray

Pad the original image to ensure that the size of the model output as the original image. Padding is needed to allow the patches on the corners of the image to be handled correctly, as the model response for each patch will only cover the center of the input voxels for that patch. Hence, add a padding of size ceil(output_size - crop_size / 2) around the original image is needed to ensure that the output size of the model is the same as the original image size.

Parameters:

• images – the image(s) to be padded, in shape: Z x Y x X or batched in shape: Batches x Z x Y x X.

• crop_size – the shape of the patches that will be taken from this image.

• output_size – the shape of the response for each patch from the model.

• padding_mode – a valid numpy padding mode.

Returns:

padded copy of the original image.

InnerEye.ML.utils.image_util.posteriors_to_segmentation(posteriors: Union[ndarray, Tensor]) → Union[ndarray, Tensor]

Perform argmax on the class dimension.

Parameters:

posteriors – Confidence maps [0,1] for each patch per class in format: Batches x Class x Z x Y x X or Class x Z x Y x X for non-batched input

Returns:

segmentation: argmaxed posteriors with each voxel belonging to a single class: Batches x Z x Y x X or Z x Y x X for non-batched input

InnerEye.ML.utils.image_util.segmentation_to_one_hot(segmentation: Tensor, use_gpu: bool, result_dtype: dtype) → Tensor

Converts a tensor that contains a segmentation multi-label map to one-hot encoding, running the time-consuming operations on the GPU if the use_gpu flag is True. The code assumes that there are no more than HDF5_NUM_SEGMENTATION_CLASSES distinct classes in the segmentation. For an input tensor of shape [B, C, Z, Y, X] with B batches, C image channels, the result will have size [B, C*HDF5_NUM_SEGMENTATION_CLASSES, Z, Y, X]

Parameters:

• segmentation – A segmentation as a multi-label map of shape [B, C, Z, Y, X]

• use_gpu – If true, and the input is not yet on the GPU, move the intermediate tensors to the GPU. The result will be on the same device as the argument segmentation

• result_dtype – The torch data type that the result tensor should have. This would be either float16 or float32

Returns:

A torch tensor with one-hot encoding of the segmentation of shape [B, C*HDF5_NUM_SEGMENTATION_CLASSES, Z, Y, X]

class InnerEye.ML.utils.io_util.DicomFileType(value)

Supported file extensions that indicate Dicom data.

Dicom = '.dcm'

class InnerEye.ML.utils.io_util.DicomTags(value)

An enumeration.

BitsAllocated = '0028|0100'

BitsStored = '0028|0101'

FrameOfReferenceUID = '0020|0052'

HighBit = '0028|0102'

ImagePositionPatient = '0020|0032'

ImageType = '0008|0008'

InstanceNumber = '0020|0013'

Modality = '0008|0060'

PatientPosition = '0018|5100'

PhotometricInterpretation = '0028|0004'

PixelRepresentation = '0028|0103'

RescaleIntercept = '0028|1052'

RescaleSlope = '0028|1053'

SeriesInstanceUID = '0020|000E'

StudyID = '0020|0010'

StudyInstanceUID = '0020|000D'

class InnerEye.ML.utils.io_util.HDF5FileType(value)

Supported file extensions that indicate HDF5 data.

HDF5 = '.h5'

HDF5_COMPRESSED_GZ = '.h5.gz'

HDF5_COMPRESSED_SZ = '.h5.sz'

HDF5_EXPLICIT = '.hdf5'

class InnerEye.ML.utils.io_util.ImageAndSegmentations(*args, **kwds)

images: TensorOrNumpyArray

segmentations: Optional[TensorOrNumpyArray] = None

class InnerEye.ML.utils.io_util.ImageWithHeader(image: ndarray, header: ImageHeader)

A 3D image with header

header: ImageHeader

image: ndarray

class InnerEye.ML.utils.io_util.MedicalImageFileType(value)

Supported types of medical image formats

NIFTI = '.nii'

NIFTI_COMPRESSED_GZ = '.nii.gz'

class InnerEye.ML.utils.io_util.NumpyFile(value)

Supported file extensions that indicate Numpy data.

NUMPY = '.npy'

NUMPY_COMPRESSED = '.npz'

class InnerEye.ML.utils.io_util.PhotometricInterpretation(value)

An enumeration.

MONOCHROME1 = 'MONOCHROME1'

MONOCHROME2 = 'MONOCHROME2'

InnerEye.ML.utils.io_util.create_dicom_series(folder: Path, size: Tuple[int, int, int], spacing: Tuple[float, float, float]) → ndarray

Create a random DICOM series and save as a set of files in folder.

Parameters:

• folder – Path to folder to save DICOM series.

• size – Final image size, as (#slices, #rows, #columns).

• spacing – Final image spacing, as (column spacing, row spacing, slice spacing) (in mm).

Returns:

The test data, a 3d ndarray of floats in the range [0, 1000.0).

InnerEye.ML.utils.io_util.is_dicom_file_path(file: Union[Path, str]) → bool

Returns true if the given file name appears to belong to a Dicom file. This is done based on extensions only. The file does not need to exist.

Parameters:

file – The file name to check.

Returns:

True if the file name indicates a Dicom file.

InnerEye.ML.utils.io_util.is_hdf5_file_path(file: Union[Path, str]) → bool

Returns true if the given file name appears to belong to a compressed or uncompressed HDF5 file. This is done based on extensions only. The file does not need to exist.

Parameters:

file – The file name to check.

Returns:

True if the file name indicates a HDF5 file.

InnerEye.ML.utils.io_util.is_nifti_file_path(file: Union[Path, str]) → bool

Returns true if the given file name appears to belong to a compressed or uncompressed Nifti file. This is done based on extensions only. The file does not need to exist.

Parameters:

file – The file name to check.

Returns:

True if the file name indicates a Nifti file.

InnerEye.ML.utils.io_util.is_numpy_file_path(file: Union[Path, str]) → bool

Returns true if the given file name appears to belong to a Numpy file. This is done based on extensions only. The file does not need to exist.

Parameters:

file – The file name to check.

Returns:

True if the file name indicates a Numpy file.

InnerEye.ML.utils.io_util.is_png(file: Union[Path, str]) → bool

Returns true if file is png

InnerEye.ML.utils.io_util.load_dicom_image(path: Union[Path, str]) → ndarray

Loads an array from a single dicom file.

Parameters:

path – The path to the dicom file.

InnerEye.ML.utils.io_util.load_dicom_series(folder: Path) → Image

Load a DICOM series into a 3d sitk image.

If the folder contains more than one series then the first will be loaded.

Parameters:

folder – Path to folder containing DICOM series.

Returns:

sitk.Image of the DICOM series.

InnerEye.ML.utils.io_util.load_dicom_series_and_save(folder: Path, file_name: Path) → None

Load a DICOM series into a 3d image and save as file_name.

If the folder contains more than one series then the first will be loaded. The file format type is determined by SimpleITK based on the file name’s suffix. List of supported file types is here: https://simpleitk.readthedocs.io/en/master/IO.html

Parameters:

• folder – Path to folder containing DICOM series.

• file_name – Path to save image.

InnerEye.ML.utils.io_util.load_hdf5_dataset_from_file(path_str: Path, dataset_name: str) → ndarray

Loads a hdf5 dataset from a file as an ndarray

Parameters:

• path_str – The path to the HDF5 file

• dataset_name – The dataset name in the HDF5 file that we want to load

Returns:

ndarray

InnerEye.ML.utils.io_util.load_hdf5_file(path_str: Union[str, Path], load_segmentation: bool = False) → HDF5Object

Loads a single HDF5 file.

Parameters:

• path_str – The path of the HDF5 file that should be loaded.

• load_segmentation – If True, the segmentation field of the result object will be populated. If False, the field will be set to None.

Returns:

HDF5Object

InnerEye.ML.utils.io_util.load_image(path: ~typing.Union[~pathlib.Path, str], image_type: ~typing.Optional[~typing.Type] = <class 'float'>) → ImageWithHeader

Loads an image with extension numpy or nifti or png For HDF5 path suffix

For images |<dataset_name>|<channel index> For segmentation binary |<dataset_name>|<channel index> For segmentation multimap |<dataset_name>|<channel index>|<multimap value> The expected dimensions to be (channel, Z, Y, X)

Parameters:

• path – The path to the file

• image_type – The type of the image

InnerEye.ML.utils.io_util.load_image_in_known_formats(file: Path, load_segmentation: bool) → ImageAndSegmentations[ndarray]

Loads an image from a file in the given path. At the moment, this supports Nifti, HDF5, numpy and dicom files.

Parameters:

• file – The path of the file to load.

• load_segmentation – If True it loads segmentation if present on the same file as the image.

Returns:

a wrapper class that contains the images and segmentation if present

InnerEye.ML.utils.io_util.load_images_and_stack(files: Iterable[Path], load_segmentation: bool, center_crop_size: Optional[Tuple[int, int, int]] = None, image_size: Optional[Tuple[int, int, int]] = None) → ImageAndSegmentations[Tensor]

Attempts to load a set of files, all of which are expected to contain 3D images of the same size (Z, X, Y) They are all stacked along dimension 0 and returned as a torch tensor of size (B, Z, X, Y) Images are returned as torch.float32 tensors, segmentations are returned as torch.uint8 tensors (multimaps).

Parameters:

• files – The paths of the files to load.

• load_segmentation – If True it loads segmentation if present on the same file as the image. This is only supported for loading from HDF5 files.

• center_crop_size – If supplied, all loaded images will be cropped to the size given here. The crop will be taken from the center of the image.

• image_size – If supplied, all loaded images will be resized immediately after loading.

Returns:

A wrapper class that contains the loaded images, and if load_segmentation is True, also the segmentations that were present in the files.

InnerEye.ML.utils.io_util.load_images_from_dataset_source(dataset_source: PatientDatasetSource, check_exclusive: bool = True) → Sample

Load images. ground truth labels and masks from the provided dataset source. With an inferred label class for the background (assumed to be not provided in the input)

Parameters:

• dataset_source – The dataset source for which channels are to be loaded into memory.

• check_exclusive – Check that the labels are mutually exclusive (defaults to True)

Returns:

a Sample object with the loaded volume (image), labels, mask and metadata.

InnerEye.ML.utils.io_util.load_labels_from_dataset_source(dataset_source: PatientDatasetSource, check_exclusive: bool = True, image_size: Optional[Tuple[int]] = None) → ndarray

Load labels containing segmentation binary labels in one-hot-encoding. In the future, this function will be used to load global class and non-imaging information as well.

Parameters:

• dataset_source – The dataset source for which channels are to be loaded into memory.

• check_exclusive – Check that the labels are mutually exclusive (defaults to True).

Returns:

A label sample object containing ground-truth information.

InnerEye.ML.utils.io_util.load_nifti_image(path: ~typing.Union[~pathlib.Path, str], image_type: ~typing.Optional[~typing.Type] = <class 'float'>) → ImageWithHeader

Loads a single .nii, or .nii.gz image from disk. The image to load must be 3D.

Parameters:

• path – The path to the image to load.

• image_type – The type to load the image in, set to None to not cast, default is float

Returns:

A numpy array of the image and header data if applicable.

Raises:

ValueError – If the path is invalid or the image is not 3D.

InnerEye.ML.utils.io_util.load_numpy_image(path: Union[Path, str], image_type: Optional[Type] = None) → ndarray

Loads an array from a numpy file (npz or npy). The array is converted to image_type or untouched if None

Parameters:

• path – The path to the numpy file.

• image_type – The dtype to cast the array

Returns:

ndarray

InnerEye.ML.utils.io_util.read_image_as_array_with_header(file_path: Path) → Tuple[ndarray, ImageHeader]

Read image with simpleITK as a ndarray.

Parameters:

file_path –

Returns:

Tuple of ndarray with image in Z Y X and Spacing in Z X Y

InnerEye.ML.utils.io_util.reverse_tuple_float3(tuple: Tuple[float, float, float]) → Tuple[float, float, float]

Reverse a tuple of 3 floats.

Parameters:

tuple – of 3 floats

Returns:

a tuple of 3 floats reversed

InnerEye.ML.utils.io_util.save_lines_to_file(file: Path, values: List[str]) → None

Writes an array of lines into a file, one value per line. End of line character is hardcoded to be `

`.

If the file exists already, it will be deleted.

param file:

The path where to save the file

param values:

A list of strings

InnerEye.ML.utils.io_util.store_as_nifti(image: ndarray, header: ImageHeader, file_name: Union[Path, str], image_type: Union[str, type, dtype], scale: bool = False, input_range: Optional[Iterable[Union[int, float]]] = None, output_range: Optional[Iterable[Union[int, float]]] = None) → Path

Saves an image in nifti format (uploading to Azure also if an online Run), and performs the following operations: 1) transpose the image back into X,Y,Z from Z,Y,X 2) if scale is true, then performs a linear scaling to either the input/output range or byte range (0,255) as default. 3) cast the image values to the given type before saving

Parameters:

• image – 3D image in shape: Z x Y x X.

• header – The image header

• file_name – The name of the file for this image.

• scale – Should perform linear scaling of the image, to desired output range or byte range by default.

• input_range – The input range the image belongs to.

• output_range – The output range to scale the image to.

• image_type – The type to save the image in.

Returns:

the path to the saved image

InnerEye.ML.utils.io_util.store_as_scaled_ubyte_nifti(image: ndarray, header: ImageHeader, file_name: Union[Path, str], input_range: Union[Iterable[int], Iterable[float]]) → Path

Saves an image in nifti format as ubyte, and performs the following operations: 1) transpose the image back into X,Y,Z from Z,Y,X 2) perform linear scaling from input range to byte range 3) cast the image values to ubyte before saving

Parameters:

• image – 3D image in shape: Z x Y x X.

• header – The image header

• file_name – The name of the file for this image.

• input_range – The input range the image belongs to.

Returns:

the path to the saved image

InnerEye.ML.utils.io_util.store_as_ubyte_nifti(image: ndarray, header: ImageHeader, file_name: Union[Path, str]) → Path

Saves an image in nifti format as ubyte, and performs the following operations: 1) transpose the image back into X,Y,Z from Z,Y,X 2) cast the image values to ubyte before saving

Parameters:

• image – 3D image in shape: Z x Y x X.

• header – The image spacing Z x Y x X

• file_name – The name of the file for this image.

Returns:

the path to the saved image

InnerEye.ML.utils.io_util.store_binary_mask_as_nifti(image: ndarray, header: ImageHeader, file_name: Union[Path, str]) → Path

Saves a binary mask to nifti format, and performs the following operations: 1) Check that the image really only contains binary values (0 and 1) 2) transpose the image back into X,Y,Z from Z,Y,X 3) cast the image values to ubyte before saving

Parameters:

• image – binary 3D image in shape: Z x Y x X.

• header – The image header

• file_name – The name of the file for this image.

Returns:

the path to the saved image

Raises:

Exception – when image is not binary

InnerEye.ML.utils.io_util.store_image_as_short_nifti(image: ndarray, header: ImageHeader, file_name: Union[Path, str], args: Optional[SegmentationModelBase]) → Path

Saves an image in nifti format as ubyte, and performs the following operations: 1) transpose the image back into X,Y,Z from Z,Y,X 2) perform linear scaling from config.output_range to window level range, or byte range (0,255) if norm_method is not CT Window and scale is True 3) cast the image values to ubyte before saving

Parameters:

• image – 3D image in shape: Z x Y x X.

• header – ImageHeader

• file_name – The name of the file for this image.

• args – The model config.

Returns:

the path to the saved image

InnerEye.ML.utils.io_util.store_posteriors_as_nifti(image: ndarray, header: ImageHeader, file_name: Union[Path, str]) → Path

Saves an array of posteriors in nifti format as ubyte, and performs the following operations: 1) transpose the image back into X,Y,Z from Z,Y,X 2) perform a linear scaling from [0, 1] to byte range 3) cast the image values to ubyte before saving

Parameters:

• image – 3D image in shape: Z x Y x X.

• header – Image header for the image

• file_name – The name of the file for this image.

Returns:

the path to the saved image

InnerEye.ML.utils.io_util.tabulate_dataframe(df: DataFrame, pefix_newline: bool = True) → str

Helper function to print a pandas Dataframe in a nicely readable table.

InnerEye.ML.utils.io_util.zip_random_dicom_series(size: Tuple[int, int, int], spacing: Tuple[float, float, float], zip_file_path: Path, scratch_folder: Path) → None

Create a zipped random reference DICOM series.

Parameters:

• size – Final image size, as (#slices, #rows, #columns).

• spacing – Final image spacing, as (column spacing, row spacing, slice spacing) (in mm).

• zip_file_path – Target zip file.

• scratch_folder – Scratch folder.

class InnerEye.ML.utils.split_dataset.DatasetSplits(train: 'pd.DataFrame', val: 'pd.DataFrame', test: 'pd.DataFrame', subject_column: 'str' = 'subject', group_column: 'Optional[str]' = None, allow_empty: 'bool' = False)

allow_empty: bool = False

static from_groups(df: DataFrame, train_groups: Sequence[str], test_groups: Sequence[str], val_groups: Sequence[str], *, group_column: str, subject_column: str = 'subject') → DatasetSplits

Assuming a DataFrame with columns subject Takes a slice of values from each data split train/test/val for the provided groups.

Parameters:

• df – the input DataFrame

• train_groups – groups for training.

• test_groups – groups for testing.

• val_groups – groups for validation.

• subject_column – subject id column name

• group_column – grouping column name; if given, samples from each group will always be in the same subset (train, val, or test) and cross-validation fold.

Returns:

Data splits with respected dataset split ids.

static from_institutions(df: DataFrame, proportion_train: float, proportion_test: float, proportion_val: float, subject_column: str = 'subject', shuffle: bool = True, random_seed: int = 0, exclude_institutions: Optional[Iterable[str]] = None, institutions_for_test_only: Optional[Iterable[str]] = None, subject_ids_for_test_only: Optional[Iterable[str]] = None) → DatasetSplits

Assuming a DataFrame with columns subject and institutionId

Takes a slice of values from each institution based on the train/test/val proportions provided, such that for each institution there is at least one subject in each of the train/test/val splits.

This method for creating DatasetSplits does not currently support grouping.

Parameters:

• df – the input DataFrame

• proportion_train – Proportion of images per institution to be used for training.

• proportion_val – Proportion of images per institution to be used for validation.

• proportion_test – Proportion of images per institution to be used for testing.

• subject_column – Name of column containing subject id.

• shuffle – If True the subjects in the dataframe will be shuffle before performing splits.

• random_seed – Random seed to be used for shuffle 0 is default.

• exclude_institutions – If given, all subjects where institutionId has the given value will be excluded from train, test, and validation set.

• institutions_for_test_only – If given, all subjects where institutionId has the given value will be placed only in the test set.

• subject_ids_for_test_only – If given, all images with the provided subject Ids will be placed in the test set.

Returns:

Data splits with respected dataset split proportions per institution.

static from_proportions(df: DataFrame, proportion_train: float, proportion_test: float, proportion_val: float, *, subject_column: str = 'subject', group_column: Optional[str] = None, shuffle: bool = True, random_seed: int = 0) → DatasetSplits

Creates a split of a dataset into train, test, and validation set, according to fixed proportions using the “subject” column in the dataframe, or the group column, if given.

Parameters:

• df – The dataframe containing all subjects.

• proportion_train – proportion for the train set.

• proportion_test – proportion for the test set.

• subject_column – Subject id column name

• group_column – grouping column name; if given, samples from each group will always be in the same subset (train, val, or test) and cross-validation fold.

• proportion_val – proportion for the validation set.

• shuffle – If True the subjects in the dataframe will be shuffle before performing splits.

• random_seed – Random seed to be used for shuffle 0 is default.

Returns:

static from_subject_ids(df: DataFrame, train_ids: Sequence[str], test_ids: Sequence[str], val_ids: Sequence[str], *, subject_column: str = 'subject', group_column: Optional[str] = None) → DatasetSplits

Assuming a DataFrame with columns subject Takes a slice of values from each data split train/test/val for the provided ids.

Parameters:

• df – the input DataFrame

• train_ids – ids for training.

• test_ids – ids for testing.

• val_ids – ids for validation.

• subject_column – subject id column name

• group_column – grouping column name; if given, samples from each group will always be in the same subset (train, val, or test) and cross-validation fold.

Returns:

Data splits with respected dataset split ids.

static get_df_from_ids(df: DataFrame, ids: Sequence[str], subject_column: str = 'subject') → DataFrame

get_k_fold_cross_validation_splits(n_splits: int, random_seed: int = 0) → List[DatasetSplits]

Creates K folds from the Train + Val splits.

If a group_column has been specified, the folds will be split such that subjects in a group will not be separated. In this case, the splits are fully deterministic, and random_seed is ignored.

Parameters:

• n_splits – number of folds to perform.

• random_seed – random seed to be used for shuffle 0 is default.

Returns:

List of K dataset splits

static get_subject_ranges_for_splits(population: Sequence[str], proportion_train: float, proportion_test: float, proportion_val: float) → Dict[ModelExecutionMode, Set[str]]

Get mutually exclusive subject ranges for each dataset split (w.r.t to the proportion provided) ensuring all sets have at least one item in them when possible.

Parameters:

• population – all subjects

• proportion_train – proportion for the train set.

• proportion_test – proportion for the test set.

• proportion_val – proportion for the validation set.

Returns:

Train, Test, and Val splits

group_column: Optional[str] = None

number_of_subjects() → int

static parse_restriction_pattern(restriction_pattern: str) → Tuple[Optional[int], Optional[int], Optional[int]]

restrict_subjects(restriction_pattern: str) → DatasetSplits

Creates a new dataset split that has at most the specified numbers of subjects in train, validation and test sets respectively.

If group_column was specified, this operation may violate the grouping constraints and the resulting splits object will have group_column == None.

Parameters:

restriction_pattern – a string containing zero or two commas, and otherwise digits or “+”. An empty substring will result in no restriction for the corresponding dataset. Thus “20,,3” means “restrict to 20 training images and 3 test images, with no restriction on validation”. A “+” value means “reassign all images from the set(s) with a numeric count (there must be at least one) to this set”. Thus “,0,+” means “leave the training set alone, but move all validation images to the test set”, and “0,2,+” means “move all training images and all but 2 validation images to the test set”.

Returns:

A new dataset split object with (at most) the numbers of subjects specified by restrict_pattern

subject_column: str = 'subject'

test: DataFrame

train: DataFrame

unique_subjects() → Tuple[Any, Any, Any]

val: DataFrame

class InnerEye.ML.utils.features_util.FeatureStatistics(mean: Tensor, std: Tensor)

Class to store statistics (mean and standard deviation) of a set of features in a given dataset. Allows to perform feature standardization for this set of features.

static compute_masked_statistics(input: Tensor, mask: Tensor, apply_bias_correction: bool = True) → FeatureStatistics

If the input features contains invalid values (e.g. from padding) they should be ignored in the computation of the standardization statistics. This function allows to provide a boolean mask (of the same shape as the input) to indicate which values should be taken into account for the computation of the statistics. All values for which mask == True will be used for computation, the other will be ignored. The statistics are computed for each feature i.e. column of the input (shape [batch_size, n_numerical_features])

Parameters:

• input – input including all values, of dimension [batch_size, n_numerical_features]

• mask – boolean tensor of the same shape as input

• apply_bias_correction – if True applies Bessel’s correction to the standard deviation estimate

Returns:

FeatureStatistics (mean and std) computed on the masked values.

static from_data_sources(sources: List[ScalarDataSource]) → FeatureStatistics

For the provided data sources, compute the mean and std across all non-image features across all entries.

Parameters:

sources – list of data sources

Returns:

a Feature Statistics object storing mean and standard deviation for each non-imaging feature of the dataset.

mean: Tensor

standardize(sources: List[ScalarDataSource]) → List[ScalarDataSource]

For the provided data sources, apply standardization to the non-image features in each source. This will standardize them to mean 0, variance 1 across all sequences. All features that have zero standard deviation (constant features) are left untouched.

Parameters:

sources – list of datasources.

Returns:

list of data sources where all non-imaging features are standardized.

std: Tensor

class InnerEye.ML.utils.transforms.CTRange(**params)

level: float = None

name = 'CTRange'

output_range: Tuple[float, float] = (0.0, 255.0)

param = <param.parameterized.Parameters object>

static transform(data: Union[Tensor, ndarray], output_range: Tuple[float, float], window: float, level: float, use_gpu: bool = False) → Union[Tensor, ndarray]

window: float = None

class InnerEye.ML.utils.transforms.Compose3D(transforms: List[Transform3D[T]])

Class that allows chaining multiple transform functions together, and applying them to a sample

static apply(compose: Optional[Compose3D[T]], sample: T) → T

Apply a composition of transfer functions to the provided sample

Parameters:

• compose – A composition of transfer functions

• sample – The sample to apply the composition on

Returns:

class InnerEye.ML.utils.transforms.LinearTransform(**params)

input_range: Tuple[float, float] = None

name = 'LinearTransform'

output_range: Tuple[float, float] = None

param = <param.parameterized.Parameters object>

static transform(data: Union[Tensor, ndarray], input_range: Tuple[float, float], output_range: Tuple[float, float], use_gpu: bool = False) → Union[Tensor, ndarray]

class InnerEye.ML.utils.transforms.Transform3D(**params)

Class that allows defining a transform function with the possibility of operating on the GPU.

get_gpu_tensor_if_possible(data: T) → Any

” Get a cuda tensor if this transform was CUDA enabled and a GPU is available, otherwise return the input.

name = 'Transform3D'

param = <param.parameterized.Parameters object>

use_gpu: bool = False

InnerEye.ML.utils.transforms.get_range_for_window_level(level: float, window: float) → Tuple[float, float]

Model Utils

class InnerEye.ML.utils.config_loader.ModelConfigLoader(**params: Any)

Helper class to manage model config loading

create_model_config_from_name(model_name: str) → DeepLearningConfig

Returns a model configuration for a model of the given name. This can be either a segmentation or classification configuration for an InnerEye built-in model, or a LightningContainer. To avoid having to import torch here, there are no references to LightningContainer. Searching for a class member called <model_name> in the search modules provided recursively.

Parameters:

model_name – Name of the model for which to get the configs for.

static get_default_search_module() → str

model_configs_namespace: Optional[str] = None

name = 'ModelConfigLoader'

param = <param.parameterized.Parameters object>

class InnerEye.ML.utils.model_util.ScalarModelInputsAndLabels(model_inputs: List[Tensor], labels: Tensor, subject_ids: List[str], data_item: ScalarItem)

Holds the results of calling get_scalar_model_inputs_and_labels: For a given sample returned by the data loader, create the model inputs, the labels, the list of subjects (data loader sample can be batched), and the reconstructed data item.

data_item: ScalarItem

labels: Tensor

model_inputs: List[Tensor]

move_to_device(device: Union[str, device]) → None

Moves the model_inputs and labels field of the present object to the given device. This is done in-place.

Parameters:

device – The target device.

subject_ids: List[str]

InnerEye.ML.utils.model_util.build_net(args: SegmentationModelBase) → BaseSegmentationModel

Build network architectures

Parameters:

args – Network configuration arguments

InnerEye.ML.utils.model_util.create_model_with_temperature_scaling(config: ModelConfigBase) → Any

Create a model with temperature scaling by wrapping the result of config.create_model with ModelWithTemperature, if temperature scaling config has been provided, otherwise return the result of config.create_model

InnerEye.ML.utils.model_util.create_optimizer(config: OptimizerParams, parameters: Iterator[Parameter]) → Optimizer

InnerEye.ML.utils.model_util.create_scalar_loss_function(config: ScalarModelBase) → Module

Returns a torch module that computes a loss function for classification and regression models.

InnerEye.ML.utils.model_util.create_segmentation_loss_component(model_config: SegmentationModelBase, loss_type: SegmentationLoss, power: Optional[float]) → SupervisedLearningCriterion

Parameters:

• model_config – model configuration to get some parameters from

• loss_type – type of loss function

• power – value for class_weight_power for the loss function

Returns:

instance of loss function

InnerEye.ML.utils.model_util.create_segmentation_loss_function(model_config: SegmentationModelBase) → SupervisedLearningCriterion

Returns a loss function from the model config; mixture losses are constructed as weighted combinations of other loss functions.

InnerEye.ML.utils.model_util.generate_and_print_model_summary(config: ModelConfigBase, model: DeviceAwareModule) → None

Writes a human readable summary of the present model to logging.info, and logs the number of trainable parameters to AzureML.

Parameters:

• config – The configuration for the model.

• model – The instantiated Pytorch model.

InnerEye.ML.utils.model_util.get_scalar_model_inputs_and_labels(model: Module, sample: Dict[str, Any]) → ScalarModelInputsAndLabels

For a model that predicts scalars, gets the model input tensors from a sample returned by the data loader.

Parameters:

• model – The instantiated PyTorch model.

• target_indices – If this list is non-empty, assume that the model is a sequence model, and build the model inputs and labels for a model that predicts those specific positions in the sequence. If the list is empty,

assume that the model is a normal (non-sequence) model.

Parameters:

sample – A training sample, as returned by a PyTorch data loader (dictionary mapping from field name to value)

Returns:

An instance of ScalarModelInputsAndLabels, containing the list of model input tensors, label tensor, subject IDs, and the data item reconstructed from the data loader output

InnerEye.ML.utils.model_util.init_weights(m: Union[Conv3d, BatchNorm3d]) → None

Initialize the weights of a Pytorch module.

Parameters:

m – A PyTorch module. Only Conv3d and BatchNorm3d are initialized.

InnerEye.ML.utils.model_util.summary_for_segmentation_models(config: ModelConfigBase, model: DeviceAwareModule) → None

Generates a human readable summary of the present segmentation model, writes it to logging.info, and stores the ModelSummary object inside the argument model.

Parameters:

• config – The configuration for the model.

• model – The instantiated Pytorch model.

InnerEye.ML.utils.layer_util.get_padding_from_kernel_size(padding: PaddingMode, kernel_size: Union[Iterable[int], int], dilation: Union[Iterable[int], int] = 1, num_dimensions: int = 3) → Tuple[int, ...]

Returns padding value required for convolution layers based on input kernel size and dilation.

Parameters:

• padding – Padding type (Enum) {zero, no_padding}. Option zero is intended to preserve the tensor shape. In no_padding option, padding is not applied and the function returns only zeros.

• kernel_size – Spatial support of the convolution kernel. It is used to determine the padding size. This can be a scalar, tuple or array.

• dilation – Dilation of convolution kernel. It is used to determine the padding size. This can be a scalar, tuple or array.

• num_dimensions – The number of dimensions that the returned padding tuple should have, if both kernel_size and dilation are scalars.

Returns:

padding value required for convolution layers based on input kernel size and dilation.

InnerEye.ML.utils.layer_util.get_upsampling_kernel_size(downsampling_factor: Union[int, Tuple[int, int, int], Iterable], num_dimensions: int) → Tuple[int, int, int]

Returns the kernel size that should be used in the transpose convolution in the decoding blocks of the UNet. Use a value that is a multiple of the downsampling factor to avoid checkerboard artefacts, see https://distill.pub/2016/deconv-checkerboard/

Parameters:

downsampling_factor – downsampling factor use for each dimension of the kernel. Can be either a list of len(num_dimension) with one factor per dimension or an int in which case the

same factor will be applied for all dimension.

Parameters:

num_dimensions – number of dimensions of the kernel

Returns:

upsampling_kernel_size

InnerEye.ML.utils.layer_util.initialise_layer_weights(module: Module) → None

Torch kernel initialisations for conv and batch_norm are based on leaky_relu activation. Apply Kaiming initialisation and adapt the kernel weights for relu activation.

Parameters:

module – Torch nn module

InnerEye.ML.utils.layer_util.set_model_to_eval_mode(model: Module) → Generator

Puts the given torch model into eval mode. At the end of the context, resets the state of the training flag to what is was before the call.

Parameters:

model – The model to modify.

InnerEye.ML.utils.model_metadata_util.generate_random_colours_list(rng: Random, size: int) → List[Tuple[int, int, int]]

Generates a list of random colours in RGB given a random number generator and the size of this list

Parameters:

• rng – random number generator

• size – size of the list

Returns:

list of random colours in RGB

InnerEye.ML.utils.model_metadata_util.random_colour(rng: Random) → Tuple[int, int, int]

Generates a random colour in RGB given a random number generator

Parameters:

rng – Random number generator

Returns:

Tuple with random colour in RGB

Training Utils

class InnerEye.ML.utils.lr_scheduler.LinearWarmUp(optimizer: Optimizer, warmup_epochs: int, final_lr: float, last_epoch: int = - 1)

Implements linear warmup up to a given initial learning rate.

get_lr() → List[float]

warmup_multiplier() → float

class InnerEye.ML.utils.lr_scheduler.PolynomialLR(gamma: float, l_rate: float, min_l_rate: float, epochs_after_warmup: int)

get_lr(epoch: int) → float

class InnerEye.ML.utils.lr_scheduler.SchedulerWithWarmUp(args: OptimizerParams, optimizer: Optimizer, num_epochs: int, last_epoch: int = - 1)

LR Scheduler which runs a warmup schedule (linear ramp-up) for a few iterations, and then switches to one of the normal schedulers.

get_last_lr() → List[float]

Return last computed learning rate by current scheduler.

get_scheduler(args: OptimizerParams) → _LRScheduler

Create the LR scheduler that will be used after warmup, based on the config params.

load_state_dict(state_dict: Dict) → None

Initializes the current object with values from state_dict. Initializes variables “_scheduler” and “_warmup_scheduler” separately, by calling load_state_dict for these variables.

state_dict() → Dict

Returns a dictionary with all the values in this objects __dict__. It creates the dictionary entry for variables “_scheduler” and “_warmup_scheduler” separately, by calling state_dict for these variables. The state dict does not include the state of the optimizer.

step(epoch: Optional[int] = None) → None

InnerEye.ML.utils.lr_scheduler.get_current_learning_rates(optimizer: Optimizer) → List[float]

Reads the current values of the learning rate(s) for all parameter groups from the optimizer.

class InnerEye.ML.utils.metrics_util.MetricsPerPatientWriter

Stores information about metrics eg: Dice, Mean and Hausdorff Distances, broken down by patient and structure.

add(patient: str, structure: str, dice: float, hausdorff_distance_mm: float, mean_distance_mm: float) → None

Adds a Dice score, Mean and Hausdorff Distances for a patient + structure combination to the present object.

Parameters:

• patient – The name of the patient.

• structure – The structure that is predicted for.

• dice – The value of the Dice score that was achieved.

• hausdorff_distance_mm – The hausdorff distance in mm

• mean_distance_mm – The mean surface distance in mm

save_aggregates_to_csv(file_path: Path, allow_incomplete_labels: bool = False) → None

Writes the per-structure aggregate Dice scores (mean, median, and others) to a CSV file. The aggregates are those that are output by the Dataframe ‘describe’ method.

Parameters:

• file_path – The name of the file to write to.

• allow_incomplete_labels – boolean flag. If false, all ground truth files must be provided. If true, ground truth files are optional and we add a total_patients count column for easy

comparison. (Defaults to False.)

to_csv(file_name: Path) → None

Writes the per-patient per-structure metrics to a CSV file. Sorting is done first by structure name, then by Dice score ascending.

Parameters:

file_name – The name of the file to write to.

to_data_frame() → DataFrame

Creates a DataFrame that holds all the per-patient per-structure results. A numeric column is added, that contains the Dice score as a numeric value.

InnerEye.ML.utils.metrics_util.binary_classification_accuracy(model_output: Union[Tensor, ndarray], label: Union[Tensor, ndarray], threshold: float = 0.5) → float

Computes the accuracy for binary classification from a real-valued model output and a label vector. The model output is assumed to be in the range between 0 and 1, a value larger than 0.5 indicates a prediction of class 1. The label vector is expected to contain class indices 0 and 1 only, but is also thresholded at 0.5.

Parameters:

• model_output – A tensor containing model outputs.

• label – A tensor containing class labels.

• threshold – the cut-off probability threshold for predictions. If model_ouput is > threshold, the predicted class is 1 else 0.

Returns:

1.0 if all predicted classes match the expected classes given in ‘labels’. 0.0 if no predicted classes match their labels.

InnerEye.ML.utils.metrics_util.convert_input_and_label(model_output: Union[Tensor, ndarray], label: Union[Tensor, ndarray]) → Tuple[Tensor, Tensor]

Ensures that both model_output and label are tensors of dtype float32. :return: a Tuple with model_output, label as float tensors.

InnerEye.ML.utils.metrics_util.format_metric(metric: float) → str

Returns a readable string from the given Dice or loss function value, rounded to 3 digits.

InnerEye.ML.utils.metrics_util.get_label_overlap_stats(labels: ndarray, label_names: List[str]) → Dict[str, int]

Computes overlap between labelled structures and returns the stats in a dictionary format.

Parameters:

• labels – nd-NumPy array containing binary masks for all labels

• label_names – A list of strings containing target label names, e.g. [spleen, liver]

InnerEye.ML.utils.metrics_util.get_label_volume(labels: ndarray, label_names: List[str], label_spacing: Tuple[float, float, float]) → Dict[str, float]

Computes volume of ground-truth labels in mL and returns it in a dictionary

Parameters:

• labels – nd-NumPy array containing binary masks for all labels

• label_names – A list of strings containing target label names, e.g. [spleen, liver]

• label_spacing – label spacing

InnerEye.ML.utils.metrics_util.get_number_of_voxels_per_class(labels: Tensor) → Tensor

Computes the number of voxels for each class in a one-hot label map.

Parameters:

labels – one-hot label map in shape Batches x Classes x Z x Y x X or Classes x Z x Y x X

Returns:

A tensor of shape [Batches x Classes] containing the number of non-zero voxels along Z, Y, X

InnerEye.ML.utils.metrics_util.is_missing_ground_truth(ground_truth: ndarray) → bool

calculate_metrics_per_class in metrics.py and plot_contours_for_all_classes in plotting.py both check whether there is ground truth missing using this simple check for NaN value at 0, 0, 0. To avoid duplicate code we bring it here as a utility function.

Parameters:

• ground_truth – ground truth binary array with dimensions: [Z x Y x X].

• label_id – Integer index of the label to check.

Returns:

True if the label is missing (signified by NaN), False otherwise.

InnerEye.ML.utils.metrics_util.mean_absolute_error(model_output: Union[Tensor, ndarray], label: Union[Tensor, ndarray]) → float

Computes the mean absolute error i.e. mean(abs(model_output - label))

InnerEye.ML.utils.metrics_util.mean_squared_error(model_output: Union[Tensor, ndarray], label: Union[Tensor, ndarray]) → float

Computes the mean squared error i.e. mean((model_output - label)^2)

InnerEye.ML.utils.metrics_util.r2_score(model_output: Union[Tensor, ndarray], label: Union[Tensor, ndarray]) → float

Computes the coefficient of determination R2. Represents the proportion of variance explained by the (independent) variables in the model. R2 = 1 - Mean(SquaredErrors) / Variance(Labels)

class InnerEye.ML.utils.sequence_utils.MaskedModelOutputAndLabelSequences(model_outputs: PackedSequence, labels: PackedSequence, subject_ids: Optional[Sequence[str]])

Dataclass to encapsulate masked model outputs, labels and associated subject ids

labels: PackedSequence

model_outputs: PackedSequence

subject_ids: Optional[Sequence[str]]

InnerEye.ML.utils.sequence_utils.apply_sequence_model_loss(loss_fn: Module, model_output: Tensor, labels: Tensor) → Tensor

Applies a loss function to a model output and labels, when the labels come from sequences with unequal length. Missing sequence elements are masked out.

Parameters:

• loss_fn – The loss function to apply to the sequence elements that are present.

• model_output – The model outputs

• labels – The ground truth labels.

Returns:

The value of the loss function.

InnerEye.ML.utils.sequence_utils.get_masked_model_outputs_and_labels(model_output: Tensor, labels: Union[ndarray, Tensor], subject_ids: Optional[Sequence[str]] = None) → Optional[MaskedModelOutputAndLabelSequences]

Helper function to get masked model outputs, labels and their associated subject ids. Masking is performed by excluding the NaN model outputs and labels based on a bool mask created using the occurrences of NaN in the labels provided.

Parameters:

• model_output – The model output tensor to mask.

• labels – The label tensor to use for mask, and use for masking.

• subject_ids – The associated subject ids.

Returns:

None if all labels are required to be masked, otherwise MaskedModelOutputAndLabelSequences

InnerEye.ML.utils.sequence_utils.map_packed_sequence_data(x: PackedSequence, f: Callable[[Tensor], Tensor]) → PackedSequence

Helper function to apply a map transform to a packed sequence

InnerEye.ML.utils.sequence_utils.sequences_to_padded_tensor(sequences: List[Tensor], padding_value: float = 0.0) → Tensor

Method to convert possibly unequal length sequences to a padded tensor.

Parameters:

• sequences – List of Tensors to pad

• padding_value – Padding value to use, default is 0.0

Returns:

Output tensor with shape B x * where * is the max dimensions from the list of provided tensors. And B is the number of tensors in the list of sequences provided.

class InnerEye.ML.utils.supervised_criterion.BinaryCrossEntropyWithLogitsLoss(num_classes: int, class_counts: Optional[Dict[float, int]] = None, num_train_samples: Optional[int] = None, **kwargs: Any)

A wrapper function for torch.nn.BCEWithLogitsLoss to enable label smoothing

forward_minibatch(output: Tensor, target: Tensor, **kwargs: Any) → Any

get_positive_class_weights() → Tensor

Returns the weights of the positive class only from the list of dictionaries containing the counts for all classes for each target position. :return: a list of weights to use for the positive class for each target position.

training: bool

class InnerEye.ML.utils.supervised_criterion.SupervisedLearningCriterion(smoothing_eps: float = 0.0, is_binary_classification: bool = False)

Base class for criterion functions used for supervised learning, with the ability to smooth labels if required.

forward(*input: Tensor, **kwargs: Any) → Any

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

abstract forward_minibatch(output: Any, target: Any, **kwargs: Any) → Any

training: bool

class InnerEye.ML.utils.run_recovery.RunRecovery(checkpoints_roots: List[Path])

Class to encapsulate information relating to run recovery (eg: check point paths for parent and child runs)

checkpoints_roots: List[Path]

get_best_checkpoint_paths() → List[Path]

get_recovery_checkpoint_paths() → List[Path]

class InnerEye.ML.utils.surface_distance_utils.Plane(value)

An enumeration.

AXIAL = 'AXIAL'

CORONAL = 'CORONAL'

SAGITTAL = 'SAGITTAL'

class InnerEye.ML.utils.surface_distance_utils.SurfaceDistanceConfig(should_validate: bool = True, throw_if_unknown_param: bool = False, **params: Any)

annotators: List[str] = []

execution_mode: ModelExecutionMode = 'Test'

ground_truth_dir: str = None

model_name: str = None

name = 'SurfaceDistanceConfig'

outlier_range: float = 1.0

output_img_dir: str = None

param = <param.parameterized.Parameters object>

plane: Plane = 'SAGITTAL'

run_mode: SurfaceDistanceRunType = 'IOV'

run_recovery_id: str = None

class InnerEye.ML.utils.surface_distance_utils.SurfaceDistanceRunType(value)

An enumeration.

IOV = 'IOV'

OUTLIERS = 'OUTLIERS'

InnerEye.ML.utils.surface_distance_utils.calculate_surface_distances(ground_truth: ndarray, pred: ndarray, voxel_spacing: Union[float, List[float]]) → ndarray

Calculate the Euclidean surface distance between a given prediction and the ‘ground truth’

Parameters:

• ground_truth – 3D binary array (X x Y x Z) of ground truth segmentation

• pred – 3D binary array (X x Y x Z) of predicted segmentation

• voxel_spacing – voxel spacing, taken from the image header (transposed if applicable)

Returns:

InnerEye.ML.utils.surface_distance_utils.dir_for_subject(azure_config: AzureConfig, model_config: SegmentationModelBase, prefix: Path) → Path

Combine the local data dir and the Azure dir we are downloading images from to get the directory the images for a subject will be downlaoded to

Parameters:

• azure_config – AzureConfig

• model_config – Config

• prefix –

Returns:

InnerEye.ML.utils.surface_distance_utils.extract_border(img: ndarray, connectivity: int = 1) → ndarray

Get contour by calculating eroded version of the image and subtracting from the original.

Parameters:

• img – Array containing structure from which to extract the border

• connectivity – integer determining which pixels are considered neighbours of the central element, ranging from 1 = no diagonal elements and rank = all elements

Returns:

InnerEye.ML.utils.surface_distance_utils.get_annotations_and_majority_vote(model_config: SegmentationModelBase, annotators: List[str], structure_name: str) → ndarray

Load each annotation and calculate the ‘gold standard’ segmentation (with majority voting)

Parameters:

• model_config – Config

• annotators – List of the annotator names as they appear in filepaths

• structure_name – Name of the anatomical structure

Returns:

InnerEye.ML.utils.surface_distance_utils.get_first_child_run(azure_config: AzureConfig) → Run

Download first child run in order to download data

Parameters:

azure_config –

Returns:

first child run

InnerEye.ML.utils.surface_distance_utils.get_majority_vote(arr_list: List[ndarray]) → ndarray

Given a list of label arrays, get the majority vote at each voxel

Parameters:

arr_list –

Returns:

InnerEye.ML.utils.surface_distance_utils.get_metrics_path(azure_config: AzureConfig, model_config: SegmentationModelBase) → Path

Get path to metrics.csv file for a downlaoded run, for the purpose of determining outliers

Parameters:

• azure_config – AzureConfig

• model_config – Config

Returns:

InnerEye.ML.utils.surface_distance_utils.get_run_output_dir(azure_config: AzureConfig, model_config: SegmentationModelBase) → Path

Get the directory where Azure run’s output data will be stored. the total filepath will depend on which container we download data from.

Parameters:

• azure_config –

• model_config –

Returns:

output_dir: directory that all artifact paths use as a prefix

InnerEye.ML.utils.surface_distance_utils.get_subject_prefix(model_config: SegmentationModelBase, train_mode: ModelExecutionMode, subject_id: int) → Path

Returns the path to subject dir for a given model and train mode

Parameters:

• model_config – Config

• train_mode – Model execution mode -i.e. train, test or val

• subject_id – ID of the subject

Returns:

prefix: the filepath prefix within the container from which to download all artifacts

InnerEye.ML.utils.surface_distance_utils.initialise_surface_distance_dictionary(annotators: List[str], arr_shape: Tuple[int, int, int]) → Dict[str, ndarray]

Given a list of annotators and the image size expected for all surface distance plots, return a dictionary where keys are annotators and values are zeros in shape of entire image, so that surface distances for each structure can be added to one plot.

Parameters:

• annotators – List of the annotator names as they appear in filepaths

• arr_shape – Shape of the array to be intialized

Returns:

InnerEye.ML.utils.surface_distance_utils.load_ground_truth_from_run(model_config: SegmentationModelBase, sd_config: SurfaceDistanceConfig, subject_id: int, structure: str) → ndarray

For outliers, load individual ground truth file for a given dataset, subject ID and structure name

Parameters:

• model_config –

• sd_config –

• subject_id – ID of the given subject

• structure – Name of the anatomical structure

Returns:

ground truth array

class InnerEye.ML.utils.temperature_scaling.ModelWithTemperature(model: DeviceAwareModule, temperature_scaling_config: TemperatureScalingConfig)

Torch module to wrap a model with temperature scaling.

forward(*x: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_input_tensors(item: T) → List[E]

Extract the input tensors from a data sample as required by the forward pass of the module.

Parameters:

item – a data sample

Returns:

the correct input tensors for the forward pass

get_last_encoder_layer_names() → List[str]

Return the name of the last encoder layers for GradCam. Default is an empty list.

set_temperature(logits: Tensor, labels: Tensor, criterion_fn: Callable[[Tensor, Tensor], Tuple[Tensor, Tensor]], use_gpu: bool) → float

Tune the temperature of the model using the provided logits and labels.

Parameters:

• logits – Logits to use to learn the temperature parameter

• labels – Labels to use to learn the temperature parameter

• criterion_fn – A criterion function s.t: (logits, labels) => (loss, ECE)

• use_gpu – If True then GPU will be used otherwise CPU will be used.

Returns:

Optimal temperature value

temperature_scale(logits: Tensor) → Tensor

Perform temperature scaling on logits

training: bool

class InnerEye.ML.utils.device_aware_module.DeviceAwareModule

Wrapper around base pytorch module class that can provide information about its devices

get_devices() → List[device]

Returns:

a list of device ids on which this module is deployed.

get_input_tensors(item: T) → List[E]

Extract the input tensors from a data sample as required by the forward pass of the module.

Parameters:

item – a data sample

Returns:

the correct input tensors for the forward pass

get_last_encoder_layer_names() → List[str]

Return the name of the last encoder layers for GradCam. Default is an empty list.

get_number_trainable_parameters() → int

Returns:

the number of trainable parameters in the module.

is_model_on_gpu() → bool

Checks if the model is cuda activated or not :return: True if the model is running on the GPU.

training: bool

Other / Misc Utils

class InnerEye.ML.utils.ml_util.RandomStateSnapshot(random_state: Any, numpy_random_state: Any, torch_random_state: Any, torch_cuda_random_state: Any)

Snapshot of all of the random generators states: python, numpy, torch.random, and torch.cuda for all gpus.

numpy_random_state: Any

random_state: Any

restore_random_state() → None

Restore the state for the random number generators of python, numpy, torch.random, and torch.cuda for all gpus.

static snapshot_random_state() → RandomStateSnapshot

Get a snapshot of all random generators state.

torch_cuda_random_state: Any

torch_random_state: Any

InnerEye.ML.utils.ml_util.check_size_matches(arg1: Union[ndarray, Tensor], arg2: Union[ndarray, Tensor], dim1: int = 0, dim2: int = 0, matching_dimensions: Optional[List[int]] = None, arg1_name: str = 'arg1', arg2_name: str = 'arg2') → None

Checks if the two given numpy arrays have matching shape. Raises a ValueError if the shapes do not match. The shape check can be restricted to a given subset of dimensions.

Parameters:

• arg1 – The first array to check.

• arg2 – The second array to check.

• dim1 – The expected number of dimensions of arg1. If zero, no check for number of dimensions will be conducted.

• dim2 – The expected number of dimensions of arg2. If zero, no check for number of dimensions will be conducted.

• matching_dimensions – The dimensions along which the two arguments have to match. For example, if arg1.ndim==4 and arg2.ndim==5, matching_dimensions==[3] checks if arg1.shape[3] == arg2.shape[3].

• arg1_name – If provided, all error messages will use that string to instead of “arg1”

• arg2_name – If provided, all error messages will use that string to instead of “arg2”

Raises:

ValueError if shapes don’t match

InnerEye.ML.utils.ml_util.is_gpu_available() → bool

Returns True if a GPU with at least 1 device is available.

InnerEye.ML.utils.ml_util.is_tensor_nan(tensor: Tensor) → bool

Returns True if any of the tensor elements is Not a Number.

Parameters:

tensor – The tensor to check.

Returns:

True if any of the tensor elements is Not a Number, False if all entries are valid numbers. If the tensor is empty, the function returns False.

InnerEye.ML.utils.ml_util.is_tensor_nan_or_inf(tensor: Tensor) → bool

Returns True if any of the tensor elements is Not a Number or Infinity.

Parameters:

tensor – The tensor to check.

Returns:

True if any of the tensor elements is Not a Number or Infinity, False if all entries are valid numbers.

InnerEye.ML.utils.ml_util.is_test_from_execution_mode(execution_mode: ModelExecutionMode) → bool

Returns a boolean by checking the execution type. The output is used to determine the properties of the forward pass, e.g. model gradient updates or metric computation.

Returns:

True if execution mode is VAL or TEST, False if TRAIN

Raises:

ValueError – if the execution mode is invalid

InnerEye.ML.utils.ml_util.set_random_seed(random_seed: int, caller_name: Optional[str] = None) → None

Set the seed for the random number generators of python, numpy, torch.random, and torch.cuda for all gpus.

Parameters:

• random_seed – random seed value to set.

• caller_name – name of the caller for logging purposes.

InnerEye.ML.utils.ml_util.string_to_integer_list(csv_list: str, length: int) → List[int]

Helper function to convert an array or list of integers saved as a list in a csv back to its original format.

Parameters:

• length – excepted length of converted list

• csv_list – list as string

Returns:

converted list of integers

InnerEye.ML.utils.ml_util.validate_dataset_paths(dataset_path: Path = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/innereye-deeplearning/checkouts/latest/docs/source'), dataset_csv: str = 'dataset.csv') → None

Validates that the required dataset csv file exists in the given path.

Parameters:

• dataset_path – The base path

• custom_dataset_csv – The name of the dataset csv file

Raises:

ValueError if the dataset does not exist.

InnerEye.ML.utils.plotting_util.get_cropped_axes(image: ndarray, boundary_width: int = 5) → Tuple[slice, ...]

Return the min and max values on both x and y axes where the image is not empty Method: find the min and max of all non-zero pixels in the image, and add a border

Parameters:

• image – the image to be cropped

• boundary_width – number of pixels boundary to add around bounding box

Returns:

InnerEye.ML.utils.plotting_util.get_view_dim_and_origin(plane: Plane) → Tuple[int, str]

Get the axis along which to slice, as well as the orientation of the origin, to ensure images are plotted as expected

Parameters:

plane – the plane in which to plot (i.e. axial, sagittal or coronal)

Returns: