Layers

class InnerEye.ML.models.layers.basic.BasicLayer(channels: ~typing.Tuple[int, int], kernel_size: ~typing.Union[~typing.Iterable[int], int], stride: ~typing.Union[~typing.Iterable[int], int] = 1, dilation: ~typing.Union[~typing.Iterable[int], int] = 1, padding: ~InnerEye.ML.config.PaddingMode = PaddingMode.NoPadding, use_bias: bool = False, activation: ~typing.Optional[~typing.Callable] = <class 'torch.nn.modules.activation.ReLU'>, use_batchnorm: bool = True)

A Basic Layer applies a 3D convolution and BatchNorm with the given channels, kernel_size, and dilation. The output of BatchNorm layer is passed through an activation function and its output is returned.

Parameters:

• channels – Number of input and output channels.

• kernel_size – Spatial support of convolution kernels

• stride – Kernel stride lenght for convolution op

• padding – Feature map padding after convolution op {“constant/zero”, “no_padding”}. When it is set to “no_padding”, no padding is applied. For “constant”, feature-map tensor size is kept the same at the output by

padding with zeros.

Parameters:

• dilation – Kernel dilation used in convolution layer

• use_bias – If set to True, a bias parameter will be added to the layer. Default is set to False as batch normalisation layer has an affine parameter which are used applied after the bias term is added.

• activation – Activation layer (e.g. nonlinearity) to be used after the convolution and batch norm operations.

forward(x: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class InnerEye.ML.models.layers.identity.Identity

Implements an identity torch module where input is passed as it is to output. There are no parameters in the module.

forward(input: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class InnerEye.ML.models.layers.pooling_layers.AveragePooling

Global average pooling operation across all spatial dimensions (e.g. 2D and 3D image grids)

forward(*input: Any, **kwargs: Any) → Any

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class InnerEye.ML.models.layers.pooling_layers.Gated3dPoolingLayer(in_features: int)

Gated pooling. Flatten each volume x [1, ZYX], feed through a one layer NN yield one weight per image. This weight is used as the mixing proportion for max_pooling features and average pooling features similar to what is done in MixPooling.

forward(*input: Any, **kwargs: Any) → Tensor

Parameters:

input – batch of size [B, C, Z, X, Y

training: bool

class InnerEye.ML.models.layers.pooling_layers.MaxPooling

Global max pooling operation across all spatial dimensions (e.g. 2D and 3D image grids)

forward(*input: Any, **kwargs: Any) → Any

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class InnerEye.ML.models.layers.pooling_layers.MixPooling

Compute a mixture of max pooling and average pooling. feature = a * avg_3d + (1-a) * max_3d given a in [0, 1] The mixing weight is a learnable parameter.

forward(*input: Any, **kwargs: Any) → Any

Parameters:

input – batch of size [B, C, Z, X, Y]

training: bool

class InnerEye.ML.models.layers.pooling_layers.ZAdaptive3dAvgLayer(in_features: int)

Performs 3D average pooling with custom weighting along the Z dimension. In short: extract the 2d average for each B-scan. Learn a weighting for averaging these features over all B-Scans.

forward(*input: Tensor, **kwargs: Any) → Tensor

Parameters:

input – batch of size [B, C, Z, X, Y]

training: bool

class InnerEye.ML.models.layers.weight_standardization.WeightStandardizedConv2d(in_channels: int, out_channels: int, kernel_size: Union[int, Tuple[int, int]], stride: Union[int, Tuple[int, int]] = 1, padding: Union[int, Tuple[int, int]] = 0, dilation: Union[int, Tuple[int, int]] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros')

Weight Standardization https://arxiv.org/pdf/1903.10520.pdf

bias: Optional[Tensor]

dilation: Tuple[int, ...]

forward(input: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

groups: int

kernel_size: Tuple[int, ...]

out_channels: int

output_padding: Tuple[int, ...]

padding: Union[str, Tuple[int, ...]]

padding_mode: str

static standardize(weights: Tensor) → Tensor

Normalize weights on a per-kernel basis for all kernels.

stride: Tuple[int, ...]

transposed: bool

weight: Tensor