ai4med.libs.models package

Bases: object

Implementation of AHNet in TensorFlow for segmentation task.

3D Anisotropic Hybrid Network (AH-Net) that transfers convolutional features learned from 2D images to 3D anisotropic volumes. Such a transfer inherits the desired strong generalization capability for within-slice information while naturally exploiting between-slice information for more effective modelling. The focal loss is further utilized for more effective end-to-end learning. We experiment with the proposed 3D AH-Net on two different medical image analysis tasks, namely lesion detection from a Digital Breast Tomosynthesis volume, and liver and liver tumor segmentation from a Computed Tomography volume and obtain the state-of-the-art results.

This implementation accepts an hdf5 file as pretrain weights through pretrain_weight_url, and if provided will initialize the network with those weights. If nothing is provided, the default initial weights will be from: https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5

For more details, please refer to the paper:

Siqi Liu, Daguang Xu, S. Kevin Zhou, Thomas Mertelmeier, Julia Wicklein, Anna Jerebko, Sasa Grbic, Olivier Pauly, Weidong Cai, Dorin Comaniciu 3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes arXiv:1711.08580 [cs.CV]. A shorter version was presented in MICCAI18: https://link.springer.com/chapter/10.1007/978-3-030-00934-2_94

Parameters

• inputs (tf.Tensor) – input data for the AHNet model

• num_classes (int) – expected class number of prediction results

• is_training (bool) – in training process or not

• if_use_psp (bool) – use “pyramid volumetric pooling” module for decoder or not

• plane (str) – permute the weight tensors of first convolution layer to change channel dimension to plane dimension.

• pretrain_weight_name (str) – local path of pretrain weight file

• pretrain_weight_url (str) – link to download pretrain weight file

• final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

• dtype (tf.dtype) – reserved argument for future features

• data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

• n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

Prediction results

loss()

Define additional loss function in AHNet model.

Parameters

None –

Returns

Loss function

model()

Define neural network layers of AHNet.

AHNet layers connect with other modules to construct TensorFlow graph.

Parameters

None –

Returns

Prediction results

Bases: object

Build AlexNet for 3D images or 2D images on image classification

with / without (fixed input size) multiple instance learning (MIL)

Parameters

• inputs – Tensor of input image (e.g NxCxHxWxD)

• num_classes – number of classes

• training – bool variable indicating training or testing

• num_feat – number of filters or hidden nodes in each layer

• mil – use multi-instance learning or not

• use_batch_norm – bool variable indicating batch norm, default True

• use_group_norm – bool variable indicating group norm, default False

• use_group_normG – number of groups

• data_format – channels_first (default) or channels_last

Returns

the output of the AlexNet (MIL)

batch_norm_relu(inputs, alpha=0, scope='batch_norm_scope')

conv(inputs, filters, kernel_size, strides, padding='same', dilation_rate=1, use_bias=None)

conv_transpose(inputs, filters, kernel_size, strides, padding='same', use_bias=None)

kernel_initializer()

loss()

model()

regularizer()

Bases: object

A building block for a dense block.

Parameters

• growth_rate (float) – growth rate at dense layers

• stage (int) – index number for this stage

• branch (int) – index number for this branch

• data_format (str) – support both channel_first or channel_last

• kernel_regularizer – Regularization method for weights

• is_training (bool) – in training process or not

• data_h5 (dict) – model weights file

• n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

output tensor for the block

Return type

model_dict

Bases: object

A dense block.

Parameters

• blocks (int) – the number of building blocks

• stage (int) – index number for this stage

• data_format (str) – support both channel_first or channel_last

• kernel_regularizer – Regularization method for weights

• is_training (bool) – in training process or not

• data_h5 (dict) – model weights file

• n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

output tensor for the block

Return type

model_dict

Bases: object

Creates a pretrained DenseNet121 model for classification.

Keras densenet121 model with intial weights taken from: https://github.com/flyyufelix/DenseNet-Keras

URL to the h5 file: https://drive.google.com/a/nvidia.com/uc?authuser=0&id=0Byy2AcGyEVxfSTA4SHJVOHNuTXc&export=download

This implementation accepts an hdf5 file as pretrain weights through pretrain_weight_url, and if provided will initialize the network with those weights.

Original source file: https://github.com/keras-team/keras-applications/blob/master/keras_applications/densenet.py,

Apart from NVIDIA contributions, all other code is licensed under the Keras license and copyrighted to the respective Keras contributers.

DenseNet models for Keras

Reference paper: - [Densely Connected Convolutional Networks] (https://arxiv.org/abs/1608.06993) (CVPR 2017 Best Paper Award)

Reference implementation:

• [Torch DenseNets]

(https://github.com/liuzhuang13/DenseNet/blob/master/models/densenet.lua)

• [TensorNets]

(https://github.com/taehoonlee/tensornets/blob/master/tensornets/densenets.py)

Parameters

• blocks (int) – numbers of building blocks for the four dense layers

• net_config – pretrain_weight_name and pretrain_weight_url configs

• input_shape (tuple) – input data dimensions

• kernel_regularizer – Regularization method for weights

• is_training (bool) – in training process or not

• dtype (tf.dtype) – reserved argument for future features

• n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet

Return type

model_dict

Bases: object

A transition block.

Parameters

• reduction (float) – compression rate at transition layers

• stage (int) – index number for this stage

• data_format (str) – support both channel_first or channel_last

• kernel_regularizer – Regularization method for weights

• is_training (bool) – in training process or not

• data_h5 (dict) – model weights file

• n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

output tensor for the block

Return type

model_dict

Bases: object

Create a pretrained DenseNet121 model for classification.

Parameters

• net_config – pretrain_weight_name and pretrain_weight_url configs

• num_classes (int) – expected class number of prediction results

• input_shape (tuple) – input data dimensions

• is_training (bool) – in training process or not

• kernel_regularizer – Regularization method for weights

• n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet121: model_dict[‘model’] is the output model_dict[‘variables’] is the list of created variables model_dict[‘pretrain_variables’] is the list of trunk variables model_dict[‘reg_losses’] is the list of reg losses model_dict[‘update_ops’] list of BN update ops

Return type

model_dict

loss()

Get the additional loss function if required.

Parameters

None –

Returns

Loss function

Bases: object

Implementation of ResNet based 3D Segmentation model.

Construct a encoder - decoder structure network based on ResNet. Residual learning framework eases the training of networks that are substantially deeper than those used previously. These residual networks are easier to optimize, and can gain accuracy from considerably increased depth. ResNet won the 1st place on the ILSVRC 2015 classification task.

Parameters

• inputs (tf.Tensor) – input data for the AHNet model

• num_classes (int) – expected class number of prediction results

• training (bool) – in training process or not

• blocks_down (str) – encoder layer number and ResNet block number of every layer

• blocks_up (str) – decoder layer number and ResNet block number of every layer

• init_filters (int) – convolution kernel number of first layer. Default: 8

• use_batch_norm (bool) – use batch normalization, but only if use_group_norm is explicitly set to false.

• use_group_norm (bool) – use group normalization or not. By default this is true.

• use_group_normG (int) – group number of group normalization. Default: 8

• reg_weight (float) – weight regularization factor

• dropout_prob (float) – dropout rate for several layers in training

• final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

• use_vae (bool) – use “Variational Auto Encoder” or not

• dtype (tf.dtype) – data type of the network

• data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

Returns

Prediction results

loss()

model()

Define neural network layers of ResNet based encoder and decoder.

Layers connect with other modules to construct TensorFlow graph.

Parameters

None –

Returns

Prediction results

Bases: object

3D Segmentation model based on UNet.

For more details, check our implemented UNet in the paper. Wentao Zhu et al., NeurReg: Neural Registration and Its Application to Image Segmentation, WACV 2020. Wentao Zhu et al., Neural Multi-Scale Self-Supervised Registration for Echocardiogram Dense Tracking, arXiv 2019.

Parameters

• num_classes (int) – expected class number of prediction results

• nf_enc (str) – number of filters in each encoder layer

• nf_dec (str) – number of filters in each decoder layer

• use_batch_norm (bool) – augmented batch normalization

• use_group_norm (bool) – augmented group normalization

• use_group_normG (int) – group numbter of group normalization

• reg_weight (float) – weight regularization factor

• dropout_prob (float) – dropout rate for several layers in training

• final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

• dtype (tf.dtype) – data type of the network

• data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

Returns

Prediction results

batch_norm_relu(inputs, alpha=0, scope='batch_norm_scope')

conv(inputs, filters, kernel_size, strides, padding='same', dilation_rate=1, use_bias=None)

conv_transpose(inputs, filters, kernel_size, strides, padding='same', use_bias=None)

kernel_initializer()

loss()

model()

regularizer()

Bases: object

3D Segmentation model based on UNet.

For more details, check our implemented UNet in the paper. Wentao Zhu et al., NeurReg: Neural Registration and Its Application to Image Segmentation, WACV 2020. Wentao Zhu et al., Neural Multi-Scale Self-Supervised Registration for Echocardiogram Dense Tracking, arXiv 2019.

Parameters

• num_classes (int) – expected class number of prediction results

• nf_enc (str) – number of filters in each encoder layer

• nf_dec (str) – number of filters in each decoder layer

• gpu_enc (str) – the gpu id for each encoder layer

• gpu_dec (str) – the gpu id for each decoder layer

• use_batch_norm (bool) – augmented batch normalization

• use_group_norm (bool) – augmented group normalization

• use_group_normG (int) – group numbter of group normalization

• reg_weight (float) – weight regularization factor

• dropout_prob (float) – dropout rate for several layers in training

• final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

• dtype (tf.dtype) – data type of the network

• data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

Returns

Prediction results

batch_norm_relu(inputs, alpha=0.0, scope='batch_norm_scope')

conv(inputs, filters, kernel_size, strides, padding='same', dilation_rate=1, use_bias=None)

conv_transpose(inputs, filters, kernel_size, strides, padding='same', use_bias=None)

kernel_initializer()

loss()

model()

regularizer()

up_sampling(inputs, size=2, interp='bilinear')