ai4med.libs.models package

3.1
class SegAhnet(inputs, num_classes, is_training, if_use_psp=False, plane='z', pretrain_weight_name=None, pretrain_weight_url=None, final_activation='softmax', dtype=tf.float32, data_format='channels_first', n_spatial_dim=2)

Bases: object

Implementation of AHNet in TensorFlow for segmentation task.

3D Anisotropic Hybrid Network (AH-Net) that transfers convolutional features learned from 2D images to 3D anisotropic volumes. Such a transfer inherits the desired strong generalization capability for within-slice information while naturally exploiting between-slice information for more effective modelling. The focal loss is further utilized for more effective end-to-end learning. We experiment with the proposed 3D AH-Net on two different medical image analysis tasks, namely lesion detection from a Digital Breast Tomosynthesis volume, and liver and liver tumor segmentation from a Computed Tomography volume and obtain the state-of-the-art results.

This implementation accepts an hdf5 file as pretrain weights through pretrain_weight_url, and if provided will initialize the network with those weights. If nothing is provided, the default initial weights will be from: https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5

For more details, please refer to the paper:

Siqi Liu, Daguang Xu, S. Kevin Zhou, Thomas Mertelmeier, Julia Wicklein, Anna Jerebko, Sasa Grbic, Olivier Pauly, Weidong Cai, Dorin Comaniciu 3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes arXiv:1711.08580 [cs.CV]. A shorter version was presented in MICCAI18: https://link.springer.com/chapter/10.1007/978-3-030-00934-2_94

Parameters
  • inputs (tf.Tensor) – input data for the AHNet model

  • num_classes (int) – expected class number of prediction results

  • is_training (bool) – in training process or not

  • if_use_psp (bool) – use “pyramid volumetric pooling” module for decoder or not

  • plane (str) – permute the weight tensors of first convolution layer to change channel dimension to plane dimension.

  • pretrain_weight_name (str) – local path of pretrain weight file

  • pretrain_weight_url (str) – link to download pretrain weight file

  • final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

  • dtype (tf.dtype) – reserved argument for future features

  • data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

Prediction results

loss()

Define additional loss function in AHNet model.

Parameters

None

Returns

Loss function

model()

Define neural network layers of AHNet.

AHNet layers connect with other modules to construct TensorFlow graph.

Parameters

None

Returns

Prediction results

class Alexnet(inputs, num_classes, training, num_feat='96,256,512,1024,1024,3072,4096', mil=False, use_batch_norm=True, use_group_norm=False, use_group_normG=8, reg_weight=0.0, dropout_prob=0.5, final_activation='softmax', dtype=tf.float32, data_format='channels_first')

Bases: object

Build AlexNet for 3D images or 2D images on image classification

with / without (fixed input size) multiple instance learning (MIL)

Parameters
  • inputs – Tensor of input image (e.g NxCxHxWxD)

  • num_classes – number of classes

  • training – bool variable indicating training or testing

  • num_feat – number of filters or hidden nodes in each layer

  • mil – use multi-instance learning or not

  • use_batch_norm – bool variable indicating batch norm, default True

  • use_group_norm – bool variable indicating group norm, default False

  • use_group_normG – number of groups

  • data_format – channels_first (default) or channels_last

Returns

the output of the AlexNet (MIL)

batch_norm_relu(inputs, alpha=0, scope='batch_norm_scope')
conv(inputs, filters, kernel_size, strides, padding='same', dilation_rate=1, use_bias=None)
conv_transpose(inputs, filters, kernel_size, strides, padding='same', use_bias=None)
kernel_initializer()
loss()
model()
regularizer()
channelsAxis(data_format)
getLen(x)
getNumberOfChannels(x, data_format)
getShape(x)
is3D(x)
isChannelsFirst(data_format)
class ConvBlock(growth_rate, stage, branch, data_format, kernel_regularizer, is_training, data_h5=None, n_spatial_dim=2)

Bases: object

A building block for a dense block.

Parameters
  • growth_rate (float) – growth rate at dense layers

  • stage (int) – index number for this stage

  • branch (int) – index number for this branch

  • data_format (str) – support both channel_first or channel_last

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • data_h5 (dict) – model weights file

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

output tensor for the block

Return type

model_dict

class DenseBlock(blocks, stage, data_format, kernel_regularizer, is_training, data_h5=None, n_spatial_dim=2)

Bases: object

A dense block.

Parameters
  • blocks (int) – the number of building blocks

  • stage (int) – index number for this stage

  • data_format (str) – support both channel_first or channel_last

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • data_h5 (dict) – model weights file

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

output tensor for the block

Return type

model_dict

class DenseNet(blocks, net_config, input_size, kernel_regularizer=None, is_training=False, dtype=tf.float32, n_spatial_dim=2)

Bases: object

Creates a pretrained DenseNet121 model for classification.

Keras densenet121 model with intial weights taken from: https://github.com/flyyufelix/DenseNet-Keras

URL to the h5 file: https://drive.google.com/a/nvidia.com/uc?authuser=0&id=0Byy2AcGyEVxfSTA4SHJVOHNuTXc&export=download

This implementation accepts an hdf5 file as pretrain weights through pretrain_weight_url, and if provided will initialize the network with those weights.

Original source file: https://github.com/keras-team/keras-applications/blob/master/keras_applications/densenet.py,

Apart from NVIDIA contributions, all other code is licensed under the Keras license and copyrighted to the respective Keras contributers.

DenseNet models for Keras

Reference paper: - [Densely Connected Convolutional Networks] (https://arxiv.org/abs/1608.06993) (CVPR 2017 Best Paper Award)

Reference implementation:

  • [Torch DenseNets]

(https://github.com/liuzhuang13/DenseNet/blob/master/models/densenet.lua)

  • [TensorNets]

(https://github.com/taehoonlee/tensornets/blob/master/tensornets/densenets.py)

Parameters
  • blocks (int) – numbers of building blocks for the four dense layers

  • net_config – pretrain_weight_name and pretrain_weight_url configs

  • input_shape (tuple) – input data dimensions

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • dtype (tf.dtype) – reserved argument for future features

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet

Return type

model_dict

class TransitionBlock(reduction, stage, data_format, kernel_regularizer, is_training, data_h5=None, n_spatial_dim=2)

Bases: object

A transition block.

Parameters
  • reduction (float) – compression rate at transition layers

  • stage (int) – index number for this stage

  • data_format (str) – support both channel_first or channel_last

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • data_h5 (dict) – model weights file

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

output tensor for the block

Return type

model_dict

create_densenet121(net_config, input_shape=224, 224, 3, kernel_regularizer=None, is_training=False, n_spatial_dim=2, **kwargs)

Create a pretrained DenseNet121 model for classification.

Parameters
  • net_config – pretrain_weight_name and pretrain_weight_url configs

  • input_shape (tuple) – input data dimensions

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet121

Return type

model_dict

create_densenet169(net_config, input_shape=224, 224, 3, kernel_regularizer=None, is_training=False, n_spatial_dim=2, **kwargs)

Create a pretrained DenseNet169 model for classification.

Parameters
  • net_config – pretrain_weight_name and pretrain_weight_url configs

  • input_shape (tuple) – input data dimensions

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet169

Return type

model_dict

create_densenet201(net_config, input_shape=224, 224, 3, kernel_regularizer=None, is_training=False, n_spatial_dim=2, **kwargs)

Create a pretrained DenseNet201 model for classification.

Parameters
  • net_config – pretrain_weight_name and pretrain_weight_url configs

  • input_shape (tuple) – input data dimensions

  • kernel_regularizer – Regularization method for weights

  • is_training (bool) – in training process or not

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet201

Return type

model_dict

prepare_weights_bn(saved_name, data_h5)

Extract weights of Batch Normalization layers.

Because the original weights were trained using a seperated scale layer for beta and gamma for BN, the saved_name is expected to be a list of two names, the first is the BN layer and the second is the scale layer

Parameters
  • saved_name (str) – name of expected weights

  • data_h5 (dict) – model weights file

Returns

weights for beta, gamma, running_mean, running_std

prepare_weights_conv(saved_name, use_bias, data_format, data_h5)

Extract weights of Convolution layers.

if data_h5 is not none, pulls the conv kernel and bias weights out based on saved_name

Parameters
  • saved_name (str) – name of expected weights

  • use_bias (bool) – use the bias part weights or not

  • data_format (str) – support both channel_first or channel_last

  • data_h5 (dict) – model weights file

Returns

weights for beta, gamma, running_mean, running_std

class DenseNet121Finetune(net_config, num_classes, input_shape=224, 224, 3, is_training=False, kernel_regularizer=None, n_spatial_dim=2)

Bases: object

Create a pretrained DenseNet121 model for classification.

Parameters
  • net_config – pretrain_weight_name and pretrain_weight_url configs

  • num_classes (int) – expected class number of prediction results

  • input_shape (tuple) – input data dimensions

  • is_training (bool) – in training process or not

  • kernel_regularizer – Regularization method for weights

  • n_spatial_dim (int) – spatial dimension count of data, support 2D and 3D data

Returns

a keras model for pretrained DenseNet121: model_dict[‘model’] is the output model_dict[‘variables’] is the list of created variables model_dict[‘pretrain_variables’] is the list of trunk variables model_dict[‘reg_losses’] is the list of reg losses model_dict[‘update_ops’] list of BN update ops

Return type

model_dict

loss()

Get the additional loss function if required.

Parameters

None

Returns

Loss function

class SegResnet(inputs, num_classes, training, blocks_down='1,2,2,4', blocks_up='1,1,1', init_filters=8, use_batch_norm=False, use_group_norm=True, use_group_normG=8, reg_weight=0.0, dropout_prob=0.0, final_activation='softmax', use_vae=False, dtype=tf.float32, data_format='channels_first')

Bases: object

Implementation of ResNet based 3D Segmentation model.

Construct a encoder - decoder structure network based on ResNet. Residual learning framework eases the training of networks that are substantially deeper than those used previously. These residual networks are easier to optimize, and can gain accuracy from considerably increased depth. ResNet won the 1st place on the ILSVRC 2015 classification task.

Parameters
  • inputs (tf.Tensor) – input data for the AHNet model

  • num_classes (int) – expected class number of prediction results

  • training (bool) – in training process or not

  • blocks_down (str) – encoder layer number and ResNet block number of every layer

  • blocks_up (str) – decoder layer number and ResNet block number of every layer

  • init_filters (int) – convolution kernel number of first layer. Default: 8

  • use_batch_norm (bool) – use batch normalization, but only if use_group_norm is explicitly set to false.

  • use_group_norm (bool) – use group normalization or not. By default this is true.

  • use_group_normG (int) – group number of group normalization. Default: 8

  • reg_weight (float) – weight regularization factor

  • dropout_prob (float) – dropout rate for several layers in training

  • final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

  • use_vae (bool) – use “Variational Auto Encoder” or not

  • dtype (tf.dtype) – data type of the network

  • data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

Returns

Prediction results

loss()
model()

Define neural network layers of ResNet based encoder and decoder.

Layers connect with other modules to construct TensorFlow graph.

Parameters

None

Returns

Prediction results

class Unet(inputs, num_classes, training, nf_enc='32,64,64,64', nf_dec='64,64,64,64,64,32,32', use_batch_norm=False, use_group_norm=True, use_group_normG=8, reg_weight=0.0, dropout_prob=0.0, final_activation='softmax', dtype=tf.float32, data_format='channels_first')

Bases: object

3D Segmentation model based on UNet.

For more details, check our implemented UNet in the paper. Wentao Zhu et al., NeurReg: Neural Registration and Its Application to Image Segmentation, WACV 2020. Wentao Zhu et al., Neural Multi-Scale Self-Supervised Registration for Echocardiogram Dense Tracking, arXiv 2019.

Parameters
  • num_classes (int) – expected class number of prediction results

  • nf_enc (str) – number of filters in each encoder layer

  • nf_dec (str) – number of filters in each decoder layer

  • use_batch_norm (bool) – augmented batch normalization

  • use_group_norm (bool) – augmented group normalization

  • use_group_normG (int) – group numbter of group normalization

  • reg_weight (float) – weight regularization factor

  • dropout_prob (float) – dropout rate for several layers in training

  • final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

  • dtype (tf.dtype) – data type of the network

  • data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

Returns

Prediction results

batch_norm_relu(inputs, alpha=0, scope='batch_norm_scope')
conv(inputs, filters, kernel_size, strides, padding='same', dilation_rate=1, use_bias=None)
conv_transpose(inputs, filters, kernel_size, strides, padding='same', use_bias=None)
kernel_initializer()
loss()
model()
regularizer()
channelsAxis(data_format)
getLen(x)
getNumberOfChannels(x, data_format)
getShape(x)
is3D(x)
isChannelsFirst(data_format)
class UnetParallel(inputs, num_classes, training, nf_enc='32,64,64,64', nf_dec='64,64,64,64,64,32,32', gpu_enc='0,0,0,0', gpu_dec='1,1,1,1,1,1,1', use_batch_norm=False, use_group_norm=False, use_group_normG=8, reg_weight=0.0, dropout_prob=0.0, final_activation='softmax', dtype=tf.float32, data_format='channels_first')

Bases: object

3D Segmentation model based on UNet.

For more details, check our implemented UNet in the paper. Wentao Zhu et al., NeurReg: Neural Registration and Its Application to Image Segmentation, WACV 2020. Wentao Zhu et al., Neural Multi-Scale Self-Supervised Registration for Echocardiogram Dense Tracking, arXiv 2019.

Parameters
  • num_classes (int) – expected class number of prediction results

  • nf_enc (str) – number of filters in each encoder layer

  • nf_dec (str) – number of filters in each decoder layer

  • gpu_enc (str) – the gpu id for each encoder layer

  • gpu_dec (str) – the gpu id for each decoder layer

  • use_batch_norm (bool) – augmented batch normalization

  • use_group_norm (bool) – augmented group normalization

  • use_group_normG (int) – group numbter of group normalization

  • reg_weight (float) – weight regularization factor

  • dropout_prob (float) – dropout rate for several layers in training

  • final_activation (str) – final activation layer, supports: softmax, sigmoid, linear

  • dtype (tf.dtype) – data type of the network

  • data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

Returns

Prediction results

batch_norm_relu(inputs, alpha=0.0, scope='batch_norm_scope')
conv(inputs, filters, kernel_size, strides, padding='same', dilation_rate=1, use_bias=None)
conv_transpose(inputs, filters, kernel_size, strides, padding='same', use_bias=None)
kernel_initializer()
loss()
model()
regularizer()
up_sampling(inputs, size=2, interp='bilinear')
channelsAxis(data_format)
getLen(x)
getNumberOfChannels(x, data_format)
getShape(x)
is3D(x)
isChannelsFirst(data_format)
batch_norm(axis=3, beta_weights=None, gamma_weights=None, mean_weights=None, std_weights=None, **kwargs)

Define batch normalization layer.

Parameters
  • axis (int) – compute Batch Normalization on axis dimension

  • beta_weights (Tensor) – weights for beta factor in batch normalization

  • gamma_weights (Tensor) – weights for gamma factor in batch normalization

  • mean_weights (Tensor) – weights for mean factor in batch normalization

  • std_weights (Tensor) – weights for std factor in batch normalization

Returns

batch normalization layer

Return type

layer

conv_layer(out_channels, kernel_size, strides, name, use_bias=True, data_format='channels_last', kernel_regularizer=None, kernel_weights=None, bias_weights=None, kernel_initializer=None, bias_initializer=None, n_spatial_dim=2, **kwargs)

Define convolution layer for 2D or 3D use case.

Parameters
  • out_channels (int) – channel number of output tensor

  • kernel_size (tuple) – size of convolution filter kernel

  • strides (tuple) – strides of convolution computation

  • name (name) – specified layer name

  • use_bias (bool) – use the bias of convolution filter or not

  • data_format (str) – input and output data dimensions supports CHANNEL_FIRST and CHANNEL_LAST

  • kernel_regularizer – Regularization method for weights

  • kernel_weights (Tensor) – weights to initialize filter kernel

  • bias_weights (Tensor) – weights to initialize filter bias

  • kernel_initializer – initialize filter kernel with weights

  • bias_initializer – initialize filter bias with weights

  • n_spatial_dim (int) – spatial dimension number

Returns

2D or 3D convolution layer

Return type

layer

get_pretrain_weights(pretrain_weight_name, url_list)

Load pretrained weights from file or download from URL.

Parameters
  • pretrain_weight_name (str) – path to load or store pretrained weights

  • url (str) – link to download pretrained weights

Returns

pretrained weights file

pull_keras_losses(layer, out_dict)

Pull loss from layer

Parameters
  • layer (tensorflow.keras.layers) – specified layer to pull

  • model_dict (dict) – target dict to store data

Returns

dict with pulled results

Return type

model_dict

pull_keras_updates(layer, out_dict)

Pull update_ops from layer

Parameters
  • layer (tensorflow.keras.layers) – specified layer to pull

  • model_dict (dict) – target dict to store data

Returns

dict with pulled results

Return type

model_dict

pull_keras_weights(layer, out_dict)

Pull weights from layer

Parameters
  • layer (tensorflow.keras.layers) – specified layer to pull

  • model_dict (dict) – target dict to store data

Returns

dict with pulled results

Return type

model_dict

run_layer_pull_all(layer, model_dict, **kwargs)

Pull model, weights, update_ops, loss from layer.

Parameters
  • layer (tensorflow.keras.layers) – specified layer to pull

  • model_dict (dict) – target dict to store data

Returns

dict with pulled results

Return type

model_dict

© Copyright 2020, NVIDIA. Last updated on Feb 2, 2023.