Action Specs

classification

evaluate

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

train_config

collection

train_config.train_dataset_path

hidden

train_config.val_dataset_path

hidden

train_config.pretrained_model_path

hidden

train_config.optimizer

collection

train_config.optimizer.sgd

collection

One of SGD / ADAM / RMSPROP

train_config.optimizer.sgd.lr

float

0.01

train_config.optimizer.sgd.decay

float

0

train_config.optimizer.sgd.momentum

float

0.9

train_config.optimizer.sgd.nesterov

bool

FALSE

train_config.optimizer.adam

collection

train_config.optimizer.adam.lr

float

train_config.optimizer.adam.beta_1

float

train_config.optimizer.adam.beta_2

float

train_config.optimizer.adam.epsilon

float

train_config.optimizer.adam.decay

float

train_config.optimizer.rmsprop

collection

train_config.optimizer.rmsprop.lr

float

train_config.optimizer.rmsprop.rho

float

train_config.optimizer.rmsprop.epsilon

float

train_config.optimizer.rmsprop.decay

float

train_config.batch_size_per_gpu

integer

256

train_config.n_epochs

integer

80

train_config.n_workers

integer

2

train_config.reg_config

collection

train_config.reg_config.type

string

L2

train_config.reg_config.scope

string

Conv2D,Dense

train_config.reg_config.weight_decay

float

0.00005

train_config.lr_config

collection

ONE OF STEP / SOFT_ANNEAL / COSINE

train_config.lr_config.step

collection

train_config.lr_config.step.learning_rate

float

train_config.lr_config.step.step_size

integer

train_config.lr_config.step.gamma

float

train_config.lr_config.soft_anneal

collection

train_config.lr_config.soft_anneal.learning_rate

float

0.05

train_config.lr_config.soft_anneal.soft_start

float

0.056

train_config.lr_config.soft_anneal.annealing_divider

float

10

train_config.lr_config.soft_anneal.annealing_points

list

List of float

[0.3,0.6,0.8]

train_config.lr_config.cosine

collection

train_config.lr_config.cosine.learning_rate

float

train_config.lr_config.cosine.min_lr_ratio

float

train_config.lr_config.cosine.soft_start

float

train_config.random_seed

integer

42

train_config.enable_random_crop

bool

train_config.enable_center_crop

bool

train_config.enable_color_augmentation

bool

train_config.label_smoothing

float

train_config.preprocess_mode

string

caffe

train_config.mixup_alpha

float

train_config.model_parallelism

list

train_config.image_mean

collection

train_config.image_mean.key

string

train_config.image_mean.value

float

train_config.disable_horizontal_flip

bool

train_config.visualizer_config

collection

train_config.visualizer

Visualizer

collection

train_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

train_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

eval_config

collection

eval_config.top_k

integer

3

eval_config.eval_dataset_path

hidden

eval_config.model_path

hidden

eval_config.batch_size

integer

256

eval_config.n_workers

integer

2

eval_config.enable_center_crop

bool

TRUE

model_config

collection

model_config.arch

string

resnet

model_config.input_image_size

string

3,224,224

yes

yes

model_config.resize_interpolation_method

string

__BILINEAR__, __BICUBIC__

model_config.n_layers

integer

18

model_config.retain_head

bool

FALSE

model_config.use_batch_norm

bool

TRUE

model_config.use_bias

bool

model_config.use_pooling

bool

model_config.all_projections

bool

TRUE

model_config.freeze_bn

bool

model_config.freeze_blocks

integer

model_config.dropout

float

1.00E-03

model_config.batch_norm_config

collection

model_config.batch_norm_config.momentum

float

model_config.batch_norm_config.epsilon

float

model_config.activation

collection

model_config.activation.activation_type

string

model_config.activation.activation_parameters

collection

model_config.activation.activation_parameters.key

string

model_config.activation.activation_parameters.value

float

export

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

version

Schema Version

const

The version of this schema

1

model

Model

hidden

UNIX path to the model file

0.1

yes

key

Encryption Key

hidden

Encryption key

tlt_encode

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

force_ptq

Force Post-Training Quantization

bool

Force generating int8 engine using Post Training Quantization

FALSE

no

cal_image_dir

hidden

data_type

Pruning Granularity

string

Number of filters to remove at a time.

fp32

int8, fp32, fp16

yes

yes

strict_type_constraints

bool

FALSE

gen_ds_config

bool

FALSE

cal_cache_file

Calibration cache file

hidden

Unix PATH to the int8 calibration cache file

yes

yes

batches

Number of calibration batches

integer

Number of batches to calibrate the model when run in INT8 mode

100

max_workspace_size

integer

Example: The integer value of 1<<30, 2<<30

max_batch_size

integer

1

batch_size

Batch size

integer

Number of images per batch when generating the TensorRT engine.

100

yes

min_batch_size

integer

1

opt_batch_size

integer

1

experiment_spec

Experiment Spec

hidden

UNIX path to the Experiment spec file used to train the model. This may be the train or retrain spec file.

yes

engine_file

Engine File

hidden

UNIX path to the model engine file.

yes

static_batch_size

integer

-1

results_dir

hidden

verbose

hidden

TRUE

classmap_json

hidden

is_byom

bool

FALSE

inference

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

cli

batch_size

Batch Size

integer

Batch size CLI parameter

1

yes

train_config.train_dataset_path

hidden

train_config.val_dataset_path

hidden

train_config.pretrained_model_path

hidden

train_config.optimizer

collection

train_config.optimizer.sgd

collection

One of SGD / ADAM / RMSPROP

train_config.optimizer.sgd.lr

float

0.01

train_config.optimizer.sgd.decay

float

0

train_config.optimizer.sgd.momentum

float

0.9

train_config.optimizer.sgd.nesterov

bool

FALSE

train_config

collection

train_config.optimizer.adam

collection

train_config.optimizer.adam.lr

float

train_config.optimizer.adam.beta_1

float

train_config.optimizer.adam.beta_2

float

train_config.optimizer.adam.epsilon

float

train_config.optimizer.adam.decay

float

train_config.optimizer.rmsprop

collection

train_config.optimizer.rmsprop.lr

float

train_config.optimizer.rmsprop.rho

float

train_config.optimizer.rmsprop.epsilon

float

train_config.optimizer.rmsprop.decay

float

train_config.batch_size_per_gpu

integer

256

train_config.n_epochs

integer

80

train_config.n_workers

integer

2

train_config.reg_config

collection

train_config.reg_config.type

string

L2

train_config.reg_config.scope

string

Conv2D,Dense

train_config.reg_config.weight_decay

float

0.00005

train_config.lr_config

collection

ONE OF STEP / SOFT_ANNEAL / COSINE

train_config.lr_config.step

collection

train_config.lr_config.step.learning_rate

float

train_config.lr_config.step.step_size

integer

train_config.lr_config.step.gamma

float

train_config.lr_config.soft_anneal

collection

train_config.lr_config.soft_anneal.learning_rate

float

0.05

train_config.lr_config.soft_anneal.soft_start

float

0.056

train_config.lr_config.soft_anneal.annealing_divider

float

10

train_config.lr_config.soft_anneal.annealing_points

list

List of float

[0.3,0.6,0.8]

train_config.lr_config.cosine

collection

train_config.lr_config.cosine.learning_rate

float

train_config.lr_config.cosine.min_lr_ratio

float

train_config.lr_config.cosine.soft_start

float

train_config.random_seed

integer

42

train_config.enable_random_crop

bool

train_config.enable_center_crop

bool

train_config.enable_color_augmentation

bool

train_config.label_smoothing

float

train_config.preprocess_mode

string

caffe

train_config.mixup_alpha

float

train_config.model_parallelism

list

train_config.image_mean

collection

train_config.image_mean.key

string

train_config.image_mean.value

float

train_config.disable_horizontal_flip

bool

train_config.visualizer_config

collection

train_config.visualizer

Visualizer

collection

train_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

train_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

eval_config

collection

eval_config.top_k

integer

3

eval_config.eval_dataset_path

hidden

eval_config.model_path

hidden

eval_config.batch_size

integer

256

eval_config.n_workers

integer

2

eval_config.enable_center_crop

bool

TRUE

model_config

collection

model_config.arch

string

resnet

model_config.input_image_size

string

3,224,224

yes

yes

model_config.resize_interpolation_method

string

__BILINEAR__, __BICUBIC__

model_config.n_layers

integer

18

model_config.retain_head

bool

FALSE

model_config.use_batch_norm

bool

TRUE

model_config.use_bias

bool

model_config.use_pooling

bool

model_config.all_projections

bool

TRUE

model_config.freeze_bn

bool

model_config.freeze_blocks

integer

model_config.dropout

float

1.00E-03

model_config.batch_norm_config

collection

model_config.batch_norm_config.momentum

float

model_config.batch_norm_config.epsilon

float

model_config.activation

collection

model_config.activation.activation_type

string

model_config.activation.activation_parameters

collection

model_config.activation.activation_parameters.key

string

model_config.activation.activation_parameters.value

float

train

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

automl_enabled

math_cond

parent_param

depends_on

eval_config

collection

FALSE

eval_config.batch_size

integer

256

1

inf

eval_config.enable_center_crop

bool

TRUE

eval_config.eval_dataset_path

hidden

FALSE

eval_config.model_path

hidden

FALSE

eval_config.n_workers

integer

2

1

inf

FALSE

eval_config.top_k

integer

3

1

inf

init_epoch

hidden

CLI Parameter initial epoch

1

1

inf

FALSE

model_config

collection

FALSE

model_config.activation

collection

FALSE

model_config.activation.activation_parameters

collection

FALSE

model_config.activation.activation_parameters.key

string

FALSE

model_config.activation.activation_parameters.value

float

FALSE

model_config.activation.activation_type

ordered

relu,swish

model_config.all_projections

bool

TRUE

model_config.arch

ordered

resnet

resnet,efficientnet_b0,efficientnet_b1,efficientnet_b2,efficientnet_b3,efficientnet_b4,efficientnet_b5,efficientnet_b6,efficientnet_b5,mobilenet_v1,mobilenet_v2,googlenet,alexnet,darknet,cspdarknet,cspdarknet_tiny,vgg,squeezenet,byom

FALSE

model_config.batch_norm_config

collection

FALSE

model_config.batch_norm_config.epsilon

float

1.00E-05

1.00E-10

inf

model_config.batch_norm_config.momentum

float

0.9

1.00E-05

inf

model_config.dropout

float

1.00E-03

0

1

model_config.freeze_blocks

integer

FALSE

model_config.freeze_bn

bool

model_config.input_image_size

string

3,224,224

yes

yes

FALSE

model_config.n_layers

integer

18

FALSE

model_config.resize_interpolation_method

ordered

__BILINEAR__,__BICUBIC__

model_config.retain_head

bool

FALSE

model_config.use_batch_norm

bool

TRUE

model_config.use_bias

bool

FALSE

model_config.use_pooling

bool

train_config

collection

FALSE

train_config.batch_size_per_gpu

integer

64

1

inf

train_config.disable_horizontal_flip

bool

train_config.enable_center_crop

bool

TRUE

train_config.enable_color_augmentation

bool

TRUE

train_config.enable_random_crop

bool

TRUE

train_config.image_mean

collection

FALSE

train_config.image_mean.key

string

FALSE

train_config.image_mean.value

float

FALSE

train_config.label_smoothing

float

0

0

1

train_config.lr_config

collection

ONE OF STEP / SOFT_ANNEAL / COSINE

FALSE

train_config.lr_config.cosine

collection

FALSE

train_config.lr_config.cosine.learning_rate

float

0

inf

FALSE

train_config.lr_config.cosine.min_lr_ratio

float

0

1

train_config.lr_config.cosine.soft_start

float

0

1

train_config.lr_config.soft_anneal

collection

0

1

FALSE

train_config.lr_config.soft_anneal.annealing_divider

float

1

inf

FALSE

train_config.lr_config.soft_anneal.annealing_points

list

List of float

FALSE

train_config.lr_config.soft_anneal.learning_rate

float

0

1

train_config.lr_config.soft_anneal.soft_start

float

0

1

train_config.lr_config.step

collection

FALSE

train_config.lr_config.step.gamma

float

0.1

1.00E-10

1

train_config.lr_config.step.learning_rate

float

0.06

0

1

TRUE

train_config.lr_config.step.step_size

integer

10

1

inf

train_config.mixup_alpha

float

0.1

0

1

train_config.model_parallelism

list

FALSE

train_config.n_epochs

integer

80

1

inf

FALSE

train_config.n_workers

integer

2

1

inf

FALSE

train_config.optimizer

collection

One of SGD / ADAM / RMSPROP

FALSE

train_config.optimizer.sgd

collection

FALSE

train_config.optimizer.sgd.decay

float

0

FALSE

train_config.optimizer.sgd.lr

float

0.01

0

inf

TRUE

train_config.optimizer.sgd.momentum

float

0.9

1.00E-10

0.99

train_config.optimizer.sgd.nesterov

bool

FALSE

TRUE

train_config.preprocess_mode

ordered

caffe

caffe,torch,tf

train_config.pretrained_model_path

hidden

FALSE

train_config.random_seed

integer

42

1

inf

FALSE

train_config.reg_config

collection

FALSE

train_config.reg_config.scope

string

Conv2D,Dense

FALSE

train_config.reg_config.type

ordered

L2

L1,L2

TRUE

train_config.reg_config.weight_decay

float

0.00005

3.00E-11

0.003

TRUE

train_config.train_dataset_path

hidden

FALSE

train_config.val_dataset_path

hidden

FALSE

train_config.visualizer

Visualizer

collection

FALSE

train_config.visualizer_config

collection

FALSE

train_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

FALSE

train_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

FALSE

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

e

engine file path

hidden

k

encode key

hidden

c

cache_file

hidden

o

outputs

string

comma separated list of output node names

d

input_dims

string

comma separated list of input dimensions (not required for TLT 3.0 new models).

yes

yes

b

batch_size

integer

calibration batch size

8

yes

m

max_batch_size

integer

maximum TensorRT engine batch size (default 16). If meet with out-of-memory issue, please decrease the batch size accordingly.

16

yes

w

max_workspace_size

integer

maximum workspace size of TensorRT engine (default 1<<30). If meet with out-of-memory issue, please increase the workspace size accordingly.

t

data_type

string

TensorRT data type

fp32

fp32, fp16, int8

yes

i

input_order

string

input dimension ordering

nchw

nchw, nhwc, nc

s

strict_type_constraints

bool

TensorRT strict_type_constraints flag for INT8 mode

FALSE

u

dla_core

int

Use DLA core N for layers that support DLA (default = -1, which means no DLA core will be utilized for inference. Note that it’ll always allow GPU fallback).

-1

p

parse_profile_shapes

string

comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape>, where each shape has x as delimiter, e.g.,NxC, NxCxHxW, NxCxDxHxW, etc. Can be specified multiple times if there are multiple input tensors for the model. This argument is only useful in dynamic shape case.

model

etlt model from export

hidden

prune

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

model

Model path

hidden

UNIX path to where the input model is located.

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

results_dir

Results directory

hidden

key

Encode key

hidden

normalizer

Normalizer

string

How to normalize

max

max, L2

equalization_criterion

Equalization Criterion

string

Criteria to equalize the stats of inputs to an element wise op layer.

union

union, intersection, arithmetic_mean,geometric_mean

no

pruning_granularity

Pruning Granularity

integer

Number of filters to remove at a time.

8

no

pruning_threshold

Pruning Threshold

float

Threshold to compare normalized norm against.

0.1

0

1

yes

yes

min_num_filters

Minimum number of filters

integer

Minimum number of filters to be kept per layer

16

no

excluded_layers

Excluded layers

string

string of list: List of excluded_layers. Examples: -i item1 item2

verbose

verbosity

hidden

TRUE

retrain

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

init_epoch

hidden

CLI Parameter initial epoch

1

train_config

collection

train_config.train_dataset_path

hidden

train_config.val_dataset_path

hidden

train_config.pretrained_model_path

hidden

train_config.optimizer

collection

train_config.optimizer.sgd

collection

One of SGD / ADAM / RMSPROP

train_config.optimizer.sgd.lr

float

0.01

train_config.optimizer.sgd.decay

float

0

train_config.optimizer.sgd.momentum

float

0.9

train_config.optimizer.sgd.nesterov

bool

FALSE

train_config.optimizer.adam

collection

train_config.optimizer.adam.lr

float

train_config.optimizer.adam.beta_1

float

train_config.optimizer.adam.beta_2

float

train_config.optimizer.adam.epsilon

float

train_config.optimizer.adam.decay

float

train_config.optimizer.rmsprop

collection

train_config.optimizer.rmsprop.lr

float

train_config.optimizer.rmsprop.rho

float

train_config.optimizer.rmsprop.epsilon

float

train_config.optimizer.rmsprop.decay

float

train_config.batch_size_per_gpu

integer

64

train_config.n_epochs

integer

80

train_config.n_workers

integer

2

train_config.reg_config

collection

train_config.reg_config.type

string

L2

train_config.reg_config.scope

string

Conv2D,Dense

train_config.reg_config.weight_decay

float

0.00005

train_config.lr_config

collection

ONE OF STEP / SOFT_ANNEAL / COSINE

train_config.lr_config.step

collection

train_config.lr_config.step.learning_rate

float

0.006

train_config.lr_config.step.step_size

integer

10

train_config.lr_config.step.gamma

float

0.1

train_config.lr_config.soft_anneal

collection

train_config.lr_config.soft_anneal.learning_rate

float

train_config.lr_config.soft_anneal.soft_start

float

train_config.lr_config.soft_anneal.annealing_divider

float

train_config.lr_config.soft_anneal.annealing_points

list

List of float

train_config.lr_config.cosine

collection

train_config.lr_config.cosine.learning_rate

float

train_config.lr_config.cosine.min_lr_ratio

float

train_config.lr_config.cosine.soft_start

float

train_config.random_seed

integer

42

train_config.enable_random_crop

bool

TRUE

train_config.enable_center_crop

bool

TRUE

train_config.enable_color_augmentation

bool

train_config.label_smoothing

float

0

train_config.preprocess_mode

string

caffe

train_config.mixup_alpha

float

0.1

train_config.model_parallelism

list

train_config.image_mean

collection

train_config.image_mean.key

string

train_config.image_mean.value

float

train_config.disable_horizontal_flip

bool

train_config.visualizer_config

collection

train_config.visualizer

Visualizer

collection

train_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

train_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

eval_config

collection

eval_config.top_k

integer

3

eval_config.eval_dataset_path

hidden

eval_config.model_path

hidden

eval_config.batch_size

integer

256

eval_config.n_workers

integer

2

eval_config.enable_center_crop

bool

TRUE

model_config

collection

model_config.arch

string

resnet

model_config.input_image_size

string

3,224,224

yes

yes

model_config.resize_interpolation_method

string

__BILINEAR__, __BICUBIC__

model_config.n_layers

integer

18

model_config.retain_head

bool

FALSE

model_config.use_batch_norm

bool

TRUE

model_config.use_bias

bool

model_config.use_pooling

bool

model_config.all_projections

bool

TRUE

model_config.freeze_bn

bool

model_config.freeze_blocks

integer

model_config.dropout

float

1.00E-03

model_config.batch_norm_config

collection

model_config.batch_norm_config.momentum

float

model_config.batch_norm_config.epsilon

float

model_config.activation

collection

model_config.activation.activation_type

string

model_config.activation.activation_parameters

collection

model_config.activation.activation_parameters.key

string

model_config.activation.activation_parameters.value

float

detectnet_v2

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

e

engine file path

hidden

k

encode key

hidden

c

cache_file

hidden

o

outputs

string

comma separated list of output node names

d

input_dims

string

comma separated list of input dimensions (not required for TLT 3.0 new models).

yes

yes

b

batch_size

integer

calibration batch size

8

yes

m

max_batch_size

integer

maximum TensorRT engine batch size (default 16). If meet with out-of-memory issue, please decrease the batch size accordingly.

16

yes

w

max_workspace_size

integer

maximum workspace size of TensorRT engine (default 1<<30). If meet with out-of-memory issue, please increase the workspace size accordingly.

t

data_type

string

TensorRT data type

fp32

fp32, fp16, int8

yes

i

input_order

string

input dimension ordering

nchw

nchw, nhwc, nc

s

strict_type_constraints

bool

TensorRT strict_type_constraints flag for INT8 mode

FALSE

u

dla_core

int

Use DLA core N for layers that support DLA (default = -1, which means no DLA core will be utilized for inference. Note that it’ll always allow GPU fallback).

-1

p

parse_profile_shapes

string

comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape>, where each shape has x as delimiter, e.g.,NxC, NxCxHxW, NxCxDxHxW, etc. Can be specified multiple times if there are multiple input tensors for the model. This argument is only useful in dynamic shape case.

model

etlt model from export

hidden

evaluate

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

popular

regex

version

Schema Version

const

The version of this schema

1

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

dataset_config

Dataset

collection

Parameters to configure the dataset

dataset_config.image_extension

Image Extension

string

Extension of the images to be used.

png

png, jpg, jpeg

yes

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

dataset_config.validation_data_source.tfrecords_path

Validation TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

dataset_config.validation_data_source.image_directory_path

Validation Image Path

hidden

/shared/users/1234/datasets/5678/training

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords, you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0, N-1]. For random split partitioning, force the validation fold index to 0 as the tfrecord is just 2-fold.

0

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

480

yes

Yes

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

272

yes

Yes

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

1

0

yes

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

1

0

yes

augmentation_config.preprocessing.output_image_channel

Image Channel

integer

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently, 1-channel input is not recommended for datasets with JPG images. For PNG images, both 3-channel RGB and 1-channel monochrome images are supported.

3

1, 3

yes

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

no

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

no

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

1

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max, rotate_rad_max].

0

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

1

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally, this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0, you can set this value to 0.5.

0.5

0.5

bbox_rasterizer_config

Bounding box rasterizer

collection

Collection of parameters to configure the bounding box rasterizer

bbox_rasterizer_config.deadzone_radius

Bounding box rasterizer deadzone radius

float

0.4

0

1

yes

model_config

Model

collection

model_config.arch

BackBone Architecture

string

The architecture of the backbone feature extractor to be used for training.

resnet

resnet

yes

model_config.pretrained_model_file

PTM File Path

hidden

This parameter defines the path to a pretrained TLT model file. If the load_graph flag is set to false, it is assumed that only the weights of the pretrained model file is to be used. In this case, TLT train constructs the feature extractor graph in the experiment and loads the weights from the pretrained model file that has matching layer names. Thus, transfer learning across different resolutions and domains are supported. For layers that may be absent in the pretrained model, the tool initializes them with random weights and skips the import for that layer.

/shared/.pretrained/resnet18/detectnet_v2_vresnet18/resnet18.hdf5

model_config.load_graph

PTM Load Graph

bool

A flag to determine whether or not to load the graph from the pretrained model file, or just the weights. For a pruned model, set this parameter to True. Pruning modifies the original graph, so the pruned model graph and the weights need to be imported.

FALSE

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template, and is different for different feature extractor templates.

0

3

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

model_config.all_projections

All Projections

bool

For templates with shortcut connections, this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers, irrespective of whether there is a change in stride across the input and output.

model_config.num_layers

Number of Layers

integer

The depth of the feature extractor for scalable templates.

18

10, 18, 34, 50, 101

yes

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True, MaxPooling is used to downsample; however, for the object-detection network, NVIDIA recommends setting this to False and using strided convolutions.

model_config.use_batch_norm

Use Batch Normalization

bool

A flag to determine whether to use Batch Normalization layers or not.

TRUE

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

1

model_config.training_precision.backend_floatx

Backend Training Precision

string

A nested parameter that sets the precision of the backend training framework.

__FLOAT32__

no

model_config.objective_set.cov

Objective COV

collection

The objectives for training the network. For object-detection networks, set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

{}

yes

model_config.objective_set.bbox.scale

Objective Bounding Box Scale

float

The objectives for training the network. For object-detection networks, set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

35

yes

model_config.objective_set.bbox.offset

Objective Bounding Box Offset

float

The objectives for training the network. For object-detection networks, set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

0.5

yes

training_config

Training

collection

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

4

1

yes

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

120

1

yes

Yes

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-06

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

5.00E-04

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.100000001

0

1

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.699999988

0

1

yes

Yes

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__NO_REG__, __L1__, __L2__

yes

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

yes

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

yes

training_config.optimizer.adam.beta1

Optimizer Adam Beta1

float

0.899999976

yes

training_config.optimizer.adam.beta2

Optimizer Adam Beta2

float

0.999000013

yes

training_config.cost_scaling.enabled

Enable Cost Scaling

bool

Enables cost scaling during training.

FALSE

yes

training_config.cost_scaling.initial_exponent

Cost Scaling Initial Exponent

float

20

yes

training_config.cost_scaling.increment

Cost Scaling Increment

float

0.005

yes

training_config.cost_scaling.decrement

Cost Scaling Decrement

float

1

yes

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

10

0

yes

evaluation_config

Evaluation

collection

yes

evaluation_config.average_precision_mode

Average Precision Mode

string

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__, __INTEGRATE__

evaluation_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

1

yes

evaluation_config.first_validation_epoch

First Validation Epoch

integer

The first epoch to start running validation. Ideally it is preferred to wait for at least 20-30% of the total number of epochs before starting evaluation, since the predictions in the initial epochs would be fairly inaccurate. Too many candidate boxes may be sent to clustering and this can cause the evaluation to slow down.

30

1

yes

cost_function_config

Cost function

collection

cost_function_config.enable_autoweighting

Auto-Weighting

bool

TRUE

yes

cost_function_config.max_objective_weight

Maximum Objective Weight

float

0.999899983

cost_function_config.min_objective_weight

Minimum Objective Weight

float

1.00E-04

classwise_config

Class-wise organized parameters

list

classwise_config.key

Class Key

string

Name of class for the classwise parameters

person

classwise_config.value.evaluation_config

Evaluation config elements per class

collection

classwise_config.value.evaluation_config.minimum_detection_ground_truth_overlap

Minimum Detection Ground Truth Overlaps

float

Minimum IOU between ground truth and predicted box after clustering to call a valid detection. This parameter is a repeatable dictionary and a separate one must be defined for every class.

0.5

0

1

yes

classwise_config.value.evaluation_config.evaluation_box_config.minimum_height

Minimum Height

integer

Minimum height in pixels for a valid ground truth and prediction bbox.

20

0

yes

classwise_config.value.evaluation_config.evaluation_box_config.maximum_height

Maximum Height

integer

Maximum height in pixels for a valid ground truth and prediction bbox.

9999

0

yes

classwise_config.value.evaluation_config.evaluation_box_config.minimum_width

Minimum Width

integer

Minimum width in pixels for a valid ground truth and prediction bbox.

10

0

yes

classwise_config.value.evaluation_config.evaluation_box_config.maximum_width

Maximum Width

integer

Maximum width in pixels for a valid ground truth and prediction bbox.

9999

0

yes

classwise_config.value.cost_function_config

Class-wise cost fuction config per class

collection

yes

classwise_config.value.cost_function_config.class_weight

Class Weight

float

4

yes

classwise_config.value.cost_function_config.coverage_foreground_weight

Coverage Forground Weight

float

0.050000001

yes

classwise_config.value.cost_function_config.objectives

Objectives

list

[{“name”: “cov”, “initial_weight”: 1.0, “weight_target”: 1.0}, {“name”: “bbox”, “initial_weight”: 10.0, “weight_target”: 10.0}]

yes

classwise_config.value.cost_function_config.objectives.name

Objective Name

string

Objective name such as cov or bbox.

cov

yes

classwise_config.value.cost_function_config.objectives.initial_weight

Initial Weight

float

Initial weight for named objective.

1

yes

classwise_config.value.cost_function_config.objectives.weight_target

Weight Target

float

Target weight for named objective.

1

yes

classwise_config.value.bbox_rasterizer_config

Rasterization

collection

yes

classwise_config.value.bbox_rasterizer_config.cov_center_x

Center of Object X-Coordinate

float

x-coordinate of the center of the object

0.5

0

1

yes

classwise_config.value.bbox_rasterizer_config.cov_center_y

Center of Object Y-Coordinate

float

y-coordinate of the center of the object

0.5

0

1

yes

classwise_config.value.bbox_rasterizer_config.cov_radius_x

Center of Object X-Radius

float

x-radius of the coverage ellipse

1

0

1

yes

classwise_config.value.bbox_rasterizer_config.cov_radius_y

Center of Object Y-Radius

float

y-radius of the coverage ellipse

1

0

1

yes

classwise_config.value.bbox_rasterizer_config.bbox_min_radius

Bounding Box Minimum Radius

float

The minimum radius of the coverage region to be drawn for boxes

1

0

1

yes

classwise_config.postprocessing_config

Post-Processing

collection

classwise_config.postprocessing_config.clustering_config.coverage_threshold

Coverage Threshold

float

The minimum threshold of the coverage tensor output to be considered a valid candidate box for clustering. The four coordinates from the bbox tensor at the corresponding indices are passed for clustering.

0.0075

0

1

yes

classwise_config.postprocessing_config.clustering_config.dbscan_eps

DBSCAN Samples Distance

float

The maximum distance between two samples for one to be considered in the neighborhood of the other. This is not a maximum bound on the distances of points within a cluster. The greater the dbscan_eps value, the more boxes are grouped together.

0.230000004

0

1

yes

classwise_config.postprocessing_config.clustering_config.dbscan_min_samples

DBSCAN Minimum Samples

float

The total weight in a neighborhood for a point to be considered as a core point. This includes the point itself.

0.050000001

0

1

yes

classwise_config.postprocessing_config.clustering_config.minimum_bounding_box_height

Minimum Bounding Box Height

integer

The minimum height in pixels to consider as a valid detection post clustering.

20

0

10000

yes

classwise_config.postprocessing_config.clustering_config.clustering_algorithm

Clustering Algorithm

string

Defines the post-processing algorithm to cluter raw detections to the final bbox render. When using HYBRID mode, ensure both DBSCAN and NMS configuration parameters are defined.

__DBSCAN__

__DBSCAN__, __NMS__, __HYBRID__

yes

classwise_config.postprocessing_config.clustering_config.dbscan_confidence_threshold

DBSCAN Confidence Threshold

float

The confidence threshold used to filter out the clustered bounding box output from DBSCAN.

0.1

0.1

yes

classwise_config.postprocessing_config.clustering_config.nms_iou_threshold

NMS IOU Threshold

float

The Intersection Over Union (IOU) threshold to filter out redundant boxes from raw detections to form final clustered outputs.

0.2

0

1

classwise_config.postprocessing_config.clustering_config.nms_confidence_threshold

NMS Confidence Threshold

float

The confidence threshold to filter out clustered bounding boxes from NMS.

0

0

1

export

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

version

Schema Version

const

The version of this schema

1

model

Model

hidden

UNIX path to the model file

0.1

yes

key

Encryption Key

hidden

Encryption key

tlt_encode

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

force_ptq

Force Post-Training Quantization

bool

Force generating int8 engine using Post Training Quantization

FALSE

no

cal_image_dir

hidden

data_type

Pruning Granularity

string

Number of filters to remove at a time.

fp32

int8, fp32, fp16

yes

yes

strict_type_constraints

bool

FALSE

gen_ds_config

bool

FALSE

cal_cache_file

Calibration cache file

hidden

Unix PATH to the int8 calibration cache file

yes

yes

batches

Number of calibration batches

integer

Number of batches to calibrate the model when run in INT8 mode

100

no

max_workspace_size

integer

Example: The integer value of 1<<30, 2<<30

max_batch_size

integer

1

batch_size

Batch size

integer

Number of images per batch when generating the TensorRT engine.

100

yes

min_batch_size

integer

1

opt_batch_size

integer

1

experiment_spec

Experiment Spec

hidden

UNIX path to the Experiment spec file used to train the model. This may be the train or retrain spec file.

yes

engine_file

Engine File

hidden

UNIX path to the model engine file.

yes

static_batch_size

integer

-1

results_dir

hidden

verbose

hidden

TRUE

inference

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

param_type (internal / hidden / inferred)

CLI

inferencer_config

collection

inferencer_config.tlt_config

collection

inferencer_config.tlt_config.model

hidden

inferencer_config.tensorrt_config

collection

inferencer_config.tensorrt_config.parser

integer

0,1,2

inferencer_config.tensorrt_config.backend_data_type

integer

0,1,2

inferencer_config.tensorrt_config.save_engine

bool

inferencer_config.tensorrt_config.trt_engine

hidden

inferencer_config.tensorrt_config.calibrator_config

collection

inferencer_config.input_nodes

list

list of string

inferencer_config.output_nodes

list

list of string

inferencer_config.batch_size

integer

16

inferencer_config.image_height

integer

544

inferencer_config.image_width

integer

960

inferencer_config.image_channels

integer

3

inferencer_config.gpu_index

integer

0

inferencer_config.target_classes

list

list of string

[“car”]

yes

yes

inferencer_config.stride

integer

bbox_handler_config

collection

bbox_handler_config.kitti_dump

bool

TRUE

bbox_handler_config.disable_overlay

bool

FALSE

bbox_handler_config.overlay_linewidth

integer

2

bbox_handler_config.classwise_bbox_handler_config

list

yes

yes

bbox_handler_config.classwise_bbox_handler_config.key

string

default

bbox_handler_config.classwise_bbox_handler_config.value

collection

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config

collection

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.coverage_threshold

Coverage Threshold

float

The minimum threshold of the coverage tensor output to be considered a valid candidate box for clustering. The four coordinates from the bbox tensor at the corresponding indices are passed for clustering.

0.005

0

1

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.dbscan_eps

DBSCAN Samples Distance

float

The maximum distance between two samples for one to be considered in the neighborhood of the other. This is not a maximum bound on the distances of points within a cluster. The greater the dbscan_eps value, the more boxes are grouped together.

0.3

0

1

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.dbscan_min_samples

DBSCAN Minimum Samples

float

The total weight in a neighborhood for a point to be considered as a core point. This includes the point itself.

0.05

0

1

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.minimum_bounding_box_height

Minimum Bounding Box Height

integer

The minimum height in pixels to consider as a valid detection post clustering.

4

0

10000

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.clustering_algorithm

Clustering Algorithm

string

Defines the post-processing algorithm to cluter raw detections to the final bbox render. When using HYBRID mode, ensure both DBSCAN and NMS configuration parameters are defined.

__DBSCAN__

__DBSCAN__, __NMS__, __HYBRID__

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.dbscan_confidence_threshold

DBSCAN Confidence Threshold

float

The confidence threshold used to filter out the clustered bounding box output from DBSCAN.

0.9

0.1

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.nms_iou_threshold

NMS IOU Threshold

float

The Intersection Over Union (IOU) threshold to filter out redundant boxes from raw detections to form final clustered outputs.

0

1

bbox_handler_config.classwise_bbox_handler_config.value.clustering_config.nms_confidence_threshold

NMS Confidence Threshold

float

The confidence threshold to filter out clustered bounding boxes from NMS.

0

1

bbox_handler_config.classwise_bbox_handler_config.value.confidence_model

string

aggregate_cov

bbox_handler_config.classwise_bbox_handler_config.value.output_map

string

bbox_handler_config.classwise_bbox_handler_config.value.bbox_color

collection

0

0,1,2

bbox_handler_config.classwise_bbox_handler_config.value.bbox_color.R

integer

255

bbox_handler_config.classwise_bbox_handler_config.value.bbox_color.G

integer

0

bbox_handler_config.classwise_bbox_handler_config.value.bbox_color.B

integer

0

bbox_handler_config.postproc_classes

list

list of string

prune

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

model

Model path

hidden

UNIX path to where the input model is located.

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

results_dir

Results directory

hidden

key

Encode key

hidden

normalizer

Normalizer

string

How to normalize

max

max, L2

equalization_criterion

Equalization Criterion

string

Criteria to equalize the stats of inputs to an element wise op layer.

union

union, intersection, arithmetic_mean,geometric_mean

no

pruning_granularity

Pruning Granularity

integer

Number of filters to remove at a time.

8

no

pruning_threshold

Pruning Threshold

float

Threshold to compare normalized norm against.

0.1

0

1

yes

yes

min_num_filters

Minimum number of filters

integer

Minimum number of filters to be kept per layer

16

no

excluded_layers

Excluded layers

string

string of list: List of excluded_layers. Examples: -i item1 item2

verbose

verbosity

hidden

TRUE

train

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

popular

regex

automl_enabled

math_cond

parent_param

depends_on

version

Schema Version

const

The version of this schema

1

FALSE

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

FALSE

dataset_config.image_extension

Image Extension

string

Extension of the images to be used.

png

png,jpg,jpeg

yes

FALSE

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

FALSE

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

FALSE

dataset_config.validation_data_source.tfrecords_path

Validation TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

FALSE

dataset_config.validation_data_source.image_directory_path

Validation Image Path

hidden

/shared/users/1234/datasets/5678/training

FALSE

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car,van,heavy_truck etc may be grouped under automobile.

FALSE

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

FALSE

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

FALSE

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords,you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0,N-1]. For random split partitioning,force the validation fold index to 0 as the tfrecord is just 2-fold.

0

FALSE

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

FALSE

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

480

inf

yes

Yes

/ 32

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

272

inf

yes

Yes

/ 32

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

1

0

inf

yes

FALSE

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

1

0

inf

yes

FALSE

augmentation_config.preprocessing.output_image_channel

Image Channel

integer

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently,1-channel input is not recommended for datasets with JPG images. For PNG images,both 3-channel RGB and 1-channel monochrome images are supported.

3

1,3

yes

FALSE

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

no

FALSE

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

no

FALSE

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

no

FALSE

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

no

FALSE

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

no

FALSE

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

no

FALSE

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

1

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

inf

<= augmentation_config.spatial_augmentation.zoom_max

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

inf

TRUE

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

FALSE

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

FALSE

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max,rotate_rad_max].

0

FALSE

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

1

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally,this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0,you can set this value to 0.5.

0.5

0

1

0.5

bbox_rasterizer_config

Bounding box rasterizer

collection

Collection of parameters to configure the bounding box rasterizer

FALSE

bbox_rasterizer_config.deadzone_radius

Bounding box rasterizer deadzone radius

float

0.4

0

1

yes

model_config

Model

collection

FALSE

model_config.arch

BackBone Architecture

string

The architecture of the backbone feature extractor to be used for training.

resnet

resnet

yes

FALSE

model_config.pretrained_model_file

PTM File Path

hidden

This parameter defines the path to a pretrained TLT model file. If the load_graph flag is set to false,it is assumed that only the weights of the pretrained model file is to be used. In this case,TLT train constructs the feature extractor graph in the experiment and loads the weights from the pretrained model file that has matching layer names. Thus,transfer learning across different resolutions and domains are supported. For layers that may be absent in the pretrained model,the tool initializes them with random weights and skips the import for that layer.

/shared/.pretrained/resnet18/detectnet_v2_vresnet18/resnet18.hdf5

FALSE

model_config.load_graph

PTM Load Graph

bool

A flag to determine whether or not to load the graph from the pretrained model file,or just the weights. For a pruned model,set this parameter to True. Pruning modifies the original graph,so the pruned model graph and the weights need to be imported.

FALSE

FALSE

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template,and is different for different feature extractor templates.

0

3

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

model_config.all_projections

All Projections

bool

For templates with shortcut connections,this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers,irrespective of whether there is a change in stride across the input and output.

model_config.num_layers

Number of Layers

ordered_int

The depth of the feature extractor for scalable templates.

18

10,18,34,50,101

yes

FALSE

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True,MaxPooling is used to downsample; however,for the object-detection network,NVIDIA recommends setting this to False and using strided convolutions.

model_config.use_batch_norm

Use Batch Normalization

bool

A flag to determine whether to use Batch Normalization layers or not.

TRUE

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

1

model_config.training_precision.backend_floatx

Backend Training Precision

string

A nested parameter that sets the precision of the backend training framework.

__FLOAT32__

no

FALSE

model_config.objective_set.cov

Objective COV

collection

The objectives for training the network. For object-detection networks,set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

{}

yes

FALSE

model_config.objective_set.bbox.scale

Objective Bounding Box Scale

float

The objectives for training the network. For object-detection networks,set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

35

yes

FALSE

model_config.objective_set.bbox.offset

Objective Bounding Box Offset

float

The objectives for training the network. For object-detection networks,set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

0.5

yes

FALSE

training_config

Training

collection

FALSE

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

4

1

32

yes

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

10

1

500

yes

Yes

FALSE

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

FALSE

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-06

0

1

yes

Yes

TRUE

< training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

5.00E-04

0

1

yes

Yes

TRUE

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.100000001

0

1

yes

Yes

TRUE

< training_config.learning_rate.soft_start_annealing_schedule.annealing

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.699999988

0

1

yes

Yes

TRUE

TRUE

training_config.regularizer.type

Regularizer Type

ordered

The type of the regularizer being used.

__L1__

__NO_REG__,__L1__,__L2__

yes

TRUE

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

3.00E-11

3.00E-03

yes

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

yes

FALSE

training_config.optimizer.adam.beta1

Optimizer Adam Beta1

float

0.899999976

0.5

0.95

yes

training_config.optimizer.adam.beta2

Optimizer Adam Beta2

float

0.999000013

0.5

0.95

yes

training_config.cost_scaling.enabled

Enable Cost Scaling

bool

Enables cost scaling during training.

FALSE

yes

FALSE

training_config.cost_scaling.initial_exponent

Cost Scaling Initial Exponent

float

20

yes

FALSE

training_config.cost_scaling.increment

Cost Scaling Increment

float

0.005

yes

FALSE

training_config.cost_scaling.decrement

Cost Scaling Decrement

float

1

yes

FALSE

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

1

0

inf

yes

FALSE

evaluation_config

Evaluation

collection

yes

FALSE

evaluation_config.average_precision_mode

Average Precision Mode

ordered

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__,__INTEGRATE__

FALSE

evaluation_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

0

inf

yes

FALSE

evaluation_config.first_validation_epoch

First Validation Epoch

integer

The first epoch to start running validation. Ideally it is preferred to wait for at least 20-30% of the total number of epochs before starting evaluation,since the predictions in the initial epochs would be fairly inaccurate. Too many candidate boxes may be sent to clustering and this can cause the evaluation to slow down.

30

1

inf

yes

FALSE

cost_function_config

Cost function

collection

FALSE

cost_function_config.enable_autoweighting

Auto-Weighting

bool

TRUE

yes

FALSE

cost_function_config.max_objective_weight

Maximum Objective Weight

float

0.999899983

FALSE

cost_function_config.min_objective_weight

Minimum Objective Weight

float

1.00E-04

FALSE

classwise_config

Class-wise organized parameters

list

FALSE

classwise_config.key

Class Key

string

Name of class for the classwise parameters

person

FALSE

classwise_config.value.evaluation_config

Evaluation config elements per class

collection

FALSE

classwise_config.value.evaluation_config.minimum_detection_ground_truth_overlap

Minimum Detection Ground Truth Overlaps

float

Minimum IOU between ground truth and predicted box after clustering to call a valid detection. This parameter is a repeatable dictionary and a separate one must be defined for every class.

0.5

0

1

yes

FALSE

classwise_config.value.evaluation_config.evaluation_box_config.minimum_height

Minimum Height

integer

Minimum height in pixels for a valid ground truth and prediction bbox.

20

0

yes

FALSE

classwise_config.value.evaluation_config.evaluation_box_config.maximum_height

Maximum Height

integer

Maximum height in pixels for a valid ground truth and prediction bbox.

9999

0

yes

FALSE

classwise_config.value.evaluation_config.evaluation_box_config.minimum_width

Minimum Width

integer

Minimum width in pixels for a valid ground truth and prediction bbox.

10

0

yes

FALSE

classwise_config.value.evaluation_config.evaluation_box_config.maximum_width

Maximum Width

integer

Maximum width in pixels for a valid ground truth and prediction bbox.

9999

0

yes

FALSE

classwise_config.value.cost_function_config

Class-wise cost fuction config per class

collection

yes

FALSE

classwise_config.value.cost_function_config.class_weight

Class Weight

float

4

1

4

yes

classwise_config.value.cost_function_config.coverage_foreground_weight

Coverage Forground Weight

float

0.050000001

yes

FALSE

classwise_config.value.cost_function_config.objectives

Objectives

list

[{“name”: “cov”,”initial_weight”: 1.0,”weight_target”: 1.0},{“name”: “bbox”,”initial_weight”: 10.0,”weight_target”: 10.0}]

yes

FALSE

classwise_config.value.cost_function_config.objectives.name

Objective Name

string

Objective name such as cov or bbox.

cov

yes

FALSE

classwise_config.value.cost_function_config.objectives.initial_weight

Initial Weight

float

Initial weight for named objective.

1

yes

FALSE

classwise_config.value.cost_function_config.objectives.weight_target

Weight Target

float

Target weight for named objective.

1

yes

FALSE

classwise_config.value.bbox_rasterizer_config

Rasterization

collection

yes

FALSE

classwise_config.value.bbox_rasterizer_config.cov_center_x

Center of Object X-Coordinate

float

x-coordinate of the center of the object

0.5

0.3

0.7

yes

classwise_config.value.bbox_rasterizer_config.cov_center_y

Center of Object Y-Coordinate

float

y-coordinate of the center of the object

0.5

0.3

0.7

yes

classwise_config.value.bbox_rasterizer_config.cov_radius_x

Center of Object X-Radius

float

x-radius of the coverage ellipse

1

0.7

1

yes

classwise_config.value.bbox_rasterizer_config.cov_radius_y

Center of Object Y-Radius

float

y-radius of the coverage ellipse

1

0.7

1

yes

classwise_config.value.bbox_rasterizer_config.bbox_min_radius

Bounding Box Minimum Radius

float

The minimum radius of the coverage region to be drawn for boxes

1

0

1

yes

classwise_config.postprocessing_config

Post-Processing

collection

FALSE

classwise_config.postprocessing_config.clustering_config.coverage_threshold

Coverage Threshold

float

The minimum threshold of the coverage tensor output to be considered a valid candidate box for clustering. The four coordinates from the bbox tensor at the corresponding indices are passed for clustering.

0.0075

0

1

yes

classwise_config.postprocessing_config.clustering_config.dbscan_eps

DBSCAN Samples Distance

float

The maximum distance between two samples for one to be considered in the neighborhood of the other. This is not a maximum bound on the distances of points within a cluster. The greater the dbscan_eps value,the more boxes are grouped together.

0.230000004

0

1

yes

classwise_config.postprocessing_config.clustering_config.dbscan_min_samples

DBSCAN Minimum Samples

float

The total weight in a neighborhood for a point to be considered as a core point. This includes the point itself.

0.050000001

0

1

yes

classwise_config.postprocessing_config.clustering_config.minimum_bounding_box_height

Minimum Bounding Box Height

integer

The minimum height in pixels to consider as a valid detection post clustering.

20

0

10000

yes

classwise_config.postprocessing_config.clustering_config.clustering_algorithm

Clustering Algorithm

ordered

Defines the post-processing algorithm to cluter raw detections to the final bbox render. When using HYBRID mode,ensure both DBSCAN and NMS configuration parameters are defined.

__DBSCAN__

__DBSCAN__,__NMS__,__HYBRID__

yes

FALSE

classwise_config.postprocessing_config.clustering_config.dbscan_confidence_threshold

DBSCAN Confidence Threshold

float

The confidence threshold used to filter out the clustered bounding box output from DBSCAN.

0.1

0.01

0.8

yes

TRUE

classwise_config.postprocessing_config.clustering_config.nms_iou_threshold

NMS IOU Threshold

float

The Intersection Over Union (IOU) threshold to filter out redundant boxes from raw detections to form final clustered outputs.

0.2

0

1

classwise_config.postprocessing_config.clustering_config.nms_confidence_threshold

NMS Confidence Threshold

float

The confidence threshold to filter out clustered bounding boxes from NMS.

0

0

1

retrain

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

popular

regex

version

Schema Version

const

The version of this schema

1

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

dataset_config

Dataset

collection

Parameters to configure the dataset

dataset_config.image_extension

Image Extension

string

Extension of the images to be used.

png

png, jpg, jpeg

yes

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

dataset_config.validation_data_source.tfrecords_path

Validation TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

dataset_config.validation_data_source.image_directory_path

Validation Image Path

hidden

/shared/users/1234/datasets/5678/training

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords, you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0, N-1]. For random split partitioning, force the validation fold index to 0 as the tfrecord is just 2-fold.

0

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

480

yes

Yes

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

272

yes

Yes

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

1

0

yes

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

1

0

yes

augmentation_config.preprocessing.output_image_channel

Image Channel

integer

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently, 1-channel input is not recommended for datasets with JPG images. For PNG images, both 3-channel RGB and 1-channel monochrome images are supported.

3

1, 3

yes

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

no

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

no

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

no

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

1

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max, rotate_rad_max].

0

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

1

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally, this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0, you can set this value to 0.5.

0.5

0.5

bbox_rasterizer_config

Bounding box rasterizer

collection

Collection of parameters to configure the bounding box rasterizer

bbox_rasterizer_config.deadzone_radius

Bounding box rasterizer deadzone radius

float

0.4

0

1

yes

model_config

Model

collection

model_config.arch

BackBone Architecture

string

The architecture of the backbone feature extractor to be used for training.

resnet

resnet

yes

model_config.pretrained_model_file

PTM File Path

hidden

This parameter defines the path to a pretrained TLT model file. If the load_graph flag is set to false, it is assumed that only the weights of the pretrained model file is to be used. In this case, TLT train constructs the feature extractor graph in the experiment and loads the weights from the pretrained model file that has matching layer names. Thus, transfer learning across different resolutions and domains are supported. For layers that may be absent in the pretrained model, the tool initializes them with random weights and skips the import for that layer.

/shared/.pretrained/resnet18/detectnet_v2_vresnet18/resnet18.hdf5

model_config.load_graph

PTM Load Graph

bool

A flag to determine whether or not to load the graph from the pretrained model file, or just the weights. For a pruned model, set this parameter to True. Pruning modifies the original graph, so the pruned model graph and the weights need to be imported.

FALSE

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template, and is different for different feature extractor templates.

0

3

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

model_config.all_projections

All Projections

bool

For templates with shortcut connections, this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers, irrespective of whether there is a change in stride across the input and output.

model_config.num_layers

Number of Layers

integer

The depth of the feature extractor for scalable templates.

18

10, 18, 34, 50, 101

yes

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True, MaxPooling is used to downsample; however, for the object-detection network, NVIDIA recommends setting this to False and using strided convolutions.

model_config.use_batch_norm

Use Batch Normalization

bool

A flag to determine whether to use Batch Normalization layers or not.

TRUE

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

1

model_config.training_precision.backend_floatx

Backend Training Precision

string

A nested parameter that sets the precision of the backend training framework.

__FLOAT32__

no

model_config.objective_set.cov

Objective COV

collection

The objectives for training the network. For object-detection networks, set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

{}

yes

model_config.objective_set.bbox.scale

Objective Bounding Box Scale

float

The objectives for training the network. For object-detection networks, set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

35

yes

model_config.objective_set.bbox.offset

Objective Bounding Box Offset

float

The objectives for training the network. For object-detection networks, set it to learn cov and bbox. These parameters should not be altered for the current training pipeline.

0.5

yes

training_config

Training

collection

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

4

1

yes

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

120

1

yes

Yes

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-06

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

5.00E-04

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.100000001

0

1

yes

Yes

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.699999988

0

1

yes

Yes

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__NO_REG__, __L1__, __L2__

yes

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

yes

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

yes

training_config.optimizer.adam.beta1

Optimizer Adam Beta1

float

0.899999976

yes

training_config.optimizer.adam.beta2

Optimizer Adam Beta2

float

0.999000013

yes

training_config.cost_scaling.enabled

Enable Cost Scaling

bool

Enables cost scaling during training.

FALSE

yes

training_config.cost_scaling.initial_exponent

Cost Scaling Initial Exponent

float

20

yes

training_config.cost_scaling.increment

Cost Scaling Increment

float

0.005

yes

training_config.cost_scaling.decrement

Cost Scaling Decrement

float

1

yes

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

10

0

yes

evaluation_config

Evaluation

collection

yes

evaluation_config.average_precision_mode

Average Precision Mode

string

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__, __INTEGRATE__

evaluation_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

1

yes

evaluation_config.first_validation_epoch

First Validation Epoch

integer

The first epoch to start running validation. Ideally it is preferred to wait for at least 20-30% of the total number of epochs before starting evaluation, since the predictions in the initial epochs would be fairly inaccurate. Too many candidate boxes may be sent to clustering and this can cause the evaluation to slow down.

30

1

yes

cost_function_config

Cost function

collection

cost_function_config.enable_autoweighting

Auto-Weighting

bool

TRUE

yes

cost_function_config.max_objective_weight

Maximum Objective Weight

float

0.999899983

cost_function_config.min_objective_weight

Minimum Objective Weight

float

1.00E-04

classwise_config

Class-wise organized parameters

list

classwise_config.key

Class Key

string

Name of class for the classwise parameters

person

classwise_config.value.evaluation_config

Evaluation config elements per class

collection

classwise_config.value.evaluation_config.minimum_detection_ground_truth_overlap

Minimum Detection Ground Truth Overlaps

float

Minimum IOU between ground truth and predicted box after clustering to call a valid detection. This parameter is a repeatable dictionary and a separate one must be defined for every class.

0.5

0

1

yes

classwise_config.value.evaluation_config.evaluation_box_config.minimum_height

Minimum Height

integer

Minimum height in pixels for a valid ground truth and prediction bbox.

20

0

yes

classwise_config.value.evaluation_config.evaluation_box_config.maximum_height

Maximum Height

integer

Maximum height in pixels for a valid ground truth and prediction bbox.

9999

0

yes

classwise_config.value.evaluation_config.evaluation_box_config.minimum_width

Minimum Width

integer

Minimum width in pixels for a valid ground truth and prediction bbox.

10

0

yes

classwise_config.value.evaluation_config.evaluation_box_config.maximum_width

Maximum Width

integer

Maximum width in pixels for a valid ground truth and prediction bbox.

9999

0

yes

classwise_config.value.cost_function_config

Class-wise cost fuction config per class

collection

yes

classwise_config.value.cost_function_config.class_weight

Class Weight

float

4

yes

classwise_config.value.cost_function_config.coverage_foreground_weight

Coverage Forground Weight

float

0.050000001

yes

classwise_config.value.cost_function_config.objectives

Objectives

list

[{“name”: “cov”, “initial_weight”: 1.0, “weight_target”: 1.0}, {“name”: “bbox”, “initial_weight”: 10.0, “weight_target”: 10.0}]

yes

classwise_config.value.cost_function_config.objectives.name

Objective Name

string

Objective name such as cov or bbox.

cov

yes

classwise_config.value.cost_function_config.objectives.initial_weight

Initial Weight

float

Initial weight for named objective.

1

yes

classwise_config.value.cost_function_config.objectives.weight_target

Weight Target

float

Target weight for named objective.

1

yes

classwise_config.value.bbox_rasterizer_config

Rasterization

collection

yes

classwise_config.value.bbox_rasterizer_config.cov_center_x

Center of Object X-Coordinate

float

x-coordinate of the center of the object

0.5

0

1

yes

classwise_config.value.bbox_rasterizer_config.cov_center_y

Center of Object Y-Coordinate

float

y-coordinate of the center of the object

0.5

0

1

yes

classwise_config.value.bbox_rasterizer_config.cov_radius_x

Center of Object X-Radius

float

x-radius of the coverage ellipse

1

0

1

yes

classwise_config.value.bbox_rasterizer_config.cov_radius_y

Center of Object Y-Radius

float

y-radius of the coverage ellipse

1

0

1

yes

classwise_config.value.bbox_rasterizer_config.bbox_min_radius

Bounding Box Minimum Radius

float

The minimum radius of the coverage region to be drawn for boxes

1

0

1

yes

classwise_config.postprocessing_config

Post-Processing

collection

classwise_config.postprocessing_config.clustering_config.coverage_threshold

Coverage Threshold

float

The minimum threshold of the coverage tensor output to be considered a valid candidate box for clustering. The four coordinates from the bbox tensor at the corresponding indices are passed for clustering.

0.0075

0

1

yes

classwise_config.postprocessing_config.clustering_config.dbscan_eps

DBSCAN Samples Distance

float

The maximum distance between two samples for one to be considered in the neighborhood of the other. This is not a maximum bound on the distances of points within a cluster. The greater the dbscan_eps value, the more boxes are grouped together.

0.230000004

0

1

yes

classwise_config.postprocessing_config.clustering_config.dbscan_min_samples

DBSCAN Minimum Samples

float

The total weight in a neighborhood for a point to be considered as a core point. This includes the point itself.

0.050000001

0

1

yes

classwise_config.postprocessing_config.clustering_config.minimum_bounding_box_height

Minimum Bounding Box Height

integer

The minimum height in pixels to consider as a valid detection post clustering.

20

0

10000

yes

classwise_config.postprocessing_config.clustering_config.clustering_algorithm

Clustering Algorithm

string

Defines the post-processing algorithm to cluter raw detections to the final bbox render. When using HYBRID mode, ensure both DBSCAN and NMS configuration parameters are defined.

__DBSCAN__

__DBSCAN__, __NMS__, __HYBRID__

yes

classwise_config.postprocessing_config.clustering_config.dbscan_confidence_threshold

DBSCAN Confidence Threshold

float

The confidence threshold used to filter out the clustered bounding box output from DBSCAN.

0.1

0.1

yes

classwise_config.postprocessing_config.clustering_config.nms_iou_threshold

NMS IOU Threshold

float

The Intersection Over Union (IOU) threshold to filter out redundant boxes from raw detections to form final clustered outputs.

0.2

0

1

classwise_config.postprocessing_config.clustering_config.nms_confidence_threshold

NMS Confidence Threshold

float

The confidence threshold to filter out clustered bounding boxes from NMS.

0

0

1

dssd

evaluate

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

param_type (internal / hidden / inferred)

CLI

version

Schema Version

const

The version of this schema

1

internal

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

dataset_config

Dataset

collection

Parameters to configure the dataset

dataset_config.data_sources.label_directory_path

KITTI label path

hidden

hidden

dataset_config.data_sources.image_directory_path

Image path

hidden

dataset_config.data_sources.tfrecords_directory_path

TFRecords path

hidden

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the dataset to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.validation_data_sources.label_directory_path

KITTI label path

hidden

dataset_config.validation_data_sources.image_directory_path

Image path

hidden

dataset_config.validation_data_sources.tfrecords_directory_path

TFRecords path

hidden

dataset_config.include_difficult_in_training

include difficult label in training

bool

Whether to use difficult objects in training

TRUE

training_config

Training

collection

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

10

1

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

80

1

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

training_config.learning_rate

collection

training_config.learning_rate.soft_start_annealing_schedule

collection

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-05

0

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

9.00E-03

0

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.1

0

1

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.8

0

1

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__L1__, __L2__

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-05

0

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

1

1

training_config.max_queue_size

Max Queue Size

integer

Maximum Queue Size in Sequence Dataset

16

1

training_config.n_workers

Workers

integer

Number of workers in sequence dataset

8

1

training_config.use_multiprocessing

Use Multiprocessing

bool

Use multiprocessing or not

training_config.early_stopping

Early Stopping

collection

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

loss, validation_loss, val_loss

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

0

training_config.visualizer

Visualizer

collection

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

3

0

eval_config

Evaluation

collection

eval_config.average_precision_mode

Average Precision Mode

string

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__, __INTEGRATE__

eval_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

1

eval_config.batch_size

Batch Size

integer

batch size for evaluation

16

1

eval_config.matching_iou_threshold

Matching IoU Threshold

float

IoU threshold

0.5

0

1

eval_config.visualize_pr_curve

Visualize PR Curve

bool

Whether or not to visualize precision-recall curve

nms_config.confidence_threshold

Confidence Threshold

float

Confidence threshold

0.01

0

1

nms_config.clustering_iou_threshold

IoU threshold

float

IoU threshold

0.6

0

1

nms_config.top_k

Top K

integer

Maximum number of objects after NMS

200

0

nms_config.infer_nms_score_bits

NMS Score Bits

integer

Number of bits for scores for optimized NMS

1

32

augmentation_config

Augmentation config

collection

augmentation_config.output_width

Model Input width

integer

960

yes

augmentation_config.output_height

Model Input height

integer

544

yes

augmentation_config.output_channel

Model Input channel

integer

3

yes

augmentation_config.random_crop_min_scale

Random Crop Min Scale

float

the minimum random crop size

0.3

0

1

augmentation_config.random_crop_max_scale

Random Crop Max Scale

float

the maximum random crop size

1

0

1

augmentation_config.random_crop_min_ar

Random Crop Max Aspect Ratio

float

the minimum random crop aspect ratio

0.5

augmentation_config.random_crop_max_ar

Random Crop MIin Aspect Ratio

float

the maximum random crop aspect ratio

2

augmentation_config.zoom_out_min_scale

Zoom Out Min Scale

float

Minimum scale of ZoomOut augmentation

1

1

augmentation_config.zoom_out_max_scale

Zoom Out Max Scale

float

Maximum scale of ZoomOut augmentation

4

1

augmentation_config.brightness

Brightness

integer

Brightness delta in color jittering augmentation

32

0

255

augmentation_config.contrast

Contrast

float

Contrast delta factor in color jitter augmentation

0.5

0

1

augmentation_config.saturation

Saturation

float

Saturation delta factor in color jitter augmentation

0.5

0

1

augmentation_config.hue

Hue

integer

Hue delta in color jittering augmentation

18

0

180

augmentation_config.random_flip

Random Flip

float

Probablity of performing random horizontal flip

augmentation_config.image_mean

Image Mean

collection

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

augmentation_config.image_mean.key

Image Mean key

string

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

augmentation_config.image_mean.value

Image Mean value

float

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

dssd_config.aspect_ratios_global

Aspect Ratio Global

string

The anchor boxes of aspect ratios defined in aspect_ratios_global will be generated for each feature layer used for prediction. Note that either the aspect_ratios_global or aspect_ratios parameter is required; you don’t need to specify both.

[1.0, 2.0, 0.5, 3.0, 1.0/3.0]

dssd_config.aspect_ratios

Aspect Ratio

srting

The aspect ratio of anchor boxes for different SSD feature layers

dssd_config.two_boxes_for_ar1

Two boxes for aspect-ratio=1

bool

If this parameter is True, two boxes will be generated with an aspect ratio of 1.

TRUE

dssd_config.clip_boxes

Clip Boxes

bool

If true, all corner anchor boxes will be truncated so they are fully inside the feature images.

FALSE

dssd_config.variances

Variance

string

A list of 4 positive floats to decode bboxes

[0.1, 0.1, 0.2, 0.2]

dssd_config.scales

Scales

string

A list of positive floats containing scaling factors per convolutional predictor layer

[0.05, 0.1, 0.25, 0.4, 0.55, 0.7, 0.85]

dssd_config.steps

Steps

string

An optional list inside quotation marks with a length that is the number of feature layers for prediction.The elements should be floats or tuples/lists of two floats. The steps define how many pixels apart the anchor-box center points should be

dssd_config.offsets

Offsets

string

An optional list of floats inside quotation marks with length equal to the number of feature layers for prediction. The first anchor box will have a margin of offsets[i]*steps[i] pixels from the left and top borders. If offsets are not provided, 0.5 will be used as default value.

dssd_config.arch

Arch

string

The backbone for feature extraction

resnet

dssd_config.nlayers

Number of Layers

integer

The number of conv layers in a specific arch

18

dssd_config.freeze_bn

Freeze BN

bool

Whether to freeze all batch normalization layers during training.

FALSE

dssd_config.freeze_blocks

Freeze Blocks

list

The list of block IDs to be frozen in the model during training

dssd_config.pred_num_channels

Prediction Layer Channel

integer

The number of channel of the DSSD prediction layer

512

1

inference

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

param_type (internal / hidden / inferred)

CLI

version

Schema Version

const

The version of this schema

1

internal

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

threshold

Threshold

float

0.3

dataset_config

Dataset

collection

Parameters to configure the dataset

dataset_config.data_sources.label_directory_path

KITTI label path

hidden

hidden

dataset_config.data_sources.image_directory_path

Image path

hidden

dataset_config.data_sources.tfrecords_directory_path

TFRecords path

hidden

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the dataset to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.validation_data_sources.label_directory_path

KITTI label path

hidden

dataset_config.validation_data_sources.image_directory_path

Image path

hidden

dataset_config.validation_data_sources.tfrecords_directory_path

TFRecords path

hidden

dataset_config.include_difficult_in_training

include difficult label in training

bool

Whether to use difficult objects in training

TRUE

training_config

Training

collection

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

10

1

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

80

1

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

training_config.learning_rate

collection

training_config.learning_rate.soft_start_annealing_schedule

collection

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-05

0

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

9.00E-03

0

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.1

0

1

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.8

0

1

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__L1__, __L2__

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-05

0

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

1

1

training_config.max_queue_size

Max Queue Size

integer

Maximum Queue Size in Sequence Dataset

16

1

training_config.n_workers

Workers

integer

Number of workers in sequence dataset

8

1

training_config.use_multiprocessing

Use Multiprocessing

bool

Use multiprocessing or not

training_config.early_stopping

Early Stopping

collection

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

loss, validation_loss, val_loss

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

0

training_config.visualizer

Visualizer

collection

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

3

0

eval_config

Evaluation

collection

eval_config.average_precision_mode

Average Precision Mode

string

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__, __INTEGRATE__

eval_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

1

eval_config.batch_size

Batch Size

integer

batch size for evaluation

16

1

eval_config.matching_iou_threshold

Matching IoU Threshold

float

IoU threshold

0.5

0

1

eval_config.visualize_pr_curve

Visualize PR Curve

bool

Whether or not to visualize precision-recall curve

nms_config.confidence_threshold

Confidence Threshold

float

Confidence threshold

0.01

0

1

nms_config.clustering_iou_threshold

IoU threshold

float

IoU threshold

0.6

0

1

nms_config.top_k

Top K

integer

Maximum number of objects after NMS

200

0

nms_config.infer_nms_score_bits

NMS Score Bits

integer

Number of bits for scores for optimized NMS

1

32

augmentation_config

Augmentation config

collection

augmentation_config.output_width

Model Input width

integer

960

yes

augmentation_config.output_height

Model Input height

integer

544

yes

augmentation_config.output_channel

Model Input channel

integer

3

yes

augmentation_config.random_crop_min_scale

Random Crop Min Scale

float

the minimum random crop size

0.3

0

1

augmentation_config.random_crop_max_scale

Random Crop Max Scale

float

the maximum random crop size

1

0

1

augmentation_config.random_crop_min_ar

Random Crop Max Aspect Ratio

float

the minimum random crop aspect ratio

0.5

augmentation_config.random_crop_max_ar

Random Crop MIin Aspect Ratio

float

the maximum random crop aspect ratio

2

augmentation_config.zoom_out_min_scale

Zoom Out Min Scale

float

Minimum scale of ZoomOut augmentation

1

1

augmentation_config.zoom_out_max_scale

Zoom Out Max Scale

float

Maximum scale of ZoomOut augmentation

4

1

augmentation_config.brightness

Brightness

integer

Brightness delta in color jittering augmentation

32

0

255

augmentation_config.contrast

Contrast

float

Contrast delta factor in color jitter augmentation

0.5

0

1

augmentation_config.saturation

Saturation

float

Saturation delta factor in color jitter augmentation

0.5

0

1

augmentation_config.hue

Hue

integer

Hue delta in color jittering augmentation

18

0

180

augmentation_config.random_flip

Random Flip

float

Probablity of performing random horizontal flip

augmentation_config.image_mean

Image Mean

collection

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

augmentation_config.image_mean.key

Image Mean key

string

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

augmentation_config.image_mean.value

Image Mean value

float

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

dssd_config.aspect_ratios_global

Aspect Ratio Global

string

The anchor boxes of aspect ratios defined in aspect_ratios_global will be generated for each feature layer used for prediction. Note that either the aspect_ratios_global or aspect_ratios parameter is required; you don’t need to specify both.

[1.0, 2.0, 0.5, 3.0, 1.0/3.0]

dssd_config.aspect_ratios

Aspect Ratio

srting

The aspect ratio of anchor boxes for different SSD feature layers

dssd_config.two_boxes_for_ar1

Two boxes for aspect-ratio=1

bool

If this parameter is True, two boxes will be generated with an aspect ratio of 1.

TRUE

dssd_config.clip_boxes

Clip Boxes

bool

If true, all corner anchor boxes will be truncated so they are fully inside the feature images.

FALSE

dssd_config.variances

Variance

string

A list of 4 positive floats to decode bboxes

[0.1, 0.1, 0.2, 0.2]

dssd_config.scales

Scales

string

A list of positive floats containing scaling factors per convolutional predictor layer

[0.05, 0.1, 0.25, 0.4, 0.55, 0.7, 0.85]

dssd_config.steps

Steps

string

An optional list inside quotation marks with a length that is the number of feature layers for prediction.The elements should be floats or tuples/lists of two floats. The steps define how many pixels apart the anchor-box center points should be

dssd_config.offsets

Offsets

string

An optional list of floats inside quotation marks with length equal to the number of feature layers for prediction. The first anchor box will have a margin of offsets[i]*steps[i] pixels from the left and top borders. If offsets are not provided, 0.5 will be used as default value.

dssd_config.arch

Arch

string

The backbone for feature extraction

resnet

dssd_config.nlayers

Number of Layers

integer

The number of conv layers in a specific arch

18

dssd_config.freeze_bn

Freeze BN

bool

Whether to freeze all batch normalization layers during training.

FALSE

dssd_config.freeze_blocks

Freeze Blocks

list

The list of block IDs to be frozen in the model during training

dssd_config.pred_num_channels

Prediction Layer Channel

integer

The number of channel of the DSSD prediction layer

512

1

train

parameter

Random Crop Max Aspect Ratio

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

param_type (internal / hidden / inferred)

CLI

automl_enabled

math_cond

parent_param

depends_on

version

Schema Version

const

The version of this schema

1

internal

FALSE

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

FALSE

initial_epoch

Initial epoch cli

hidden

1

CLI argument

FALSE

use_multiprocessing

CLI parameter

hidden

FALSE

FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

FALSE

dataset_config.data_sources.label_directory_path

KITTI label path

hidden

hidden

FALSE

dataset_config.data_sources.image_directory_path

Image path

hidden

FALSE

dataset_config.data_sources.tfrecords_directory_path

TFRecords path

hidden

FALSE

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the dataset to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car,van,heavy_truck etc may be grouped under automobile.

FALSE

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

FALSE

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

FALSE

dataset_config.validation_data_sources.label_directory_path

KITTI label path

hidden

FALSE

dataset_config.validation_data_sources.image_directory_path

Image path

hidden

FALSE

dataset_config.validation_data_sources.tfrecords_directory_path

TFRecords path

hidden

FALSE

dataset_config.include_difficult_in_training

include difficult label in training

bool

Whether to use difficult objects in training

TRUE

training_config

Training

collection

FALSE

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

10

1

inf

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

80

1

inf

FALSE

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

FALSE

training_config.learning_rate

collection

FALSE

training_config.learning_rate.soft_start_annealing_schedule

collection

FALSE

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-05

0

inf

TRUE

< training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

9.00E-03

0

inf

TRUE

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.1

0

1

TRUE

< training_config.learning_rate.soft_start_annealing_schedule.annealing

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.8

0

1

TRUE

TRUE

training_config.regularizer.type

Regularizer Type

ordered

The type of the regularizer being used.

__L1__

__L1__,__L2__

TRUE

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-05

0

inf

TRUE

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

1

1

inf

FALSE

training_config.max_queue_size

Max Queue Size

integer

Maximum Queue Size in Sequence Dataset

16

1

inf

FALSE

training_config.n_workers

Workers

integer

Number of workers in sequence dataset

8

1

inf

FALSE

training_config.use_multiprocessing

Use Multiprocessing

bool

Use multiprocessing or not

FALSE

training_config.early_stopping

Early Stopping

collection

FALSE

training_config.early_stopping.monitor

Monitor

ordered

The name of the quantity to be monitored for early stopping

loss,validation_loss,val_loss

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

0

1

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

1

5

training_config.visualizer

Visualizer

collection

FALSE

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

FALSE

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

3

0

inf

FALSE

eval_config

Evaluation

collection

FALSE

eval_config.average_precision_mode

Average Precision Mode

ordered

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__,__INTEGRATE__

FALSE

eval_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

1

inf

FALSE

eval_config.batch_size

Batch Size

integer

batch size for evaluation

16

1

inf

FALSE

eval_config.matching_iou_threshold

Matching IoU Threshold

float

IoU threshold

0.5

0

1

FALSE

eval_config.visualize_pr_curve

Visualize PR Curve

bool

Whether or not to visualize precision-recall curve

FALSE

nms_config.confidence_threshold

Confidence Threshold

float

Confidence threshold

0.01

0

1

nms_config.clustering_iou_threshold

IoU threshold

float

IoU threshold

0.6

0

1

nms_config.top_k

Top K

integer

Maximum number of objects after NMS

200

0

inf

nms_config.infer_nms_score_bits

NMS Score Bits

integer

Number of bits for scores for optimized NMS

1

32

augmentation_config

Augmentation config

collection

FALSE

augmentation_config.output_width

Model Input width

integer

960

1

inf

yes

augmentation_config.output_height

Model Input height

integer

544

1

inf

yes

augmentation_config.output_channel

Model Input channel

ordered_int

3

1,3

yes

FALSE

augmentation_config.random_crop_min_scale

Random Crop Min Scale

float

the minimum random crop size

0.3

0

1

TRUE

< augmentation_config.random_crop_max_scale

augmentation_config.random_crop_max_scale

Random Crop Max Scale

float

the maximum random crop size

1

0

1

TRUE

TRUE

augmentation_config.random_crop_min_ar

Random Crop Min Aspect Ratio

float

the minimum random crop aspect ratio

0.5

0.1

10

< augmentation_config.random_crop_max_ar

augmentation_config.random_crop_max_ar

Random Crop Max Aspect Ratio

float

the maximum random crop aspect ratio

2

0.1

10

TRUE

augmentation_config.zoom_out_min_scale

Zoom Out Min Scale

float

Minimum scale of ZoomOut augmentation

1

1

inf

< augmentation_config.zoom_out_max_scale

augmentation_config.zoom_out_max_scale

Zoom Out Max Scale

float

Maximum scale of ZoomOut augmentation

4

1

inf

TRUE

augmentation_config.brightness

Brightness

integer

Brightness delta in color jittering augmentation

32

0

255

augmentation_config.contrast

Contrast

float

Contrast delta factor in color jitter augmentation

0.5

0

1

augmentation_config.saturation

Saturation

float

Saturation delta factor in color jitter augmentation

0.5

0

1

augmentation_config.hue

Hue

integer

Hue delta in color jittering augmentation

18

0

180

augmentation_config.random_flip

Random Flip

float

Probablity of performing random horizontal flip

0

1

augmentation_config.image_mean

Image Mean

collection

A key/value pair to specify image mean values. If omitted,ImageNet mean will be used for image preprocessing. If set,depending on output_channel,either ‘r/g/b’ or ‘l’ key/value pair must be configured.

FALSE

augmentation_config.image_mean.key

Image Mean key

string

A key/value pair to specify image mean values. If omitted,ImageNet mean will be used for image preprocessing. If set,depending on output_channel,either ‘r/g/b’ or ‘l’ key/value pair must be configured.

FALSE

augmentation_config.image_mean.value

Image Mean value

float

A key/value pair to specify image mean values. If omitted,ImageNet mean will be used for image preprocessing. If set,depending on output_channel,either ‘r/g/b’ or ‘l’ key/value pair must be configured.

0

255

dssd_config.aspect_ratios_global

Aspect Ratio Global

string

The anchor boxes of aspect ratios defined in aspect_ratios_global will be generated for each feature layer used for prediction. Note that either the aspect_ratios_global or aspect_ratios parameter is required; you don’t need to specify both.

[1.0,2.0,0.5,3.0,1.0/3.0]

FALSE

dssd_config.aspect_ratios

Aspect Ratio

string

The aspect ratio of anchor boxes for different SSD feature layers

FALSE

dssd_config.two_boxes_for_ar1

Two boxes for aspect-ratio=1

bool

If this parameter is True,two boxes will be generated with an aspect ratio of 1.

TRUE

dssd_config.clip_boxes

Clip Boxes

bool

If true,all corner anchor boxes will be truncated so they are fully inside the feature images.

FALSE

dssd_config.variances

Variance

string

A list of 4 positive floats to decode bboxes

[0.1,0.1,0.2,0.2]

FALSE

dssd_config.scales

Scales

string

A list of positive floats containing scaling factors per convolutional predictor layer

[0.05,0.1,0.25,0.4,0.55,0.7,0.85]

FALSE

dssd_config.steps

Steps

string

An optional list inside quotation marks with a length that is the number of feature layers for prediction.The elements should be floats or tuples/lists of two floats. The steps define how many pixels apart the anchor-box center points should be

FALSE

dssd_config.offsets

Offsets

string

An optional list of floats inside quotation marks with length equal to the number of feature layers for prediction. The first anchor box will have a margin of offsets[i]*steps[i] pixels from the left and top borders. If offsets are not provided,0.5 will be used as default value.

FALSE

dssd_config.arch

Arch

ordered

The backbone for feature extraction

resnet

resnet

FALSE

dssd_config.nlayers

Number of Layers

ordered_int

The number of conv layers in a specific arch

18

10,18,34,50,101,152

FALSE

dssd_config.freeze_bn

Freeze BN

bool

Whether to freeze all batch normalization layers during training.

FALSE

dssd_config.freeze_blocks

Freeze Blocks

list

The list of block IDs to be frozen in the model during training

FALSE

dssd_config.pred_num_channels

Prediction Layer Channel

integer

The number of channel of the DSSD prediction layer

512

1

512

FALSE

prune

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

model

Model path

hidden

UNIX path to where the input model is located.

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

results_dir

Results directory

hidden

key

Encode key

hidden

normalizer

Normalizer

string

How to normalize

max

max, L2

equalization_criterion

Equalization Criterion

string

Criteria to equalize the stats of inputs to an element wise op layer.

union

union, intersection, arithmetic_mean,geometric_mean

no

pruning_granularity

Pruning Granularity

integer

Number of filters to remove at a time.

8

no

pruning_threshold

Pruning Threshold

float

Threshold to compare normalized norm against.

0.1

0

1

yes

yes

min_num_filters

Minimum number of filters

integer

Minimum number of filters to be kept per layer

16

no

excluded_layers

Excluded layers

string

string of list: List of excluded_layers. Examples: -i item1 item2

verbose

verbosity

hidden

TRUE

export

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

version

Schema Version

const

The version of this schema

1

model

Model

hidden

UNIX path to the model file

0.1

yes

key

Encryption Key

hidden

Encryption key

tlt_encode

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

force_ptq

Force Post-Training Quantization

bool

Force generating int8 engine using Post Training Quantization

FALSE

no

cal_image_dir

hidden

data_type

Pruning Granularity

string

Number of filters to remove at a time.

fp32

int8, fp32, fp16

yes

yes

strict_type_constraints

bool

FALSE

gen_ds_config

bool

FALSE

cal_cache_file

Calibration cache file

hidden

Unix PATH to the int8 calibration cache file

yes

yes

batches

Number of calibration batches

integer

Number of batches to calibrate the model when run in INT8 mode

100

no

max_workspace_size

integer

Example: The integer value of 1<<30, 2<<30

max_batch_size

integer

1

batch_size

Batch size

integer

Number of images per batch when generating the TensorRT engine.

100

yes

min_batch_size

integer

1

opt_batch_size

integer

1

experiment_spec

Experiment Spec

hidden

UNIX path to the Experiment spec file used to train the model. This may be the train or retrain spec file.

yes

engine_file

Engine File

hidden

UNIX path to the model engine file.

yes

static_batch_size

integer

-1

results_dir

hidden

verbose

hidden

TRUE

retrain

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

param_type (internal / hidden / inferred)

CLI

version

Schema Version

const

The version of this schema

1

internal

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

initial_epoch

Initial epoch cli

hidden

1

CLI argument

use_multiprocessing

CLI parameter

bool

FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

dataset_config.data_sources.label_directory_path

KITTI label path

hidden

hidden

dataset_config.data_sources.image_directory_path

Image path

hidden

dataset_config.data_sources.tfrecords_directory_path

TFRecords path

hidden

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the dataset to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

dataset_config.validation_data_sources.label_directory_path

KITTI label path

hidden

dataset_config.validation_data_sources.image_directory_path

Image path

hidden

dataset_config.validation_data_sources.tfrecords_directory_path

TFRecords path

hidden

dataset_config.include_difficult_in_training

include difficult label in training

bool

Whether to use difficult objects in training

TRUE

training_config

Training

collection

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

10

1

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

80

1

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

training_config.learning_rate

collection

training_config.learning_rate.soft_start_annealing_schedule

collection

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

5.00E-05

0

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

9.00E-03

0

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.1

0

1

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.8

0

1

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__L1__, __L2__

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-05

0

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

1

1

training_config.max_queue_size

Max Queue Size

integer

Maximum Queue Size in Sequence Dataset

16

1

training_config.n_workers

Workers

integer

Number of workers in sequence dataset

8

1

training_config.use_multiprocessing

Use Multiprocessing

bool

Use multiprocessing or not

training_config.early_stopping

Early Stopping

collection

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

loss, validation_loss, val_loss

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

0

training_config.visualizer

Visualizer

collection

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

3

0

eval_config

Evaluation

collection

eval_config.average_precision_mode

Average Precision Mode

string

The mode in which the average precision for each class is calculated.

__SAMPLE__

__SAMPLE__, __INTEGRATE__

eval_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

10

1

eval_config.batch_size

Batch Size

integer

batch size for evaluation

16

1

eval_config.matching_iou_threshold

Matching IoU Threshold

float

IoU threshold

0.5

0

1

eval_config.visualize_pr_curve

Visualize PR Curve

bool

Whether or not to visualize precision-recall curve

nms_config.confidence_threshold

Confidence Threshold

float

Confidence threshold

0.01

0

1

nms_config.clustering_iou_threshold

IoU threshold

float

IoU threshold

0.6

0

1

nms_config.top_k

Top K

integer

Maximum number of objects after NMS

200

0

nms_config.infer_nms_score_bits

NMS Score Bits

integer

Number of bits for scores for optimized NMS

1

32

augmentation_config

Augmentation config

collection

augmentation_config.output_width

Model Input width

integer

960

yes

augmentation_config.output_height

Model Input height

integer

544

yes

augmentation_config.output_channel

Model Input channel

integer

3

yes

augmentation_config.random_crop_min_scale

Random Crop Min Scale

float

the minimum random crop size

0.3

0

1

augmentation_config.random_crop_max_scale

Random Crop Max Scale

float

the maximum random crop size

1

0

1

augmentation_config.random_crop_min_ar

Random Crop Max Aspect Ratio

float

the minimum random crop aspect ratio

0.5

augmentation_config.random_crop_max_ar

Random Crop MIin Aspect Ratio

float

the maximum random crop aspect ratio

2

augmentation_config.zoom_out_min_scale

Zoom Out Min Scale

float

Minimum scale of ZoomOut augmentation

1

1

augmentation_config.zoom_out_max_scale

Zoom Out Max Scale

float

Maximum scale of ZoomOut augmentation

4

1

augmentation_config.brightness

Brightness

integer

Brightness delta in color jittering augmentation

32

0

255

augmentation_config.contrast

Contrast

float

Contrast delta factor in color jitter augmentation

0.5

0

1

augmentation_config.saturation

Saturation

float

Saturation delta factor in color jitter augmentation

0.5

0

1

augmentation_config.hue

Hue

integer

Hue delta in color jittering augmentation

18

0

180

augmentation_config.random_flip

Random Flip

float

Probablity of performing random horizontal flip

augmentation_config.image_mean

Image Mean

collection

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

augmentation_config.image_mean.key

Image Mean key

string

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

augmentation_config.image_mean.value

Image Mean value

float

A key/value pair to specify image mean values. If omitted, ImageNet mean will be used for image preprocessing. If set, depending on output_channel, either ‘r/g/b’ or ‘l’ key/value pair must be configured.

dssd_config.aspect_ratios_global

Aspect Ratio Global

string

The anchor boxes of aspect ratios defined in aspect_ratios_global will be generated for each feature layer used for prediction. Note that either the aspect_ratios_global or aspect_ratios parameter is required; you don’t need to specify both.

[1.0, 2.0, 0.5, 3.0, 1.0/3.0]

dssd_config.aspect_ratios

Aspect Ratio

srting

The aspect ratio of anchor boxes for different SSD feature layers

dssd_config.two_boxes_for_ar1

Two boxes for aspect-ratio=1

bool

If this parameter is True, two boxes will be generated with an aspect ratio of 1.

TRUE

dssd_config.clip_boxes

Clip Boxes

bool

If true, all corner anchor boxes will be truncated so they are fully inside the feature images.

FALSE

dssd_config.variances

Variance

string

A list of 4 positive floats to decode bboxes

[0.1, 0.1, 0.2, 0.2]

dssd_config.scales

Scales

string

A list of positive floats containing scaling factors per convolutional predictor layer

[0.05, 0.1, 0.25, 0.4, 0.55, 0.7, 0.85]

dssd_config.steps

Steps

string

An optional list inside quotation marks with a length that is the number of feature layers for prediction.The elements should be floats or tuples/lists of two floats. The steps define how many pixels apart the anchor-box center points should be

dssd_config.offsets

Offsets

string

An optional list of floats inside quotation marks with length equal to the number of feature layers for prediction. The first anchor box will have a margin of offsets[i]*steps[i] pixels from the left and top borders. If offsets are not provided, 0.5 will be used as default value.

dssd_config.arch

Arch

string

The backbone for feature extraction

resnet

dssd_config.nlayers

Number of Layers

integer

The number of conv layers in a specific arch

18

dssd_config.freeze_bn

Freeze BN

bool

Whether to freeze all batch normalization layers during training.

FALSE

dssd_config.freeze_blocks

Freeze Blocks

list

The list of block IDs to be frozen in the model during training

dssd_config.pred_num_channels

Prediction Layer Channel

integer

The number of channel of the DSSD prediction layer

512

1

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

e

engine file path

hidden

k

encode key

hidden

c

cache_file

hidden

o

outputs

string

comma separated list of output node names

d

input_dims

string

comma separated list of input dimensions (not required for TLT 3.0 new models).

yes

yes

b

batch_size

integer

calibration batch size

8

yes

m

max_batch_size

integer

maximum TensorRT engine batch size (default 16). If meet with out-of-memory issue, please decrease the batch size accordingly.

16

yes

w

max_workspace_size

integer

maximum workspace size of TensorRT engine (default 1<<30). If meet with out-of-memory issue, please increase the workspace size accordingly.

t

data_type

string

TensorRT data type

fp32

fp32, fp16, int8

yes

i

input_order

string

input dimension ordering

nchw

nchw, nhwc, nc

s

strict_type_constraints

bool

TensorRT strict_type_constraints flag for INT8 mode

FALSE

u

dla_core

int

Use DLA core N for layers that support DLA (default = -1, which means no DLA core will be utilized for inference. Note that it’ll always allow GPU fallback).

-1

p

parse_profile_shapes

string

comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape>, where each shape has x as delimiter, e.g.,NxC, NxCxHxW, NxCxDxHxW, etc. Can be specified multiple times if there are multiple input tensors for the model. This argument is only useful in dynamic shape case.

model

etlt model from export

hidden

efficientdet

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

e

engine file path

hidden

k

encode key

hidden

c

cache_file

hidden

o

outputs

string

comma separated list of output node names

b

batch_size

integer

calibration batch size

8

yes

m

max_batch_size

integer

maximum TensorRT engine batch size (default 16). If meet with out-of-memory issue, please decrease the batch size accordingly.

16

yes

w

max_workspace_size

integer

maximum workspace size of TensorRT engine (default 1<<30). If meet with out-of-memory issue, please increase the workspace size accordingly.

t

data_type

string

TensorRT data type

fp32

fp32, fp16, int8

yes

i

input_order

string

input dimension ordering

nchw

nchw, nhwc, nc

s

strict_type_constraints

bool

TensorRT strict_type_constraints flag for INT8 mode

FALSE

u

dla_core

int

Use DLA core N for layers that support DLA (default = -1, which means no DLA core will be utilized for inference. Note that it’ll always allow GPU fallback).

-1

p

parse_profile_shapes

string

comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape>, where each shape has x as delimiter, e.g.,NxC, NxCxHxW, NxCxDxHxW, etc. Can be specified multiple times if there are multiple input tensors for the model. This argument is only useful in dynamic shape case.

model

etlt model from export

hidden

evaluate

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

CLI

version

Schema Version

const

The version of this schema

1

training_config

Training config

collection

Parameters to configure the training process

training_config.train_batch_size

training batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus.

16

0

training_config.iterations_per_loop

integer

10

training_config.num_epochs

number of epochs

integer

The number of epochs to train the network

6

0

training_config.num_examples_per_epoch

number of images per epoch per gpu

integer

Total number of images in the training set divided by the number of GPUs

118288

0

training_config.checkpoint

path to pretrained model

hidden

The path to the pretrained model, if any

training_config.pruned_model_path

path to pruned model

hidden

The path to a TAO pruned model for re-training, if any

training_config.checkpoint_period

checkpoint period

integer

The number of training epochs that should run per model checkpoint/validation

2

0

training_config.amp

AMP

bool

Whether to use mixed precision training

TRUE

training_config.moving_average_decay

moving average decay

float

Moving average decay

0.9999

training_config.l2_weight_decay

L2 weight decay

float

L2 weight decay

0.00004

training_config.l1_weight_decay

L1 weight decay

float

L1 weight decay

0

training_config.lr_warmup_epoch

learning rate warmup epoch

integer

The number of warmup epochs in the learning rate schedule

3

0

training_config.lr_warmup_init

initial learning rate during warmup

float

The initial learning rate in the warmup period

0.002

training_config.learning_rate

maximum learning rate

float

The maximum learning rate

0.02

training_config.tf_random_seed

random seed

integer

The random seed

42

0

training_config.clip_gradients_norm

clip gradient by norm

float

Clip gradients by the norm value

5.00E+00

training_config.skip_checkpoint_variables

skip checkpoint variables

string

If specified, the weights of the layers with matching regular expressions will not be loaded. This is especially helpful for transfer learning.

-predict*

eval_config

evaluation config

collection

Parameters to configure evaluation

eval_config.eval_epoch_cycle

evaluation epoch cycle

integer

The number of training epochs that should run per validation

2

0

eval_config.max_detections_per_image

maximum detections per image

integer

The maximum number of detections to visualize

100

0

eval_config.min_score_thresh

minimum confidence threshold

float

The lowest confidence of the predicted box and ground truth box that can be considered a match

0.4

eval_config.eval_batch_size

evaluation batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus

16

0

eval_config.eval_samples

number of samples for evaluation

integer

The number of samples for evaluation

500

dataset_config

dataset config

collection

Parameters to configure dataset

dataset_config.image_size

image size

string

The image dimension as a tuple within quote marks. (height, width) indicates the dimension of the resized and padded input.

1024,1024

yes

dataset_config.training_file_pattern

training file pattern

hidden

The TFRecord path for training

dataset_config.validation_file_pattern

validation file pattern

hidden

The TFRecord path for validation

dataset_config.validation_json_file

validation json file

hidden

The annotation file path for validation

dataset_config.num_classes

number of classes

integer

The number of classes. If there are N categories in the annotation, num_classes should be N+1 (background class)

91

yes

dataset_config.max_instances_per_image

maximum instances per image

integer

The maximum number of object instances to parse (default: 100)

100

dataset_config.skip_crowd_during_training

skip crowd during training

bool

Specifies whether to skip crowd during training

TRUE

model_config

model config

collection

Parameters to configure model

model_config.model_name

model name

string

Model name

efficientdet-d0

model_config.min_level

minimum level

integer

The minimum level of the output feature pyramid

3

model_config.max_level

maximum level

integer

The maximum level of the output feature pyramid

7

model_config.num_scales

number of scales

integer

The number of anchor octave scales on each pyramid level (e.g. if set to 3, the anchor scales are [2^0, 2^(1/3), 2^(2/3)])

3

model_config.aspect_ratios

aspect ratios

string

A list of tuples representing the aspect ratios of anchors on each pyramid level

[(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]

model_config.anchor_scale

anchor scale

integer

Scale of the base-anchor size to the feature-pyramid stride

4

augmentation_config

augmentation config

collection

Parameters to configure model

augmentation_config.rand_hflip

random horizontal flip

bool

Whether to perform random horizontal flip

TRUE

augmentation_config.random_crop_min_scale

minimum scale of random crop

float

The minimum scale of RandomCrop augmentation.

0.1

augmentation_config.random_crop_max_scale

maximum scale of random crop

float

The maximum scale of RandomCrop augmentation.

2

export

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

version

Schema Version

const

The version of this schema

1

experiment_spec_file

Experiment Spec

hidden

UNIX path to the Experiment spec file used to train the model. This may be the train or retrain spec file.

yes

model_path

Model

hidden

UNIX path to the model file

0.1

yes

output_path

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

key

Encryption Key

hidden

Encryption key

tlt_encode

yes

data_type

Pruning Granularity

string

Number of filters to remove at a time.

fp32

int8, fp32, fp16

yes

yes

cal_image_dir

hidden

cal_cache_file

Calibration cache file

hidden

Unix PATH to the int8 calibration cache file

yes

yes

engine_file

Engine File

hidden

UNIX path to the model engine file.

yes

max_batch_size

integer

1

batch_size

Batch size

integer

Number of images per batch when generating the TensorRT engine.

100

yes

batches

Number of calibration batches

integer

Number of batches to calibrate the model when run in INT8 mode

100

max_workspace_size

integer

Example: The integer value of 1<<30, 2<<30

verbose

hidden

TRUE

inference

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

CLI

version

Schema Version

const

The version of this schema

1

training_config

Training config

collection

Parameters to configure the training process

training_config.train_batch_size

training batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus.

16

0

training_config.iterations_per_loop

integer

10

training_config.num_epochs

number of epochs

integer

The number of epochs to train the network

6

0

training_config.num_examples_per_epoch

number of images per epoch per gpu

integer

Total number of images in the training set divided by the number of GPUs

118288

0

training_config.checkpoint

path to pretrained model

hidden

The path to the pretrained model, if any

training_config.pruned_model_path

path to pruned model

hidden

The path to a TAO pruned model for re-training, if any

training_config.checkpoint_period

checkpoint period

integer

The number of training epochs that should run per model checkpoint/validation

2

0

training_config.amp

AMP

bool

Whether to use mixed precision training

TRUE

training_config.moving_average_decay

moving average decay

float

Moving average decay

0.9999

training_config.l2_weight_decay

L2 weight decay

float

L2 weight decay

0.00004

training_config.l1_weight_decay

L1 weight decay

float

L1 weight decay

0

training_config.lr_warmup_epoch

learning rate warmup epoch

integer

The number of warmup epochs in the learning rate schedule

3

0

training_config.lr_warmup_init

initial learning rate during warmup

float

The initial learning rate in the warmup period

0.002

training_config.learning_rate

maximum learning rate

float

The maximum learning rate

0.02

training_config.tf_random_seed

random seed

integer

The random seed

42

0

training_config.clip_gradients_norm

clip gradient by norm

float

Clip gradients by the norm value

5.00E+00

training_config.skip_checkpoint_variables

skip checkpoint variables

string

If specified, the weights of the layers with matching regular expressions will not be loaded. This is especially helpful for transfer learning.

-predict*

eval_config

evaluation config

collection

Parameters to configure evaluation

eval_config.eval_epoch_cycle

evaluation epoch cycle

integer

The number of training epochs that should run per validation

2

0

eval_config.max_detections_per_image

maximum detections per image

integer

The maximum number of detections to visualize

100

0

eval_config.min_score_thresh

minimum confidence threshold

float

The lowest confidence of the predicted box and ground truth box that can be considered a match

0.4

eval_config.eval_batch_size

evaluation batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus

16

0

eval_config.eval_samples

number of samples for evaluation

integer

The number of samples for evaluation

500

dataset_config

dataset config

collection

Parameters to configure dataset

dataset_config.image_size

image size

string

The image dimension as a tuple within quote marks. (height, width) indicates the dimension of the resized and padded input.

1024,1024

yes

dataset_config.training_file_pattern

training file pattern

hidden

The TFRecord path for training

dataset_config.validation_file_pattern

validation file pattern

hidden

The TFRecord path for validation

dataset_config.validation_json_file

validation json file

hidden

The annotation file path for validation

dataset_config.num_classes

number of classes

integer

The number of classes. If there are N categories in the annotation, num_classes should be N+1 (background class)

91

yes

dataset_config.max_instances_per_image

maximum instances per image

integer

The maximum number of object instances to parse (default: 100)

100

dataset_config.skip_crowd_during_training

skip crowd during training

bool

Specifies whether to skip crowd during training

TRUE

model_config

model config

collection

Parameters to configure model

model_config.model_name

model name

string

Model name

efficientdet-d0

model_config.min_level

minimum level

integer

The minimum level of the output feature pyramid

3

model_config.max_level

maximum level

integer

The maximum level of the output feature pyramid

7

model_config.num_scales

number of scales

integer

The number of anchor octave scales on each pyramid level (e.g. if set to 3, the anchor scales are [2^0, 2^(1/3), 2^(2/3)])

3

model_config.aspect_ratios

aspect ratios

string

A list of tuples representing the aspect ratios of anchors on each pyramid level

[(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]

model_config.anchor_scale

anchor scale

integer

Scale of the base-anchor size to the feature-pyramid stride

4

augmentation_config

augmentation config

collection

Parameters to configure model

augmentation_config.rand_hflip

random horizontal flip

bool

Whether to perform random horizontal flip

TRUE

augmentation_config.random_crop_min_scale

minimum scale of random crop

float

The minimum scale of RandomCrop augmentation.

0.1

augmentation_config.random_crop_max_scale

maximum scale of random crop

float

The maximum scale of RandomCrop augmentation.

2

prune

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

model

Model path

hidden

UNIX path to where the input model is located.

yes

output_dir

Output Directory

hidden

UNIX path to where the pruned model will be saved.

yes

key

Encode key

hidden

normalizer

Normalizer

string

How to normalize

max

max, L2

equalization_criterion

Equalization Criterion

string

Criteria to equalize the stats of inputs to an element wise op layer.

union

union, intersection, arithmetic_mean,geometric_mean

no

pruning_granularity

Pruning Granularity

integer

Number of filters to remove at a time.

8

no

pruning_threshold

Pruning Threshold

float

Threshold to compare normalized norm against.

0.1

0

1

yes

yes

min_num_filters

Minimum number of filters

integer

Minimum number of filters to be kept per layer

16

no

excluded_layers

Excluded layers

string

string of list: List of excluded_layers. Examples: -i item1 item2

verbose

verbosity

hidden

TRUE

train

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

CLI

automl_enabled

math_cond

parent_param

depends_on

augmentation_config

augmentation config

collection

Parameters to configure model

FALSE

augmentation_config.rand_hflip

random horizontal flip

bool

Whether to perform random horizontal flip

TRUE

augmentation_config.random_crop_max_scale

maximum scale of random crop

float

The maximum scale of RandomCrop augmentation.

2

1.00E-05

inf

TRUE

augmentation_config.random_crop_min_scale

minimum scale of random crop

float

The minimum scale of RandomCrop augmentation.

0.1

1.00E-05

inf

TRUE

< augmentation_config.random_crop_max_scale

dataset_config

dataset config

collection

Parameters to configure dataset

FALSE

dataset_config.image_size

image size

string

The image dimension as a tuple within quote marks. (height, width) indicates the dimension of the resized and padded input.

1024,1024

yes

FALSE

dataset_config.max_instances_per_image

maximum instances per image

integer

The maximum number of object instances to parse (default: 100)

100

1

inf

dataset_config.num_classes

number of classes

integer

The number of classes. If there are N categories in the annotation, num_classes should be N+1 (background class)

91

2

inf

yes

FALSE

dataset_config.skip_crowd_during_training

skip crowd during training

bool

Specifies whether to skip crowd during training

TRUE

dataset_config.training_file_pattern

training file pattern

hidden

The TFRecord path for training

FALSE

dataset_config.validation_file_pattern

validation file pattern

hidden

The TFRecord path for validation

FALSE

dataset_config.validation_json_file

validation json file

hidden

The annotation file path for validation

FALSE

eval_config

evaluation config

collection

Parameters to configure evaluation

FALSE

eval_config.eval_batch_size

evaluation batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus

16

1

inf

FALSE

eval_config.eval_epoch_cycle

evaluation epoch cycle

integer

The number of training epochs that should run per validation

2

1

inf

FALSE

eval_config.eval_samples

number of samples for evaluation

integer

The number of samples for evaluation

500

1

inf

FALSE

eval_config.max_detections_per_image

maximum detections per image

integer

The maximum number of detections to visualize

100

1

inf

eval_config.min_score_thresh

minimum confidence threshold

float

The lowest confidence of the predicted box and ground truth box that can be considered a match

0.4

0

inf

model_config

model config

collection

Parameters to configure model

FALSE

model_config.anchor_scale

anchor scale

integer

Scale of the base-anchor size to the feature-pyramid stride

4

1

inf

TRUE

model_config.aspect_ratios

aspect ratios

string

A list of tuples representing the aspect ratios of anchors on each pyramid level

[(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]

FALSE

model_config.max_level

maximum level

integer

The maximum level of the output feature pyramid

7

7

FALSE

model_config.min_level

minimum level

integer

The minimum level of the output feature pyramid

3

3

FALSE

model_config.model_name

model name

ordered

Model name

efficientdet-d0

efficientdet-d0, efficientdet-d1, efficientdet-d2, efficientdet-d3, efficientdet-d4, efficientdet-d5

FALSE

model_config.num_scales

number of scales

integer

The number of anchor octave scales on each pyramid level (e.g. if set to 3, the anchor scales are [2^0, 2^(1/3), 2^(2/3)])

3

1

inf

training_config

Training config

collection

Parameters to configure the training process

FALSE

training_config.amp

AMP

bool

Whether to use mixed precision training

TRUE

training_config.checkpoint

path to pretrained model

hidden

The path to the pretrained model, if any

FALSE

training_config.checkpoint_period

checkpoint period

integer

The number of training epochs that should run per model checkpoint/validation

2

1

inf

FALSE

training_config.clip_gradients_norm

clip gradient by norm

float

Clip gradients by the norm value

5.00E+00

0

inf

training_config.iterations_per_loop

integer

10

1

inf

FALSE

training_config.l1_weight_decay

L1 weight decay

float

L1 weight decay

0

0

1

training_config.l2_weight_decay

L2 weight decay

float

L2 weight decay

0.00004

0

inf

TRUE

training_config.learning_rate

maximum learning rate

float

The maximum learning rate

0.02

0

inf

TRUE

training_config.lr_warmup_epoch

learning rate warmup epoch

integer

The number of warmup epochs in the learning rate schedule

3

0

inf

FALSE

<= training_config.num_epochs

training_config.lr_warmup_init

initial learning rate during warmup

float

The initial learning rate in the warmup period

0.002

0

inf

TRUE

training_config.moving_average_decay

moving average decay

float

Moving average decay

0.9999

0

1

TRUE

training_config.num_epochs

number of epochs

integer

The number of epochs to train the network

6

1

inf

FALSE

training_config.num_examples_per_epoch

number of images per epoch per gpu

integer

Total number of images in the training set divided by the number of GPUs

118288

1

inf

FALSE

training_config.pruned_model_path

path to pruned model

hidden

The path to a TAO pruned model for re-training, if any

FALSE

training_config.skip_checkpoint_variables

skip checkpoint variables

string

If specified, the weights of the layers with matching regular expressions will not be loaded. This is especially helpful for transfer learning.

-predict*

FALSE

training_config.tf_random_seed

random seed

integer

The random seed

42

1

inf

FALSE

training_config.train_batch_size

training batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus.

16

1

inf

version

Schema Version

const

The version of this schema

1

FALSE

retrain

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

CLI

version

Schema Version

const

The version of this schema

1

training_config

Training config

collection

Parameters to configure the training process

training_config.train_batch_size

training batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus.

16

0

training_config.iterations_per_loop

integer

10

training_config.num_epochs

number of epochs

integer

The number of epochs to train the network

6

0

training_config.num_examples_per_epoch

number of images per epoch per gpu

integer

Total number of images in the training set divided by the number of GPUs

118288

0

training_config.checkpoint

path to pretrained model

hidden

The path to the pretrained model, if any

training_config.pruned_model_path

path to pruned model

hidden

The path to a TAO pruned model for re-training, if any

training_config.checkpoint_period

checkpoint period

integer

The number of training epochs that should run per model checkpoint/validation

2

0

training_config.amp

AMP

bool

Whether to use mixed precision training

TRUE

training_config.moving_average_decay

moving average decay

float

Moving average decay

0.9999

training_config.l2_weight_decay

L2 weight decay

float

L2 weight decay

0.00004

training_config.l1_weight_decay

L1 weight decay

float

L1 weight decay

0

training_config.lr_warmup_epoch

learning rate warmup epoch

integer

The number of warmup epochs in the learning rate schedule

3

0

training_config.lr_warmup_init

initial learning rate during warmup

float

The initial learning rate in the warmup period

0.002

training_config.learning_rate

maximum learning rate

float

The maximum learning rate

0.02

training_config.tf_random_seed

random seed

integer

The random seed

42

0

training_config.clip_gradients_norm

clip gradient by norm

float

Clip gradients by the norm value

5.00E+00

training_config.skip_checkpoint_variables

skip checkpoint variables

string

If specified, the weights of the layers with matching regular expressions will not be loaded. This is especially helpful for transfer learning.

-predict*

eval_config

evaluation config

collection

Parameters to configure evaluation

eval_config.eval_epoch_cycle

evaluation epoch cycle

integer

The number of training epochs that should run per validation

2

0

eval_config.max_detections_per_image

maximum detections per image

integer

The maximum number of detections to visualize

100

0

eval_config.min_score_thresh

minimum confidence threshold

float

The lowest confidence of the predicted box and ground truth box that can be considered a match

0.4

eval_config.eval_batch_size

evaluation batch size

integer

The batch size for each GPU, so the effective batch size is batch_size_per_gpu * num_gpus

16

0

eval_config.eval_samples

number of samples for evaluation

integer

The number of samples for evaluation

500

dataset_config

dataset config

collection

Parameters to configure dataset

dataset_config.image_size

image size

string

The image dimension as a tuple within quote marks. (height, width) indicates the dimension of the resized and padded input.

1024,1024

yes

dataset_config.training_file_pattern

training file pattern

hidden

The TFRecord path for training

dataset_config.validation_file_pattern

validation file pattern

hidden

The TFRecord path for validation

dataset_config.validation_json_file

validation json file

hidden

The annotation file path for validation

dataset_config.num_classes

number of classes

integer

The number of classes. If there are N categories in the annotation, num_classes should be N+1 (background class)

91

yes

dataset_config.max_instances_per_image

maximum instances per image

integer

The maximum number of object instances to parse (default: 100)

100

dataset_config.skip_crowd_during_training

skip crowd during training

bool

Specifies whether to skip crowd during training

TRUE

model_config

model config

collection

Parameters to configure model

model_config.model_name

model name

string

Model name

efficientdet-d0

model_config.min_level

minimum level

integer

The minimum level of the output feature pyramid

3

model_config.max_level

maximum level

integer

The maximum level of the output feature pyramid

7

model_config.num_scales

number of scales

integer

The number of anchor octave scales on each pyramid level (e.g. if set to 3, the anchor scales are [2^0, 2^(1/3), 2^(2/3)])

3

model_config.aspect_ratios

aspect ratios

string

A list of tuples representing the aspect ratios of anchors on each pyramid level

[(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]

model_config.anchor_scale

anchor scale

integer

Scale of the base-anchor size to the feature-pyramid stride

4

augmentation_config

augmentation config

collection

Parameters to configure model

augmentation_config.rand_hflip

random horizontal flip

bool

Whether to perform random horizontal flip

TRUE

augmentation_config.random_crop_min_scale

minimum scale of random crop

float

The minimum scale of RandomCrop augmentation.

0.1

augmentation_config.random_crop_max_scale

maximum scale of random crop

float

The maximum scale of RandomCrop augmentation.

2

faster_rcnn

export

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

version

Schema Version

const

The version of this schema

1

model

Model

hidden

UNIX path to the model file

0.1

yes

key

Encryption Key

hidden

Encryption key

tlt_encode

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

force_ptq

Force Post-Training Quantization

bool

Force generating int8 engine using Post Training Quantization

FALSE

no

cal_image_dir

hidden

data_type

Pruning Granularity

string

Number of filters to remove at a time.

fp32

int8, fp32, fp16

yes

yes

strict_type_constraints

bool

FALSE

gen_ds_config

bool

FALSE

cal_cache_file

Calibration cache file

hidden

Unix PATH to the int8 calibration cache file

yes

yes

batches

Number of calibration batches

integer

Number of batches to calibrate the model when run in INT8 mode

100

no

max_workspace_size

integer

Example: The integer value of 1<<30, 2<<30

max_batch_size

integer

1

batch_size

Batch size

integer

Number of images per batch when generating the TensorRT engine.

100

yes

min_batch_size

integer

1

opt_batch_size

integer

1

experiment_spec

Experiment Spec

hidden

UNIX path to the Experiment spec file used to train the model. This may be the train or retrain spec file.

yes

engine_file

Engine File

hidden

UNIX path to the model engine file.

yes

static_batch_size

integer

-1

results_dir

hidden

verbose

hidden

TRUE

prune

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

model

Model path

hidden

UNIX path to where the input model is located.

yes

output_file

Output File

hidden

UNIX path to where the pruned model will be saved.

yes

results_dir

Results directory

hidden

key

Encode key

hidden

normalizer

Normalizer

string

How to normalize

max

max, L2

equalization_criterion

Equalization Criterion

string

Criteria to equalize the stats of inputs to an element wise op layer.

union

union, intersection, arithmetic_mean,geometric_mean

no

pruning_granularity

Pruning Granularity

integer

Number of filters to remove at a time.

8

no

pruning_threshold

Pruning Threshold

float

Threshold to compare normalized norm against.

0.1

0

1

yes

yes

min_num_filters

Minimum number of filters

integer

Minimum number of filters to be kept per layer

16

no

excluded_layers

Excluded layers

string

string of list: List of excluded_layers. Examples: -i item1 item2

verbose

verbosity

hidden

TRUE

train

comments

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

automl_enabled

math_cond

parent_param

depends_on

version

Schema Version

const

The version of this schema

1

FALSE

Generates randomness around a point. Seed is where you begin try converging towards. Only required if needed to replicate a run. Does the log push out this value?

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

1

inf

FALSE

verbose

Verbose

bool

Flag of verbosity

TRUE

FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

FALSE

JPG/PNG - auto pick this up

dataset_config.image_extension

Image Extension

ordered

Extension of the images to be used.

png

jpeg,png,jpg

yes

FALSE

Can be system generated - after conversion. This is the dataset preparation step.

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

FALSE

Where the dataset is - where the images are. Will it figure it out from the parent directory?

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

FALSE

Read all labels in the label file (car,truck,suv,person). Ask the user to map it to Vehicle/Person.

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car,van,heavy_truck etc may be grouped under automobile.

FALSE

Class you want to train for (vehicle)

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

FALSE

Class defined in the label file (car,truck,suv -> map to vehicle)

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

FALSE

Default - 0

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords,you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0,N-1]. For random split partitioning,force the validation fold index to 0 as the tfrecord is just 2-fold.

0

0

inf

FALSE

Dataset specific config - augmentation

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

FALSE

The resolution at which the network should be trained for. Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16.

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

160

inf

yes

Yes

FALSE

/ 16

Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

160

inf

yes

Yes

FALSE

/ 16

Smaller side of image(height or width)

augmentation_config.preprocessing.output_image_min

Image smaller side’s size

integer

The smaller side of image size. This is used for resize and keep aspect ratio in FasterRCNN. If this value is postive,preprocessor will resize the image and keep aspect ratio,such that the smaller side’s size is this value. The other side will scale accordingly by aspect ratio. This value has to be a multiple of 16.

0

160

inf

FALSE

/ 16

Limit of larger side’s size of an image when resize and keep aspect ratio

augmentation_config.preprocessing.output_image_max

Limit of larger side’s size when resize and keep aspect ratio

integer

The maximum size of image’s larger side. If after resize and keeping aspect ratio,the larger side is exceeds this limit,the image will be resized such that the larger side’s size is this value,and hence the smaller side’s size is smaller than output_image_min. This value has to be a multiple of 16.

0

160

inf

FALSE

/ 16

Flag to enable automatic image scaling

augmentation_config.preprocessing.enable_auto_resize

Flag to enable or disable automatic image scaling

bool

If True,automatic image scaling will be enabled. Otherwise,disabled.

TRUE

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

10

1

inf

yes

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

10

1

inf

yes

3 channel default

augmentation_config.preprocessing.output_image_channel

Image Channel

ordered_int

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently,1-channel input is not recommended for datasets with JPG images. For PNG images,both 3-channel RGB and 1-channel monochrome images are supported.

3

1

3

1,3

yes

FALSE

0

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

0

inf

yes

0

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

0

inf

yes

0

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

0

inf

yes

0

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

0

inf

yes

0

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

0

inf

yes

0

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

0

inf

yes

Enable - go to default,disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

Enable - go to default,disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

0

1

Enable - go to default,disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

1

Enable - go to default,disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

inf

Enable - go to default,disable - go to 0. Check for the right default values with TAO Toolkit Engg which will disable vs enable.

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

inf

Enable - go to default,disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

inf

Enable go tyo default,disable - 0

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max,rotate_rad_max].

0.69

0

inf

augmentation_config.spatial_augmentation.rotate_probability

Image Rotation

float

The probability of image rotation. The range is [0,1]

0

1

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

0

1

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally,this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0,you can set this value to 0.5.

0.5

0

1

0.5

Might need different defaults based on task/scenario

model_config

Model

collection

FALSE

model_config.arch

BackBone Architecture

ordered

The architecture of the backbone feature extractor to be used for training.

resnet:18

resnet:10,resnet:18,resnet:34,resnet:50,resnet:101,vgg16,vgg:16,vgg:19,googlenet,mobilenet_v1,mobilenet_v2,darknet:19,darknet:53,resnet101,efficientnet:b0,efficientnet:b1

yes

FALSE

Confirm correct default values

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template,and is different for different feature extractor templates.

0

3

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

TRUE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.all_projections

All Projections

bool

For templates with shortcut connections,this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers,irrespective of whether there is a change in stride across the input and output.

TRUE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True,MaxPooling is used to downsample; however,for the object-detection network,NVIDIA recommends setting this to False and using strided convolutions.

FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

0

1

model_config.input_image_config

Input Image

collection

Configuration for input images

FALSE

model_config.input_image_config.size_height_width

collection

FALSE

model_config.input_image_config.size_height_width.height

integer

544

160

inf

/ 16

TRUE

model_config.input_image_config.size_height_width.width

integer

960

160

inf

/ 16

TRUE

model_config.input_image_config.image_type

Image Type

enum

The type of images,either RGB or GRAYSCALE

__RGB__

__RGB__,__GRAYSCALE__

FALSE

model_config.input_image_config.size_min

Image smaller side’s size

integer

The size of an image’s smaller side,should be a multiple of 16. This should be consistent with the size in augmentation_config. This is used when resizing images and keeping aspect ratio

160

inf

FALSE

model_config.input_image_config.image_channel_order

Image Channel Order

ordered

The channel order of images. Should be either “rgb” or “bgr” for RGB images and “l” for GRAYSCALE images

bgr

rgb,bgr,l

FALSE

model_config.input_image_config.image_channel_mean

Image Channel Means

list

A dict from ‘r’,’g’,’b’ or ‘l’(for GRAYSCALE images) to per-channel mean values.

[{“key”:”r”,”value”:103.0},{“key”:”g”,”value”:103.0},{“key”:”b”,”value”:103.0}]

FALSE

model_config.input_image_config.image_channel_mean.key

channel means key

string

string => one of r,g,b

FALSE

model_config.input_image_config.image_channel_mean.value

channel means value

float

value in float

0

255

model_config.input_image_config.image_scaling_factor

Image Scaling Factor

float

A scalar to normalize the images after mean subtraction.

1

0

inf

model_config.input_image_config.max_objects_num_per_image

Max Objects Num

integer

The maximum number of objects in an image. This is used for padding in data loader as different images can have different number of objects in its labels.

100

1

inf

model_config.anchor_box_config

Anchor Boxes

collection

FALSE

model_config.anchor_box_config.scale

Anchor Scales

list

The list of anchor sizes(scales).

[64.0,128.0,256.0]

FALSE

model_config.anchor_box_config.ratio

Anchor Ratios

list

The list of anchor aspect ratios.

[1.0,0.5,2.0]

FALSE

model_config.roi_mini_batch

ROI Batch Size

integer

The batch size of ROIs for training the RCNN in the model

16

0

inf

model_config.rpn_stride

RPN stride

integer

The stride of RPN feature map,compared to input resolutions. Currently only 16 is supported.

16

16

16

FALSE

model_config.drop_connect_rate

Drop Connect Rate

float

The rate of DropConnect. This is only useful for EfficientNet backbones.

0

1

model_config.rpn_cls_activation_type

RPN Classification Activation Type

string

Type of RPN classification head’s activation function. Currently only “sigmoid” is supported.

FALSE

model_config.use_bias

Use Bias

bool

Whether or not to use bias for convolutional layers

FALSE

FALSE

model_config.roi_pooling_config

ROI Pooling

collection

Confiuration fo ROI Pooling layer

FALSE

model_config.roi_pooling_config.pool_size

Pool Size

integer

Pool size of the ROI Pooling operation.

7

0

inf

model_config.roi_pooling_config.pool_size_2x

Pool Size Doubled

bool

Whether or not to double the pool size and apply a 2x downsampling after ROI Pooling

FALSE

model_config.activation

Activation

collection

Activation function for the model backbone. This is only useful for EfficientNet backbones.

FALSE

model_config.activation.activation_type

Activation Type

ordered

Type of the activation function of backbone.

relu,swish

model_config.activation.activation_parameters

Activation Parameters

dict

A dict the maps name of a parameter to its value.

FALSE

training_config

Training

collection

FALSE

IMPORTANT. Open to user - default should smarty calculate. Check factors that influence.

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

8

1

inf

yes

Default - what is the optimal number of epcohs for each model. Smart feature in TAO Toolkit to auto stop once model converges

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

120

1

inf

yes

Yes

FALSE

Toggle for end user

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

FALSE

Default

training_config.learning_rate.soft_start .base_lr

Maximum learning rate during the training

float

5.00E-04

0

inf

Yes

TRUE

Default

training_config.learning_rate.soft_start .start_lr

The initial learning rate at the start

float

5.00E-06

0

inf

Yes

TRUE

< training_config.learning_rate.soft_start .base_lr

Default

training_config.learning_rate.soft_start .soft_start

Soft Start

float

0.100000001

0

1

Yes

TRUE

< training_config.learning_rate.soft_start .annealing_points

Default

training_config.learning_rate.soft_start .annealing_points

Annealing

float

0.8

0

1

Yes

TRUE

Default

training_config.learning_rate.soft_start .annealing_divider

Annealing

float

10

0

inf

Yes

Default

training_config.regularizer.type

Regularizer Type

ordered

The type of the regularizer being used.

__L1__

__NO_REG__,__L1__,__L2__

yes

TRUE

Default

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

0

1

yes

TRUE

Default

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

0

1

yes

Default

training_config.optimizer.adam.beta_1

Optimizer Adam Beta1

float

0.899999976

0

1

yes

Default

training_config.optimizer.adam.beta_2

Optimizer Adam Beta2

float

0.999000013

0

1

yes

TRUE

Use default as 10. Provide last checpoint to user

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

10

0

inf

yes

FALSE

training_config.enable_augmentation

Enable Augmentation

bool

Whether or not to enable data augmentation

TRUE

training_config.retrain_pruned_model

Pruned Model

hidden

The path of pruned model to be retrained

FALSE

training_config.pretrained_weights

Pretrained Weights

hidden

The path of the pretrained model(weights) used to initialize the model being trained

FALSE

training_config.resume_from_model

Resume Model

hidden

The path of the model used to resume a interrupted training

FALSE

training_config.rpn_min_overlap

RPN Min Overlap

float

The lower IoU threshold used to match anchor boxes to groundtruth boxes.

0.1

0

1

training_config.rpn_max_overlap

RPN Max Overlap

float

The higher IoU threshold used to match anchor boxes to groundtruth boxes.

1

0

1

training_config.classifier_min_overlap

Classifier Min Overlap

float

The lower IoU threshold used to generate the proposal target.

0.1

0

1

training_config.classifier_max_overlap

Classifier Max Overlap

float

The higher IoU threshold used to generate the proposal target.

1

0

1

training_config.gt_as_roi

Gt As ROI

bool

A flag to include groundtruth boxes in the positive ROIs for training the RCNN

training_config.std_scaling

RPN Regression Loss Scaling

float

A scaling factor (multiplier) for RPN regression loss

1

0

inf

training_config.classifier_regr_std

RCNN Regression Loss Scaling

list

Scaling factors (denominators) for the RCNN regression loss. A map from ¡®x¡¯,¡®y¡¯,¡®w¡¯,¡®h¡¯ to its corresponding scaling factor,respectively

[{“key”:”x”,”value”:10.0},{“key”:”y”,”value”:10.0},{“key”:”w”,”value”:5.0},{“key”:”h”,”value”:5.0}]

FALSE

training_config.classifier_regr_std.key

RCNN Regression Loss Scaling Key

string

one of x,y,h,w

FALSE

training_config.classifier_regr_std.value

RCNN Regression Loss Scaling Value

float

float value for key

0

inf

FALSE

training_config.output_model

Output Model Path

hidden

Path of the output model

FALSE

training_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer

12000

1

inf

training_config.rpn_mini_batch

RPN Mini Batch

integer

The batch size to train RPN

16

1

inf

training_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

2000

1

inf

training_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

0

1

training_config.lambda_rpn_regr

RPN Regression Loss Weighting

float

Weighting factor for RPN regression loss

1

0

inf

training_config.lambda_rpn_class

RPN classification Loss Weighting

float

Weighting factor for RPN classification loss.

1

0

inf

training_config.lambda_cls_regr

RCNN Regression Loss Weighting

float

Weighting factor for RCNN regression loss

1

0

inf

training_config.lambda_cls_class

RCNN Classification Loss Weighting

float

Weighting factor for RCNN classification loss

1

training_config.model_parallelism

Model Parallelism

list of floats

List of fractions for model parallelism

FALSE

training_config.early_stopping

Early Stopping

collection

FALSE

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

loss

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

0

1

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

1

5

training_config.visualizer

Visualizer

collection

FALSE

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

FALSE

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

1

inf

FALSE

evaluation_config

Evaluation

collection

yes

FALSE

evaluation_config.model

Model Path

string

The path to the model to run inference

FALSE

evaluation_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during evaluation

6000

1

inf

evaluation_config.rpn_nms_overlap_threshold

RPN overlap threshold

float

0.7

0

1

evaluation_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

1

inf

evaluation_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

1

inf

evaluation_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

0

1

evaluation_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

0

1

evaluation_config.use_voc07_11point_metric

Use VOC 11-point Metric

bool

Whether to use PASCAL-VOC 11-point metric

FALSE

evaluation_config.validation_period_during_training

Validation Period

integer

The period(number of epochs) to run validation during training

1

inf

FALSE

evaluation_config.batch_size

Batch Size

integer

The batch size for evaluation

1

inf

FALSE

evaluation_config.trt_evaluation

TensorRT Evaluation

collection

TensorRT evaluation

FALSE

evaluation_config.trt_evaluation.trt_engine

Trt Engine

string

TRT Engine

FALSE

evaluation_config.gt_matching_iou_threshold

Gt Matching IoU Threshold

float

The IoU threshold to match groundtruth to detected objects. Only one of this collection or gt_matching_iou_threshold_range

0.5

0

1

evaluation_config.gt_matching_iou_threshold_range

Gt Matching IoU Threshold Range

collection

Only one of this collection or gt_matching_iou_threshold

FALSE

evaluation_config.gt_matching_iou_threshold_range.start

Start

float

The starting value of the IoU range

0

1

evaluation_config.gt_matching_iou_threshold_range.end

End

float

The end point of the IoU range(exclusive)

0

1

evaluation_config.gt_matching_iou_threshold_range.step

Step

float

The step size of the IoU range

0

1

evaluation_config.visualize_pr_curve

Visualize PR Curve

bool

Visualize precision-recall curve or not

FALSE

inference_config

FALSE

inference_config.images_dir

Images Directory

hidden

Path to the directory of images to run inference on

FALSE

inference_config.model

Model Path

hidden

Path to the model to run inference on

FALSE

inference_config.batch_size

Batch Size

integer

The batch size for inference

1

inf

FALSE

inference_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during inference

6000

1

inf

FALSE

inference_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

1

inf

FALSE

inference_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

0

1

FALSE

inference_config.bbox_visualize_threshold

Visualization Threshold

float

The confidence threshold for visualizing the bounding boxes

0.6

0

1

FALSE

inference_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

0

1

FALSE

inference_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

1

inf

FALSE

inference_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

0

1

FALSE

inference_config.detection_image_output_dir

Image Output Directory

string

Path to the directory to save the output images during inference

FALSE

inference_config.bbox_caption_on

Bbox Caption

bool

Enable text caption for bounding box or not

FALSE

inference_config.labels_dump_dir

Labels Ouptut Directory

hidden

Path to the directory to save the output labels

FALSE

inference_config.nms_score_bits

NMS Score Bits

integer

Number of score bits in optimized NMS

1

10

FALSE

inference_config.trt_inference

TensorRT Inference

Collection

TensorRT inference configurations

FALSE

inference_config.trt_inference.trt_engine

TensorRT Engine

hidden

Path to the TensorRT engine to run inference

FALSE

inference

comments

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

valid_options_description

version

Schema Version

const

The version of this schema

1

Generates randomness around a point. Seed is where you begin try converging towards. Only required if needed to replicate a run. Does the log push out this value?

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

>=0

verbose

Verbose

bool

Flag of verbosity

TRUE

TRUE, FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

JPG/PNG - auto pick this up

dataset_config.image_extension

Image Extension

string

Extension of the images to be used.

png

png,jpg

yes

__png__, __jpg__, __jpeg__

Can be system generated - after conversion. This is the dataset preparation step.

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

Where the dataset is - where the images are. Will it figure it out from the parent directory?

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

Read all labels in the label file (car, truck, suv, person). Ask the user to map it to Vehicle/Person.

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

Class you want to train for (vehicle)

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

Class defined in the label file (car, truck, suv -> map to vehicle)

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

Default - 0

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords, you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0, N-1]. For random split partitioning, force the validation fold index to 0 as the tfrecord is just 2-fold.

0

Dataset specific config - augmentation

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

The resolution at which the network should be trained for. Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16.

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

480

yes

Yes

Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

272

yes

Yes

Smaller side of image(height or width)

augmentation_config.preprocessing.output_image_min

Image smaller side’s size

integer

The smaller side of image size. This is used for resize and keep aspect ratio in FasterRCNN. If this value is postive, preprocessor will resize the image and keep aspect ratio, such that the smaller side’s size is this value. The other side will scale accordingly by aspect ratio. This value has to be a multiple of 16.

0

Limit of larger side’s size of an image when resize and keep aspect ratio

augmentation_config.preprocessing.output_image_max

Limit of larger side’s size when resize and keep aspect ratio

integer

The maximum size of image’s larger side. If after resize and keeping aspect ratio, the larger side is exceeds this limit, the image will be resized such that the larger side’s size is this value, and hence the smaller side’s size is smaller than output_image_min. This value has to be a multiple of 16.

0

Flag to enable automatic image scaling

augmentation_config.preprocessing.enable_auto_resize

Flag to enable or disable automatic image scaling

bool

If True, automatic image scaling will be enabled. Otherwise, disabled.

TRUE

TRUE, FALSE

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

1

0

yes

>=0

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

1

0

yes

>=0

3 channel default

augmentation_config.preprocessing.output_image_channel

Image Channel

integer

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently, 1-channel input is not recommended for datasets with JPG images. For PNG images, both 3-channel RGB and 1-channel monochrome images are supported.

3

1, 3

yes

3, 1

0

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

0

yes

>=0

0

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

0

yes

>=0

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

[0, 1)

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

0

1

[0, 1)

Enable - go to default, disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

(0, 1]

Enable - go to default, disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

[1, 2)

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg which will disable vs enable.

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

>=0

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

>=0

Enable go tyo default, disable - 0

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max, rotate_rad_max].

0.69

0

>=0

augmentation_config.spatial_augmentation.rotate_probability

Image Rotation

float

The probability of image rotation. The range is [0, 1]

[0, 1)

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

0

1

[0, 1)

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

[0, 360)

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

[0, 1)

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

[0, 1)

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally, this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0, you can set this value to 0.5.

0.5

0.5

0.5

Might need different defaults based on task/scenario

model_config

Model

collection

model_config.arch

BackBone Architecture

string

The architecture of the backbone feature extractor to be used for training.

resnet:18

resnet:18

yes

resnet:10’,

‘resnet:18’, ‘resnet:34’, ‘resnet:50’, ‘resnet:101’, ‘vgg16’, ‘vgg:16’, ‘vgg:19’, ‘googlenet’, ‘mobilenet_v1’, ‘mobilenet_v2’, ‘darknet:19’, ‘darknet:53’, ‘resnet101’, ‘efficientnet:b0’, ‘efficientnet:b1’,

Confirm correct default values

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template, and is different for different feature extractor templates.

0

3

depends on arch

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

FALSE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.all_projections

All Projections

bool

For templates with shortcut connections, this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers, irrespective of whether there is a change in stride across the input and output.

TRUE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True, MaxPooling is used to downsample; however, for the object-detection network, NVIDIA recommends setting this to False and using strided convolutions.

FALSE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

0

0.1

[0, 1)

model_config.input_image_config

Input Image

collection

Configuration for input images

model_config.input_image_config.size_height_width

collection

model_config.input_image_config.size_height_width.height

integer

544

model_config.input_image_config.size_height_width.width

integer

960

model_config.input_image_config.image_type

Image Type

enum

The type of images, either RGB or GRAYSCALE

__RGB__

__RGB__, __GRAYSCALE__

model_config.input_image_config.size_min

Image smaller side’s size

integer

The size of an image’s smaller side, should be a multiple of 16. This should be consistent with the size in augmentation_config. This is used when resizing images and keeping aspect ratio

>=0

model_config.input_image_config.size_height_width

Image size by height and width

collection

The size of images by specifying height and width.

model_config.input_image_config.size_height_width.height

Image Height

integer

The height of images

>=0

model_config.input_image_config.size_height_width.width

Image Width

integer

The width of images

>=0

model_config.input_image_config.image_channel_order

Image Channel Order

string

The channel order of images. Should be either “rgb” or “bgr” for RGB images and “l” for GRAYSCALE images

bgr

rgb’, ‘bgr’, ‘l’

model_config.input_image_config.image_channel_mean

Image Channel Means

list

A dict from ‘r’, ‘g’, ‘b’ or ‘l’(for GRAYSCALE images) to per-channel mean values.

[{“key”:”r”,”value”:103.0}, {“key”:”g”,”value”:103.0}, {“key”:”b”,”value”:103.0}]

model_config.input_image_config.image_channel_mean.key

channel means key

string

string => one of r,g,b

r’, ‘g’, ‘b’, ‘l’

model_config.input_image_config.image_channel_mean.value

channel means value

float

value in float

(0, 255)

model_config.input_image_config.image_scaling_factor

Image Scaling Factor

float

A scalar to normalize the images after mean subtraction.

1

>0

model_config.input_image_config.max_objects_num_per_image

Max Objects Num

integer

The maximum number of objects in an image. This is used for padding in data loader as different images can have different number of objects in its labels.

100

>=1

model_config.anchor_box_config

Anchor Boxes

Collection

model_config.anchor_box_config.scale

Anchor Scales

list

The list of anchor sizes(scales).

[64.0,128.0,256.0]

>0

model_config.anchor_box_config.ratio

Anchor Ratios

list

The list of anchor aspect ratios.

[1.0,0.5,2.0]

>0

model_config.roi_mini_batch

ROI Batch Size

integer

The batch size of ROIs for training the RCNN in the model

16

>0

model_config.rpn_stride

RPN stride

integer

The stride of RPN feature map, compared to input resolutions. Currently only 16 is supported.

16

16

model_config.drop_connect_rate

Drop Connect Rate

float

The rate of DropConnect. This is only useful for EfficientNet backbones.

(0, 1)

model_config.rpn_cls_activation_type

RPN Classification Activation Type

string

Type of RPN classification head’s activation function. Currently only “sigmoid” is supported.

sigmoid

model_config.use_bias

Use Bias

bool

Whether or not to use bias for convolutional layers

TRUE, FALSE

model_config.roi_pooling_config

ROI Pooling

collection

Confiuration fo ROI Pooling layer

model_config.roi_pooling_config.pool_size

Pool Size

integer

Pool size of the ROI Pooling operation.

7

>0

model_config.roi_pooling_config.pool_size_2x

Pool Size Doubled

bool

Whether or not to double the pool size and apply a 2x downsampling after ROI Pooling

FALSE

TRUE, FALSE

model_config.activation

Activation

collection

Activation function for the model backbone. This is only useful for EfficientNet backbones.

model_config.activation.activation_type

Activation Type

string

Type of the activation function of backbone.

relu, swish

model_config.activation.activation_parameters

Activation Parameters

dict

A dict the maps name of a parameter to its value.

training_config

Training

collection

>0

IMPORTANT. Open to user - default should smarty calculate. Check factors that influence.

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

8

1

yes

>0

Default - what is the optimal number of epcohs for each model. Smart feature in TAO Toolkit to auto stop once model converges

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

120

1

yes

Yes

TRUE, FALSE

Toggle for end user

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

>0

Default

training_config.learning_rate.soft_start .base_lr

Minimum Learning Rate

float

5.00E-06

Yes

>0

Default

training_config.learning_rate.soft_start .start_lr

Maximum Learning Rate

float

5.00E-04

Yes

(0, 1)

Default

training_config.learning_rate.soft_start .soft_start

Soft Start

float

0.100000001

0

1

Yes

>1

Default

training_config.learning_rate.soft_start .annealing_divider

Annealing

float

0.699999988

0

1

Yes

__NO_REG__, __L1__, __L2__

Default

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__NO_REG__, __L1__, __L2__

yes

>0

Default

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

yes

(0, 1)

Default

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

yes

(0, 1)

Default

training_config.optimizer.adam.beta_1

Optimizer Adam Beta1

float

0.899999976

yes

(0, 1)

Default

training_config.optimizer.adam.beta_2

Optimizer Adam Beta2

float

0.999000013

yes

>=1

Use default as 10. Provide last checpoint to user

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

10

0

yes

TRUE, FALSE

training_config.enable_augmentation

Enable Augmentation

bool

Whether or not to enable data augmentation

TRUE

training_config.retrain_pruned_model

Pruned Model

hidden

The path of pruned model to be retrained

training_config.pretrained_weights

Pretrained Weights

hidden

The path of the pretrained model(weights) used to initialize the model being trained

training_config.resume_from_model

Resume Model

hidden

The path of the model used to resume a interrupted training

(0, 1)

training_config.rpn_min_overlap

RPN Min Overlap

float

The lower IoU threshold used to match anchor boxes to groundtruth boxes.

0.1

(0, 1)

training_config.rpn_max_overlap

RPN Max Overlap

float

The higher IoU threshold used to match anchor boxes to groundtruth boxes.

1

[0, 1)

training_config.classifier_min_overlap

Classifier Min Overlap

float

The lower IoU threshold used to generate the proposal target.

0.1

(0, 1)

training_config.classifier_max_overlap

Classifier Max Overlap

float

The higher IoU threshold used to generate the proposal target.

1

TRUE, FALSE

training_config.gt_as_roi

Gt As ROI

bool

A flag to include groundtruth boxes in the positive ROIs for training the RCNN

>0

training_config.std_scaling

RPN Regression Loss Scaling

float

A scaling factor (multiplier) for RPN regression loss

1

training_config.classifier_regr_std

RCNN Regression Loss Scaling

list

Scaling factors (denominators) for the RCNN regression loss. A map from ¡®x¡¯, ¡®y¡¯, ¡®w¡¯, ¡®h¡¯ to its corresponding scaling factor, respectively

[{“key”:”x”,”value”:10.0},{“key”:”y”,”value”:10.0},{“key”:”w”,”value”:5.0},{“key”:”h”,”value”:5.0}]

training_config.classifier_regr_std.key

RCNN Regression Loss Scaling Key

string

one of x,y,h,w

>0

training_config.classifier_regr_std.value

RCNN Regression Loss Scaling Value

float

float value for key

training_config.output_model

Output Model Path

hidden

Path of the output model

>0

training_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer

12000

>=1

training_config.rpn_mini_batch

RPN Mini Batch

integer

The batch size to train RPN

16

>0

training_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

2000

(0, 1)

training_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

>0

training_config.lambda_rpn_regr

RPN Regression Loss Weighting

float

Weighting factor for RPN regression loss

1

>0

training_config.lambda_rpn_class

RPN classification Loss Weighting

float

Weighting factor for RPN classification loss.

1

>0

training_config.lambda_cls_regr

RCNN Regression Loss Weighting

float

Weighting factor for RCNN regression loss

1

>0

training_config.lambda_cls_class

RCNN Classification Loss Weighting

float

Weighting factor for RCNN classification loss

1

list of floats

training_config.model_parallelism

Model Parallelism

list of floats

List of fractions for model parallelism

training_config.early_stopping

Early Stopping

collection

“loss”

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

>=0

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

>0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

training_config.visualizer

Visualizer

collection

TRUE, False

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

>=1

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

evaluation_config

Evaluation

collection

yes

evaluation_config.model

Model Path

string

The path to the model to run inference

>=1

evaluation_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during evaluation

6000

(0, 1)

evaluation_config.rpn_nms_overlap_threshold

RPN overlap threshold

float

0.7

>0

evaluation_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

>0

evaluation_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

(0, 1)

evaluation_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

(0, 1)

evaluation_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

TRUE, FALSE

evaluation_config.use_voc07_11point_metric

Use VOC 11-point Metric

bool

Whether to use PASCAL-VOC 11-point metric

>=1

evaluation_config.validation_period_during_training

Validation Period

integer

The period(number of epochs) to run validation during training

>=1

evaluation_config.batch_size

Batch Size

integer

The batch size for evaluation

(0, 1)

evaluation_config.trt_evaluation

TensorRT Evaluation

Collection

TensorRT evaluation

evaluation_config.trt_evaluation.trt_engine

Trt Engine

String

TRT Engine

(0, 1)

evaluation_config.gt_matching_iou_threshold

Gt Matching IoU Threshold

float

The IoU threshold to match groundtruth to detected objects. Only one of this collection or gt_matching_iou_threshold_range

0.5

(0, 1)

evaluation_config.gt_matching_iou_threshold_range

Gt Matching IoU Threshold Range

collection

Only one of this collection or gt_matching_iou_threshold

(0, 1)

evaluation_config.gt_matching_iou_threshold_range.start

Start

float

The starting value of the IoU range

TRUE, FALSE

evaluation_config.gt_matching_iou_threshold_range.end

End

float

The end point of the IoU range(exclusive)

evaluation_config.gt_matching_iou_threshold_range.step

Step

float

The step size of the IoU range

evaluation_config.visualize_pr_curve

Visualize PR Curve

bool

Visualize precision-recall curve or not

inference_config

>=1

inference_config.images_dir

Images Directory

hidden

Path to the directory of images to run inference on

>0

inference_config.model

Model Path

hidden

Path to the model to run inference on

>0

inference_config.batch_size

Batch Size

integer

The batch size for inference

(0, 1)

inference_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during inference

6000

(0, 1)

inference_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

(0, 1)

inference_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

>0

inference_config.bbox_visualize_threshold

Visualization Threshold

float

The confidence threshold for visualizing the bounding boxes

0.6

(0, 1)

inference_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

inference_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

True, False

inference_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

inference_config.detection_image_output_dir

Image Output Directory

string

Path to the directory to save the output images during inference

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

inference_config.bbox_caption_on

Bbox Caption

bool

Enable text caption for bounding box or not

inference_config.labels_dump_dir

Labels Ouptut Directory

hidden

Path to the directory to save the output labels

inference_config.nms_score_bits

NMS Score Bits

integer

Number of score bits in optimized NMS

inference_config.trt_inference

TensorRT Inference

Collection

TensorRT inference configurations

inference_config.trt_inference.trt_engine

TensorRT Engine

hidden

Path to the TensorRT engine to run inference

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

e

engine file path

hidden

k

encode key

hidden

c

cache_file

hidden

o

outputs

string

comma separated list of output node names

d

input_dims

string

comma separated list of input dimensions (not required for TLT 3.0 new models).

yes

yes

b

batch_size

integer

calibration batch size

8

yes

m

max_batch_size

integer

maximum TensorRT engine batch size (default 16). If meet with out-of-memory issue, please decrease the batch size accordingly.

16

yes

w

max_workspace_size

integer

maximum workspace size of TensorRT engine (default 1<<30). If meet with out-of-memory issue, please increase the workspace size accordingly.

t

data_type

string

TensorRT data type

fp32

fp32, fp16, int8

yes

i

input_order

string

input dimension ordering

nchw

nchw, nhwc, nc

s

strict_type_constraints

bool

TensorRT strict_type_constraints flag for INT8 mode

FALSE

u

dla_core

int

Use DLA core N for layers that support DLA (default = -1, which means no DLA core will be utilized for inference. Note that it’ll always allow GPU fallback).

-1

p

parse_profile_shapes

string

comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape>, where each shape has x as delimiter, e.g.,NxC, NxCxHxW, NxCxDxHxW, etc. Can be specified multiple times if there are multiple input tensors for the model. This argument is only useful in dynamic shape case.

model

etlt model from export

hidden

evaluate

comments

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

valid_options_description

version

Schema Version

const

The version of this schema

1

Generates randomness around a point. Seed is where you begin try converging towards. Only required if needed to replicate a run. Does the log push out this value?

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

>=0

verbose

Verbose

bool

Flag of verbosity

TRUE

TRUE, FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

JPG/PNG - auto pick this up

dataset_config.image_extension

Image Extension

string

Extension of the images to be used.

png

png,jpg, __jpeg__

yes

__png__, __jpg__, __jpeg__

Can be system generated - after conversion. This is the dataset preparation step.

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

Where the dataset is - where the images are. Will it figure it out from the parent directory?

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

Read all labels in the label file (car, truck, suv, person). Ask the user to map it to Vehicle/Person.

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

Class you want to train for (vehicle)

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

Class defined in the label file (car, truck, suv -> map to vehicle)

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

Default - 0

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords, you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0, N-1]. For random split partitioning, force the validation fold index to 0 as the tfrecord is just 2-fold.

0

Dataset specific config - augmentation

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

The resolution at which the network should be trained for. Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16.

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

480

yes

Yes

Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

272

yes

Yes

Smaller side of image(height or width)

augmentation_config.preprocessing.output_image_min

Image smaller side’s size

integer

The smaller side of image size. This is used for resize and keep aspect ratio in FasterRCNN. If this value is postive, preprocessor will resize the image and keep aspect ratio, such that the smaller side’s size is this value. The other side will scale accordingly by aspect ratio. This value has to be a multiple of 16.

0

Limit of larger side’s size of an image when resize and keep aspect ratio

augmentation_config.preprocessing.output_image_max

Limit of larger side’s size when resize and keep aspect ratio

integer

The maximum size of image’s larger side. If after resize and keeping aspect ratio, the larger side is exceeds this limit, the image will be resized such that the larger side’s size is this value, and hence the smaller side’s size is smaller than output_image_min. This value has to be a multiple of 16.

0

Flag to enable automatic image scaling

augmentation_config.preprocessing.enable_auto_resize

Flag to enable or disable automatic image scaling

bool

If True, automatic image scaling will be enabled. Otherwise, disabled.

TRUE

TRUE, FALSE

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

1

0

yes

>=0

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

1

0

yes

>=0

3 channel default

augmentation_config.preprocessing.output_image_channel

Image Channel

integer

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently, 1-channel input is not recommended for datasets with JPG images. For PNG images, both 3-channel RGB and 1-channel monochrome images are supported.

3

1, 3

yes

3, 1

0

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

0

yes

>=0

0

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

0

yes

>=0

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

[0, 1)

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

0

1

[0, 1)

Enable - go to default, disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

(0, 1]

Enable - go to default, disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

[1, 2)

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg which will disable vs enable.

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

>=0

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

>=0

Enable go tyo default, disable - 0

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max, rotate_rad_max].

0.69

0

>=0

augmentation_config.spatial_augmentation.rotate_probability

Image Rotation

float

The probability of image rotation. The range is [0, 1]

[0, 1)

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

0

1

[0, 1)

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

[0, 360)

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

[0, 1)

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

[0, 1)

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally, this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0, you can set this value to 0.5.

0.5

0.5

0.5

Might need different defaults based on task/scenario

model_config

Model

collection

model_config.arch

BackBone Architecture

string

The architecture of the backbone feature extractor to be used for training.

resnet:18

resnet:18

yes

resnet:10’,

‘resnet:18’, ‘resnet:34’, ‘resnet:50’, ‘resnet:101’, ‘vgg16’, ‘vgg:16’, ‘vgg:19’, ‘googlenet’, ‘mobilenet_v1’, ‘mobilenet_v2’, ‘darknet:19’, ‘darknet:53’, ‘resnet101’, ‘efficientnet:b0’, ‘efficientnet:b1’,

Confirm correct default values

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template, and is different for different feature extractor templates.

0

3

depends on arch

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

FALSE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.all_projections

All Projections

bool

For templates with shortcut connections, this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers, irrespective of whether there is a change in stride across the input and output.

TRUE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True, MaxPooling is used to downsample; however, for the object-detection network, NVIDIA recommends setting this to False and using strided convolutions.

FALSE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

0

0.1

[0, 1)

model_config.input_image_config

Input Image

collection

Configuration for input images

model_config.input_image_config.size_height_width

collection

model_config.input_image_config.size_height_width.height

integer

544

model_config.input_image_config.size_height_width.width

integer

960

model_config.input_image_config.image_type

Image Type

enum

The type of images, either RGB or GRAYSCALE

__RGB__

__RGB__, __GRAYSCALE__

model_config.input_image_config.size_min

Image smaller side’s size

integer

The size of an image’s smaller side, should be a multiple of 16. This should be consistent with the size in augmentation_config. This is used when resizing images and keeping aspect ratio

>=0

model_config.input_image_config.size_height_width

Image size by height and width

collection

The size of images by specifying height and width.

model_config.input_image_config.size_height_width.height

Image Height

integer

The height of images

>=0

model_config.input_image_config.size_height_width.width

Image Width

integer

The width of images

>=0

model_config.input_image_config.image_channel_order

Image Channel Order

string

The channel order of images. Should be either “rgb” or “bgr” for RGB images and “l” for GRAYSCALE images

bgr

rgb’, ‘bgr’, ‘l’

model_config.input_image_config.image_channel_mean

Image Channel Means

list

A dict from ‘r’, ‘g’, ‘b’ or ‘l’(for GRAYSCALE images) to per-channel mean values.

[{“key”:”r”,”value”:103.0}, {“key”:”g”,”value”:103.0}, {“key”:”b”,”value”:103.0}]

model_config.input_image_config.image_channel_mean.key

channel means key

string

string => one of r,g,b

r’, ‘g’, ‘b’, ‘l’

model_config.input_image_config.image_channel_mean.value

channel means value

float

value in float

(0, 255)

model_config.input_image_config.image_scaling_factor

Image Scaling Factor

float

A scalar to normalize the images after mean subtraction.

1

>0

model_config.input_image_config.max_objects_num_per_image

Max Objects Num

integer

The maximum number of objects in an image. This is used for padding in data loader as different images can have different number of objects in its labels.

100

>=1

model_config.anchor_box_config

Anchor Boxes

Collection

model_config.anchor_box_config.scale

Anchor Scales

list

The list of anchor sizes(scales).

[64.0,128.0,256.0]

>0

model_config.anchor_box_config.ratio

Anchor Ratios

list

The list of anchor aspect ratios.

[1.0,0.5,2.0]

>0

model_config.roi_mini_batch

ROI Batch Size

integer

The batch size of ROIs for training the RCNN in the model

16

>0

model_config.rpn_stride

RPN stride

integer

The stride of RPN feature map, compared to input resolutions. Currently only 16 is supported.

16

16

model_config.drop_connect_rate

Drop Connect Rate

float

The rate of DropConnect. This is only useful for EfficientNet backbones.

(0, 1)

model_config.rpn_cls_activation_type

RPN Classification Activation Type

string

Type of RPN classification head’s activation function. Currently only “sigmoid” is supported.

sigmoid

model_config.use_bias

Use Bias

bool

Whether or not to use bias for convolutional layers

TRUE, FALSE

model_config.roi_pooling_config

ROI Pooling

collection

Confiuration fo ROI Pooling layer

model_config.roi_pooling_config.pool_size

Pool Size

integer

Pool size of the ROI Pooling operation.

7

>0

model_config.roi_pooling_config.pool_size_2x

Pool Size Doubled

bool

Whether or not to double the pool size and apply a 2x downsampling after ROI Pooling

FALSE

TRUE, FALSE

model_config.activation

Activation

collection

Activation function for the model backbone. This is only useful for EfficientNet backbones.

model_config.activation.activation_type

Activation Type

string

Type of the activation function of backbone.

relu, swish

model_config.activation.activation_parameters

Activation Parameters

dict

A dict the maps name of a parameter to its value.

training_config

Training

collection

>0

IMPORTANT. Open to user - default should smarty calculate. Check factors that influence.

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

8

1

yes

>0

Default - what is the optimal number of epcohs for each model. Smart feature in TAO Toolkit to auto stop once model converges

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

120

1

yes

Yes

TRUE, FALSE

Toggle for end user

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

>0

Default

training_config.learning_rate.soft_start .base_lr

Minimum Learning Rate

float

5.00E-06

Yes

>0

Default

training_config.learning_rate.soft_start .start_lr

Maximum Learning Rate

float

5.00E-04

Yes

(0, 1)

Default

training_config.learning_rate.soft_start .soft_start

Soft Start

float

0.100000001

0

1

Yes

>1

Default

training_config.learning_rate.soft_start .annealing_divider

Annealing

float

0.699999988

0

1

Yes

__NO_REG__, __L1__, __L2__

Default

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__NO_REG__, __L1__, __L2__

yes

>0

Default

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

yes

(0, 1)

Default

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

yes

(0, 1)

Default

training_config.optimizer.adam.beta_1

Optimizer Adam Beta1

float

0.899999976

yes

(0, 1)

Default

training_config.optimizer.adam.beta_2

Optimizer Adam Beta2

float

0.999000013

yes

>=1

Use default as 10. Provide last checpoint to user

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

10

0

yes

TRUE, FALSE

training_config.enable_augmentation

Enable Augmentation

bool

Whether or not to enable data augmentation

TRUE

training_config.retrain_pruned_model

Pruned Model

hidden

The path of pruned model to be retrained

training_config.pretrained_weights

Pretrained Weights

hidden

The path of the pretrained model(weights) used to initialize the model being trained

training_config.resume_from_model

Resume Model

hidden

The path of the model used to resume a interrupted training

(0, 1)

training_config.rpn_min_overlap

RPN Min Overlap

float

The lower IoU threshold used to match anchor boxes to groundtruth boxes.

0.1

(0, 1)

training_config.rpn_max_overlap

RPN Max Overlap

float

The higher IoU threshold used to match anchor boxes to groundtruth boxes.

1

[0, 1)

training_config.classifier_min_overlap

Classifier Min Overlap

float

The lower IoU threshold used to generate the proposal target.

0.1

(0, 1)

training_config.classifier_max_overlap

Classifier Max Overlap

float

The higher IoU threshold used to generate the proposal target.

1

TRUE, FALSE

training_config.gt_as_roi

Gt As ROI

bool

A flag to include groundtruth boxes in the positive ROIs for training the RCNN

>0

training_config.std_scaling

RPN Regression Loss Scaling

float

A scaling factor (multiplier) for RPN regression loss

1

training_config.classifier_regr_std

RCNN Regression Loss Scaling

list

Scaling factors (denominators) for the RCNN regression loss. A map from ¡®x¡¯, ¡®y¡¯, ¡®w¡¯, ¡®h¡¯ to its corresponding scaling factor, respectively

[{“key”:”x”,”value”:10.0},{“key”:”y”,”value”:10.0},{“key”:”w”,”value”:5.0},{“key”:”h”,”value”:5.0}]

training_config.classifier_regr_std.key

RCNN Regression Loss Scaling Key

string

one of x,y,h,w

>0

training_config.classifier_regr_std.value

RCNN Regression Loss Scaling Value

float

float value for key

training_config.output_model

Output Model Path

hidden

Path of the output model

>0

training_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer

12000

>=1

training_config.rpn_mini_batch

RPN Mini Batch

integer

The batch size to train RPN

16

>0

training_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

2000

(0, 1)

training_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

>0

training_config.lambda_rpn_regr

RPN Regression Loss Weighting

float

Weighting factor for RPN regression loss

1

>0

training_config.lambda_rpn_class

RPN classification Loss Weighting

float

Weighting factor for RPN classification loss.

1

>0

training_config.lambda_cls_regr

RCNN Regression Loss Weighting

float

Weighting factor for RCNN regression loss

1

>0

training_config.lambda_cls_class

RCNN Classification Loss Weighting

float

Weighting factor for RCNN classification loss

1

list of floats

training_config.model_parallelism

Model Parallelism

list of floats

List of fractions for model parallelism

training_config.early_stopping

Early Stopping

collection

“loss”

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

>=0

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

>0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

training_config.visualizer

Visualizer

collection

TRUE, False

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

>=1

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

evaluation_config

Evaluation

collection

yes

evaluation_config.model

Model Path

string

The path to the model to run inference

>=1

evaluation_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during evaluation

6000

(0, 1)

evaluation_config.rpn_nms_overlap_threshold

RPN overlap threshold

float

0.7

>0

evaluation_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

>0

evaluation_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

(0, 1)

evaluation_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

(0, 1)

evaluation_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

TRUE, FALSE

evaluation_config.use_voc07_11point_metric

Use VOC 11-point Metric

bool

Whether to use PASCAL-VOC 11-point metric

>=1

evaluation_config.validation_period_during_training

Validation Period

integer

The period(number of epochs) to run validation during training

>=1

evaluation_config.batch_size

Batch Size

integer

The batch size for evaluation

(0, 1)

evaluation_config.trt_evaluation

TensorRT Evaluation

Collection

TensorRT evaluation

evaluation_config.trt_evaluation.trt_engine

Trt Engine

String

TRT Engine

(0, 1)

evaluation_config.gt_matching_iou_threshold

Gt Matching IoU Threshold

float

The IoU threshold to match groundtruth to detected objects. Only one of this collection or gt_matching_iou_threshold_range

0.5

(0, 1)

evaluation_config.gt_matching_iou_threshold_range

Gt Matching IoU Threshold Range

collection

Only one of this collection or gt_matching_iou_threshold

(0, 1)

evaluation_config.gt_matching_iou_threshold_range.start

Start

float

The starting value of the IoU range

TRUE, FALSE

evaluation_config.gt_matching_iou_threshold_range.end

End

float

The end point of the IoU range(exclusive)

evaluation_config.gt_matching_iou_threshold_range.step

Step

float

The step size of the IoU range

evaluation_config.visualize_pr_curve

Visualize PR Curve

bool

Visualize precision-recall curve or not

inference_config

>=1

inference_config.images_dir

Images Directory

hidden

Path to the directory of images to run inference on

>0

inference_config.model

Model Path

hidden

Path to the model to run inference on

>0

inference_config.batch_size

Batch Size

integer

The batch size for inference

(0, 1)

inference_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during inference

6000

(0, 1)

inference_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

(0, 1)

inference_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

>0

inference_config.bbox_visualize_threshold

Visualization Threshold

float

The confidence threshold for visualizing the bounding boxes

0.6

(0, 1)

inference_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

inference_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

True, False

inference_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

inference_config.detection_image_output_dir

Image Output Directory

string

Path to the directory to save the output images during inference

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

inference_config.bbox_caption_on

Bbox Caption

bool

Enable text caption for bounding box or not

inference_config.labels_dump_dir

Labels Ouptut Directory

hidden

Path to the directory to save the output labels

inference_config.nms_score_bits

NMS Score Bits

integer

Number of score bits in optimized NMS

inference_config.trt_inference

TensorRT Inference

Collection

TensorRT inference configurations

inference_config.trt_inference.trt_engine

TensorRT Engine

hidden

Path to the TensorRT engine to run inference

retrain

comments

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

valid_options_description

version

Schema Version

const

The version of this schema

1

Generates randomness around a point. Seed is where you begin try converging towards. Only required if needed to replicate a run. Does the log push out this value?

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

>=0

verbose

Verbose

bool

Flag of verbosity

TRUE

TRUE, FALSE

dataset_config

Dataset

collection

Parameters to configure the dataset

JPG/PNG - auto pick this up

dataset_config.image_extension

Image Extension

string

Extension of the images to be used.

png

png,jpg

yes

__png__, __jpg__, __jpeg__

Can be system generated - after conversion. This is the dataset preparation step.

dataset_config.data_sources.tfrecords_path

TFRecord Path

hidden

/shared/users/1234/datasets/5678/tfrecords/kitti_trainval/*

Where the dataset is - where the images are. Will it figure it out from the parent directory?

dataset_config.data_sources.image_directory_path

Image Path

hidden

/shared/users/1234/datasets/5678/training

Read all labels in the label file (car, truck, suv, person). Ask the user to map it to Vehicle/Person.

dataset_config.target_class_mapping

Target Class Mappings

list

This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For example: car, van, heavy_truck etc may be grouped under automobile.

Class you want to train for (vehicle)

dataset_config.target_class_mapping.key

Class Key

string

The “key” field is the value of the class name in the tfrecords file.

person

^[-a-zA-Z0-9_]{1,40}$

Class defined in the label file (car, truck, suv -> map to vehicle)

dataset_config.target_class_mapping.value

Class Value

string

The “value” field corresponds to the value that the network is expected to learn.

person

^[-a-zA-Z0-9_]{1,40}$

Default - 0

dataset_config.validation_fold

Validation Fold

integer

In case of an n fold tfrecords, you define the index of the fold to use for validation. For sequencewise validation choose the validation fold in the range [0, N-1]. For random split partitioning, force the validation fold index to 0 as the tfrecord is just 2-fold.

0

Dataset specific config - augmentation

augmentation_config

Data Augmentation

collection

Collection of parameters to configure the preprocessing and on the fly data augmentation

Yes

The resolution at which the network should be trained for. Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16.

augmentation_config.preprocessing.output_image_width

Image Width

integer

The width of the augmentation output. This is the same as the width of the network input and must be a multiple of 16.

960

480

yes

Yes

Get the max dimesnion of images in the dataset and set the as the default behind the scenes - has to be multiple of 16

augmentation_config.preprocessing.output_image_height

Image Height

integer

The height of the augmentation output. This is the same as the height of the network input and must be a multiple of 16.

544

272

yes

Yes

Smaller side of image(height or width)

augmentation_config.preprocessing.output_image_min

Image smaller side’s size

integer

The smaller side of image size. This is used for resize and keep aspect ratio in FasterRCNN. If this value is postive, preprocessor will resize the image and keep aspect ratio, such that the smaller side’s size is this value. The other side will scale accordingly by aspect ratio. This value has to be a multiple of 16.

0

Limit of larger side’s size of an image when resize and keep aspect ratio

augmentation_config.preprocessing.output_image_max

Limit of larger side’s size when resize and keep aspect ratio

integer

The maximum size of image’s larger side. If after resize and keeping aspect ratio, the larger side is exceeds this limit, the image will be resized such that the larger side’s size is this value, and hence the smaller side’s size is smaller than output_image_min. This value has to be a multiple of 16.

0

Flag to enable automatic image scaling

augmentation_config.preprocessing.enable_auto_resize

Flag to enable or disable automatic image scaling

bool

If True, automatic image scaling will be enabled. Otherwise, disabled.

TRUE

TRUE, FALSE

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_width

Bounding Box Width

float

The minimum width of the object labels to be considered for training.

1

0

yes

>=0

Limit of what min dimension you DONT want to train for. Default 10x10

augmentation_config.preprocessing.min_bbox_height

Bounding Box Height

float

The minimum height of the object labels to be considered for training.

1

0

yes

>=0

3 channel default

augmentation_config.preprocessing.output_image_channel

Image Channel

integer

The channel depth of the augmentation output. This is the same as the channel depth of the network input. Currently, 1-channel input is not recommended for datasets with JPG images. For PNG images, both 3-channel RGB and 1-channel monochrome images are supported.

3

1, 3

yes

3, 1

0

augmentation_config.preprocessing.crop_right

Crop Right

integer

The right boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_left

Crop Left

integer

The left boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_top

Crop Top

integer

The top boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.crop_bottom

Crop Bottom

integer

The bottom boundary of the crop to be extracted from the original image.

0

0

yes

>=0

0

augmentation_config.preprocessing.scale_height

Scale Height

float

The floating point factor to scale the height of the cropped images.

0

0

yes

>=0

0

augmentation_config.preprocessing.scale_width

Scale Width

float

The floating point factor to scale the width of the cropped images.

0

0

yes

>=0

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.hflip_probability

Horizontal-Flip Probability

float

The probability to flip an input image horizontally.

0.5

0

1

[0, 1)

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.vflip_probability

Vertical-Flip Probability

float

The probability to flip an input image vertically.

0

0

1

[0, 1)

Enable - go to default, disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_min

Minimum Zoom Scale

float

The minimum zoom scale of the input image.

1

0

(0, 1]

Enable - go to default, disable - go to 1. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.zoom_max

Maximum Zoom Scale

float

The maximum zoom scale of the input image.

1

0

[1, 2)

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg which will disable vs enable.

augmentation_config.spatial_augmentation.translate_max_x

X-Axis Maximum Traslation

float

The maximum translation to be added across the x axis.

8

0

>=0

Enable - go to default, disable - go to 0. Check for the right default values with TAO Toolkit Engg.

augmentation_config.spatial_augmentation.translate_max_y

Y-Axis Maximum Translation

float

The maximum translation to be added across the y axis.

8

0

>=0

Enable go tyo default, disable - 0

augmentation_config.spatial_augmentation.rotate_rad_max

Image Rotation

float

The angle of rotation to be applied to the images and the training labels. The range is defined between [-rotate_rad_max, rotate_rad_max].

0.69

0

>=0

augmentation_config.spatial_augmentation.rotate_probability

Image Rotation

float

The probability of image rotation. The range is [0, 1]

[0, 1)

augmentation_config.color_augmentation.color_shift_stddev

Color Shift Standard Deviation

float

The standard devidation value for the color shift.

0

0

1

[0, 1)

augmentation_config.color_augmentation.hue_rotation_max

Hue Maximum Rotation

float

The maximum rotation angle for the hue rotation matrix.

25

0

360

[0, 360)

augmentation_config.color_augmentation.saturation_shift_max

Saturation Maximum Shift

float

The maximum shift that changes the saturation. A value of 1.0 means no change in saturation shift.

0.2

0

1

[0, 1)

augmentation_config.color_augmentation.contrast_scale_max

Contrast Maximum Scale

float

The slope of the contrast as rotated around the provided center. A value of 0.0 leaves the contrast unchanged.

0.1

0

1

[0, 1)

augmentation_config.color_augmentation.contrast_center

Contrast Center

float

The center around which the contrast is rotated. Ideally, this is set to half of the maximum pixel value. Since our input images are scaled between 0 and 1.0, you can set this value to 0.5.

0.5

0.5

0.5

Might need different defaults based on task/scenario

model_config

Model

collection

model_config.arch

BackBone Architecture

string

The architecture of the backbone feature extractor to be used for training.

resnet:18

resnet:18

yes

resnet:10’,

‘resnet:18’, ‘resnet:34’, ‘resnet:50’, ‘resnet:101’, ‘vgg16’, ‘vgg:16’, ‘vgg:19’, ‘googlenet’, ‘mobilenet_v1’, ‘mobilenet_v2’,,,,,image_type ‘darknet:19’, ‘darknet:53’, ‘resnet101’, ‘efficientnet:b0’, ‘efficientnet:b1’,

bgr

Confirm correct default values

model_config.freeze_blocks

Freeze Blocks

integer

This parameter defines which blocks may be frozen from the instantiated feature extractor template, and is different for different feature extractor templates.

0

3

depends on arch

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.freeze_bn

Freeze Batch Normalization

bool

A flag to determine whether to freeze the Batch Normalization layers in the model during training.

FALSE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.all_projections

All Projections

bool

For templates with shortcut connections, this parameter defines whether or not all shortcuts should be instantiated with 1x1 projection layers, irrespective of whether there is a change in stride across the input and output.

TRUE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.use_pooling

Use Pooling

bool

Choose between using strided convolutions or MaxPooling while downsampling. When True, MaxPooling is used to downsample; however, for the object-detection network, NVIDIA recommends setting this to False and using strided convolutions.

FALSE

TRUE, FALSE

Default values. Verify with TAO Toolkit. 2 sets of defaults required.

model_config.dropout_rate

Dropout Rate

float

Probability for drop out

0

0

0.1

[0, 1)

model_config.input_image_config

Input Image

collection

Configuration for input images

model_config.input_image_config.size_height_width

collection

model_config.input_image_config.size_height_width.height

integer

544

model_config.input_image_config.size_height_width.width

integer

960

model_config.input_image_config.image_type

Image Type

enum

The type of images, either RGB or GRAYSCALE

__RGB__

__RGB__, __GRAYSCALE__

model_config.input_image_config.size_min

Image smaller side’s size

integer

The size of an image’s smaller side, should be a multiple of 16. This should be consistent with the size in augmentation_config. This is used when resizing images and keeping aspect ratio

>=0

model_config.input_image_config.size_height_width

Image size by height and width

collection

The size of images by specifying height and width.

model_config.input_image_config.size_height_width.height

Image Height

integer

The height of images

>=0

model_config.input_image_config.size_height_width.width

Image Width

integer

The width of images

>=0

model_config.input_image_config.image_channel_order

Image Channel Order

string

The channel order of images. Should be either “rgb” or “bgr” for RGB images and “l” for GRAYSCALE images

bgr

rgb’, ‘bgr’, ‘l’

model_config.input_image_config.image_channel_mean

Image Channel Means

list

A dict from ‘r’, ‘g’, ‘b’ or ‘l’(for GRAYSCALE images) to per-channel mean values.

[{“key”:”r”,”value”:103.0}, {“key”:”g”,”value”:103.0}, {“key”:”b”,”value”:103.0}]

model_config.input_image_config.image_channel_mean.key

channel means key

string

string => one of r,g,b

r’, ‘g’, ‘b’, ‘l’

model_config.input_image_config.image_channel_mean.value

channel means value

float

value in float

(0, 255)

model_config.input_image_config.image_scaling_factor

Image Scaling Factor

float

A scalar to normalize the images after mean subtraction.

1

>0

model_config.input_image_config.max_objects_num_per_image

Max Objects Num

integer

The maximum number of objects in an image. This is used for padding in data loader as different images can have different number of objects in its labels.

100

>=1

model_config.anchor_box_config

Anchor Boxes

Collection

model_config.anchor_box_config.scale

Anchor Scales

list

The list of anchor sizes(scales).

[64.0,128.0,256.0]

>0

model_config.anchor_box_config.ratio

Anchor Ratios

list

The list of anchor aspect ratios.

[1.0,0.5,2.0]

>0

model_config.roi_mini_batch

ROI Batch Size

integer

The batch size of ROIs for training the RCNN in the model

16

>0

model_config.rpn_stride

RPN stride

integer

The stride of RPN feature map, compared to input resolutions. Currently only 16 is supported.

16

16

model_config.drop_connect_rate

Drop Connect Rate

float

The rate of DropConnect. This is only useful for EfficientNet backbones.

(0, 1)

model_config.rpn_cls_activation_type

RPN Classification Activation Type

string

Type of RPN classification head’s activation function. Currently only “sigmoid” is supported.

sigmoid

model_config.use_bias

Use Bias

bool

Whether or not to use bias for convolutional layers

TRUE, FALSE

model_config.roi_pooling_config

ROI Pooling

collection

Confiuration fo ROI Pooling layer

model_config.roi_pooling_config.pool_size

Pool Size

integer

Pool size of the ROI Pooling operation.

7

>0

model_config.roi_pooling_config.pool_size_2x

Pool Size Doubled

bool

Whether or not to double the pool size and apply a 2x downsampling after ROI Pooling

FALSE

TRUE, FALSE

model_config.activation

Activation

collection

Activation function for the model backbone. This is only useful for EfficientNet backbones.

model_config.activation.activation_type

Activation Type

string

Type of the activation function of backbone.

relu, swish

model_config.activation.activation_parameters

Activation Parameters

dict

A dict the maps name of a parameter to its value.

training_config

Training

collection

>0

IMPORTANT. Open to user - default should smarty calculate. Check factors that influence.

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

8

1

yes

>0

Default - what is the optimal number of epcohs for each model. Smart feature in TAO Toolkit to auto stop once model converges

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

120

1

yes

Yes

TRUE, FALSE

Toggle for end user

training_config.enable_qat

Enable Quantization Aware Training

bool

bool

FALSE

yes

Yes

>0

Default

training_config.learning_rate.soft_start .base_lr

Minimum Learning Rate

float

5.00E-06

Yes

>0

Default

training_config.learning_rate.soft_start .start_lr

Maximum Learning Rate

float

5.00E-04

Yes

(0, 1)

Default

training_config.learning_rate.soft_start .soft_start

Soft Start

float

0.100000001

0

1

Yes

>1

Default

training_config.learning_rate.soft_start .annealing_divider

Annealing

float

0.699999988

0

1

Yes

__NO_REG__, __L1__, __L2__

Default

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L1__

__NO_REG__, __L1__, __L2__

yes

>0

Default

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

3.00E-09

yes

(0, 1)

Default

training_config.optimizer.adam.epsilon

Optimizer Adam Epsilon

float

A very small number to prevent any division by zero in the implementation.

1.00E-08

yes

(0, 1)

Default

training_config.optimizer.adam.beta_1

Optimizer Adam Beta1

float

0.899999976

yes

(0, 1)

Default

training_config.optimizer.adam.beta_2

Optimizer Adam Beta2

float

0.999000013

yes

>=1

Use default as 10. Provide last checpoint to user

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

10

0

yes

TRUE, FALSE

training_config.enable_augmentation

Enable Augmentation

bool

Whether or not to enable data augmentation

TRUE

training_config.retrain_pruned_model

Pruned Model

hidden

The path of pruned model to be retrained

training_config.pretrained_weights

Pretrained Weights

hidden

The path of the pretrained model(weights) used to initialize the model being trained

training_config.resume_from_model

Resume Model

hidden

The path of the model used to resume a interrupted training

(0, 1)

training_config.rpn_min_overlap

RPN Min Overlap

float

The lower IoU threshold used to match anchor boxes to groundtruth boxes.

0.1

(0, 1)

training_config.rpn_max_overlap

RPN Max Overlap

float

The higher IoU threshold used to match anchor boxes to groundtruth boxes.

1

[0, 1)

training_config.classifier_min_overlap

Classifier Min Overlap

float

The lower IoU threshold used to generate the proposal target.

0.1

(0, 1)

training_config.classifier_max_overlap

Classifier Max Overlap

float

The higher IoU threshold used to generate the proposal target.

1

TRUE, FALSE

training_config.gt_as_roi

Gt As ROI

bool

A flag to include groundtruth boxes in the positive ROIs for training the RCNN

>0

training_config.std_scaling

RPN Regression Loss Scaling

float

A scaling factor (multiplier) for RPN regression loss

1

training_config.classifier_regr_std

RCNN Regression Loss Scaling

list

Scaling factors (denominators) for the RCNN regression loss. A map from ¡®x¡¯, ¡®y¡¯, ¡®w¡¯, ¡®h¡¯ to its corresponding scaling factor, respectively

[{“key”:”x”,”value”:10.0},{“key”:”y”,”value”:10.0},{“key”:”w”,”value”:5.0},{“key”:”h”,”value”:5.0}]

training_config.classifier_regr_std.key

RCNN Regression Loss Scaling Key

string

one of x,y,h,w

>0

training_config.classifier_regr_std.value

RCNN Regression Loss Scaling Value

float

float value for key

training_config.output_model

Output Model Path

hidden

Path of the output model

>0

training_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer

12000

>=1

training_config.rpn_mini_batch

RPN Mini Batch

integer

The batch size to train RPN

16

>0

training_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

2000

(0, 1)

training_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

>0

training_config.lambda_rpn_regr

RPN Regression Loss Weighting

float

Weighting factor for RPN regression loss

1

>0

training_config.lambda_rpn_class

RPN classification Loss Weighting

float

Weighting factor for RPN classification loss.

1

>0

training_config.lambda_cls_regr

RCNN Regression Loss Weighting

float

Weighting factor for RCNN regression loss

1

>0

training_config.lambda_cls_class

RCNN Classification Loss Weighting

float

Weighting factor for RCNN classification loss

1

list of floats

training_config.model_parallelism

Model Parallelism

list of floats

List of fractions for model parallelism

training_config.early_stopping

Early Stopping

collection

“loss”

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

>=0

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

>0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

training_config.visualizer

Visualizer

collection

TRUE, False

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

>=1

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

evaluation_config

Evaluation

collection

yes

evaluation_config.model

Model Path

string

The path to the model to run inference

>=1

evaluation_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during evaluation

6000

(0, 1)

evaluation_config.rpn_nms_overlap_threshold

RPN overlap threshold

float

0.7

>0

evaluation_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

>0

evaluation_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

(0, 1)

evaluation_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

(0, 1)

evaluation_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

TRUE, FALSE

evaluation_config.use_voc07_11point_metric

Use VOC 11-point Metric

bool

Whether to use PASCAL-VOC 11-point metric

>=1

evaluation_config.validation_period_during_training

Validation Period

integer

The period(number of epochs) to run validation during training

>=1

evaluation_config.batch_size

Batch Size

integer

The batch size for evaluation

(0, 1)

evaluation_config.trt_evaluation

TensorRT Evaluation

Collection

TensorRT evaluation

evaluation_config.trt_evaluation.trt_engine

Trt Engine

String

TRT Engine

(0, 1)

evaluation_config.gt_matching_iou_threshold

Gt Matching IoU Threshold

float

The IoU threshold to match groundtruth to detected objects. Only one of this collection or gt_matching_iou_threshold_range

0.5

(0, 1)

evaluation_config.gt_matching_iou_threshold_range

Gt Matching IoU Threshold Range

collection

Only one of this collection or gt_matching_iou_threshold

(0, 1)

evaluation_config.gt_matching_iou_threshold_range.start

Start

float

The starting value of the IoU range

TRUE, FALSE

evaluation_config.gt_matching_iou_threshold_range.end

End

float

The end point of the IoU range(exclusive)

evaluation_config.gt_matching_iou_threshold_range.step

Step

float

The step size of the IoU range

evaluation_config.visualize_pr_curve

Visualize PR Curve

bool

Visualize precision-recall curve or not

inference_config

>=1

inference_config.images_dir

Images Directory

hidden

Path to the directory of images to run inference on

>0

inference_config.model

Model Path

hidden

Path to the model to run inference on

>0

inference_config.batch_size

Batch Size

integer

The batch size for inference

(0, 1)

inference_config.rpn_pre_nms_top_N

RPN Pre-NMS Top N

integer

The number of boxes (ROIs) to be retained before the NMS in Proposal layer during inference

6000

(0, 1)

inference_config.rpn_nms_max_boxes

RPN NMS Max Boxes

integer

The maximum number of boxes (ROIs) to be retained after the NMS in Proposal layer

300

(0, 1)

inference_config.rpn_nms_overlap_threshold

RPN NMS IoU Threshold

float

The IoU threshold for NMS in Proposal layer

0.7

>0

inference_config.bbox_visualize_threshold

Visualization Threshold

float

The confidence threshold for visualizing the bounding boxes

0.6

(0, 1)

inference_config.object_confidence_thres

Object Confidence Threshold

float

The objects confidence threshold

0.00001

inference_config.classifier_nms_max_boxes

Classifier NMS Max Boxes

integer

The maxinum numbere of boxes for RCNN NMS

100

True, False

inference_config.classifier_nms_overlap_threshold

Classifier NMS Overlap Threshold

float

The NMS overlap threshold in RCNN

0.3

inference_config.detection_image_output_dir

Image Output Directory

string

Path to the directory to save the output images during inference

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

inference_config.bbox_caption_on

Bbox Caption

bool

Enable text caption for bounding box or not

inference_config.labels_dump_dir

Labels Ouptut Directory

hidden

Path to the directory to save the output labels

inference_config.nms_score_bits

NMS Score Bits

integer

Number of score bits in optimized NMS

inference_config.trt_inference

TensorRT Inference

Collection

TensorRT inference configurations

inference_config.trt_inference.trt_engine

TensorRT Engine

hidden

Path to the TensorRT engine to run inference

semantic_segmentation

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

popular

regex

num_files

num_files

integer

Number of images to convert from COCO json to VOC.

results_dir

results_dir

string

Where it will be stored inside the root

masks

instance_segmentation

convert

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

popular

regex

notes

coco_config

collection

coco_config.root_directory_path

hidden

coco_config.image_dir

string

List of image directories correspoding to each partition

images

The order of image directories must match annotation_files based on partitions

coco_config.annotations_file

string

List of JSON files with COCO dataset format

annotations.json

coco_config.num_shards

integer

The number of shards per fold. If the size of num_shards is 1, then same number of shards will be applied to every partition

coco_config.tag

string

sample_modifier_config

collection

sample_modifier_config.filter_samples_containing_only

list

list of string

sample_modifier_config.dominant_target_classes

list

list of string

sample_modifier_config.minimum_target_class_imbalance

list

list of string

sample_modifier_config.minimum_target_class_imbalance.key

string

sample_modifier_config.minimum_target_class_imbalance.value

float

sample_modifier_config.num_duplicates

integer

sample_modifier_config.max_training_samples

integer

sample_modifier_config.source_to_target_class_mapping

list

list of string

sample_modifier_config.source_to_target_class_mapping.key

string

sample_modifier_config.source_to_target_class_mapping.value

string

image_directory_path

hidden

target_class_mapping

list

list of string

target_class_mapping.key

Class Key

string

target_class_mapping.value

Class Value

string

lprnet

evaluate

parameter

display_name

value_type

description

default_value

examples

valid_min

valid_max

valid_options

required

regex

popular

CLI

version

Schema Version

const

The version of this schema

1

random_seed

Random Seed

integer

Seed value for the random number generator in the network

42

dataset_config

Dataset

collection

Parameters to configure the dataset

dataset_config.data_sources.label_directory_path

Label Path

hidden

dataset_config.data_sources.image_directory_path

Image Path

hidden

dataset_config.validation_data_sources.label_directory_path

Label Path

hidden

dataset_config.validation_data_sources.image_directory_path

Image Path

hidden

dataset_config.characters_list_file

Characters List Path

string

training_config

Training

collection

training_config.batch_size_per_gpu

Batch Size Per GPU

integer

The number of images per batch per GPU.

32

1

training_config.num_epochs

Number of Epochs

integer

The total number of epochs to run the experiment.

24

1

training_config.learning_rate.soft_start_annealing_schedule.min_learning_rate

Minimum Learning Rate

float

The minimum learning rate in the learning rate schedule.

1.00E-06

0

training_config.learning_rate.soft_start_annealing_schedule.max_learning_rate

Maximum Learning Rate

float

The maximum learning rate in the learning rate schedule.

1.00E-05

0

training_config.learning_rate.soft_start_annealing_schedule.soft_start

Soft Start

float

The time to ramp up the learning rate from minimum learning rate to maximum learning rate.

0.001

0

1

training_config.learning_rate.soft_start_annealing_schedule.annealing

Annealing

float

The time to cool down the learning rate from maximum learning rate to minimum learning rate. Greater than soft_start.

0.5

0

1

training_config.regularizer.type

Regularizer Type

string

The type of the regularizer being used.

__L2__

__L1__, __L2__

training_config.regularizer.weight

Regularizer Weight

float

The floating point weight of the regularizer.

5.00E-04

0

training_config.checkpoint_interval

Checkpoint Interval

integer

The interval (in epochs) at which train saves intermediate models.

1

1

training_config.max_queue_size

Max Queue Size

integer

Maximum Queue Size in Sequence Dataset

16

1

training_config.n_workers

Workers

integer

Number of workers in sequence dataset

8

1

training_config.use_multiprocessing

Use Multiprocessing

bool

Use multiprocessing or not

training_config.early_stopping

Early Stopping

collection

training_config.early_stopping.monitor

Monitor

string

The name of the quantity to be monitored for early stopping

loss

training_config.early_stopping.min_delta

Min Delta

float

Minimum delta of the quantity to be regarded as changed

0

training_config.early_stopping.patience

Patience

integer

The number of epochs to be waited for before stopping the training

0

training_config.visualizer

Visualizer

collection

training_config.visualizer.enabled

Enable

bool

Enable the visualizer or not

training_config.visualizer.num_images

Max Num Images

integer

Maximum number of images to be displayed in TensorBoard

3

0

eval_config

Evaluation

collection

eval_config.validation_period_during_training

Validation Period During Training

integer

The interval at which evaluation is run during training. The evaluation is run at this interval starting from the value of the first validation epoch parameter as specified below.

5

1

eval_config.batch_size

Batch Size

integer

batch size for evaluation

1

1

augmentation_config

Augmentation config

collection

augmentation_config.output_width

Model Input width

integer

96

1

yes

augmentation_config.output_height

Model Input height

integer

48

1

yes

augmentation_config.output_channel

Model Input channel

integer

3

1

1,3

yes

augmentation_config.max_rotate_degree

Max Rotation Degree

integer

The maximum rotation angle for augmentation

5

1

augmentation_config.keep_original_prob

Keep Original Probability

float

The probability for keeping original images. Only resized will be applied to am image with this probability

0.3

0

1

augmentation_config.rotate_prob

Rotation Probability

float

The probability for rotating the image

0.5

0

1

augmentation_config.gaussian_kernel_size

Gaussian Kernel Size

list

The kernel size of the Gaussian blur

[5,7,15]

1

augmentation_config.blur_prob

Gaussian Blur Probability

float

The probability for blurring the image with Gaussian blur

0.5

0

1

augmentation_config.reverse_color_prob

Reverse Color Probability

float

The probability for reversing the color of the image

0.5

0

1

lpr_config.hidden_units

Hidden Units

integer

The number of hidden units in the LSTM layers of LPRNet

512

1

lpr_config.max_label_length

Max Label Length

integer

The maximum length of license plates in the dataset

8

lpr_config.arch

Architecture

string

The architecture of LPRNet

baseline

baseline

lpr_config.nlayers

Number of Layers

integer

The number of convolution layers in LPRNet

18

10, 18

export