Struct NetworkOptions

Parameters for Engine creation.

Public Members

bool use_fp16 = true

Use FP16 in engine generation.

std::vector<std::vector<int32_t>> batch_sizes = {{1, 1, 1}}

Batch sizes supported.

int32_t max_batch_size = 256

Max batch size allowed.

size_t max_memory = 10000000000

Maximum GPU memory allocated for model conversion.

int device_index = 0

GPU device.

int32_t dla_core = -1

The DLA core index to execute the engine on, starts at 0. Set to -1 (the default) to disable DLA.

bool dla_gpu_fallback = true

If DLA is enabled, use the GPU if a layer cannot be executed on DLA. If the fallback is disabled, engine creation will fail if a layer cannot executed on DLA.