Advanced Topics

This section covers advanced TensorRT features and configuration options. For Version Compatibility and hardware compatibility, use the top-level Inference Library entry. That entry was promoted from this hub in 11.2.1.

In this guide

• Refitting an Engine: update engine weights without a full rebuild.

• Precision Control: algorithm selection, strongly typed networks, and computational precision.

• Data Formats and Tensors: I/O formats, structured sparsity, empty tensors, and input buffer reuse.

• Data Format Descriptions: supported data types and layout formats.

• Engine Tools and Debugging: Engine Inspector, optimizer callbacks, preview features, and debug tensors.

• Weight Streaming: offload weights to host memory for larger models or batches.

• Multi-Device Inference: scale inference across multiple GPUs with NCCL collectives.