Release 1.3.0
Summary
This release adds support for GB200 GPU with ARM architecture and extends sequence processing capabilities for both PyTorch and TensorRT backends.
Key Features
- GB200 Support: Added support for GB200 GPU with ARM architecture
- Extended Sequence Processing:
- PyTorch backend supports up to 4096 length sequences
- TensorRT backend supports up to 2048 length sequences
- Enhanced compatibility with ARM-based systems
Notes and Limitations
- Ensure you use this NIM with GPUs with at least 48 GB of VRAM.
- CUDA Driver Requirements:
- Minimum version 580 with CUDA 13.0
- TensorRT Engine Speedup: To achieve optimal prediction speedup using TRT engines, this version uses TensorRT 10.11.0. Refer to the TensorRT 10.11.0 Support Matrix to ensure your CUDA Driver and CUDA Toolkit versions are compatible.
Note: While there are many options for tuning this NIM’s performance, for most users, the defaults will provide a balanced performance experience.