Release 1.5.0
Summary
This release adds support for NVIDIA B300 and GB300 GPUs and introduces a new, fully NVIDIA-optimized inference backend that replaces the previous hybrid implementation.
Key Features
- NVIDIA B300: Added support for NVIDIA B300 GPU
- NVIDIA GB300: Added support for NVIDIA GB300 superchip
- New optimized inference backend: The NIM now runs on a fully NVIDIA-tuned inference path, delivering faster and more consistent predictions out of the box.
Backend Optimizations
The new inference backend introduces custom GPU kernels and graph-level optimizations:
- New kernels: Fused Adaptive Layer Norm; DualGemm (sm80, two variants); gated sigmoid (sm80); FlashAttention v2/v3 for Triangle Attention and Attention Pair Bias.
- Pre-computed masks and buffers: Pre-computed, left-aligned attention masks feed directly into the new dual-GEMM and attention kernels, with pre-computed buffers for the Pairformer and Diffusion Transformer (DiT) stacks.
- Lower peak memory: The confidence module is evaluated in a loop to reduce memory usage.
- Diffusion Transformer: Removed the bias computation for Attention Pair Bias, replacing it with three kernels (one reduce, one GEMM, one padding).
- Optimized query-to-keys path in the attention computation.
Notes and Limitations
- The optional PyTorch-only inference mode from earlier releases is no longer available. All requests run on the new optimized backend.
- All features from release 1.4.0 remain supported.