Is this page helpful?

Release Notes for NVIDIA NeMo Retriever Reranking NIM#

This documentation contains the release notes for NVIDIA NeMo Retriever Reranking NIM.

Note

Some releases are labelled “Production Branch” or “(PB)”. Production Branches provide reliable, stable versions of the NIM. Non-production branch releases (sometimes called Feature Branch (FB) releases) contain the latest features, improvements, and optimizations.

Release 2.4#

Nemotron Rerank 2.4 brings the Nemotron 3.5 Rerank 8B model into a GPU-accelerated NIM, helping teams refine retrieved results across text and visually rich documents before passing context to enterprise agentic applications. This NIM provides multimodal, multilingual reranking for enterprise documents that consist of text, page images, tables, charts, and infographics.

Highlights#

  • Added the Nemotron 3.5 Rerank 8B multimodal reranking model, nvidia/Nemotron-3.5-Rerank-8B-BF16. The model scores a text query against text passages, image passages, or passages that have both.

  • The nvidia/Nemotron-3.5-Rerank-8B-BF16 model has a BF16 checkpoint and runs in FP16 by default on NVIDIA RTX PRO 6000 Blackwell Server Edition, NVIDIA GB200, NVIDIA B300, NVIDIA GB10 (DGX Spark), NVIDIA H100 80GB HBM3, NVIDIA H100 NVL, NVIDIA A100 SXM4 80GB, NVIDIA L40S, and NVIDIA A10G. For details, refer to Support Matrix for NVIDIA NeMo Retriever Reranking NIM.

  • Starting with release 2.4, the NIM container name and the model name are different.

    • The container is nvcr.io/nim/nvidia/nemotron-rerank.

    • The 2.4 release container serves nvidia/Nemotron-3.5-Rerank-8B-BF16.

    • Use the model ID that /v1/models returns in the model field of API requests. For details, refer to Get Started.

  • The model supports up to 32,768 tokens for each query and passage pair. The NIM limits each pair to 10,240 tokens by default. To allow longer pairs, set NIM_PIPELINE_MAX_SEQ_LEN. For details, refer to Environment Variables for NVIDIA NeMo Retriever Reranking NIM.

  • Added gRPC inference through the KServe V2 protocol for Nemotron 3.5 Rerank 8B. For details, refer to API Usage Examples (gRPC) for NVIDIA NeMo Retriever Reranking NIM.

  • Added performance and memory data for Nemotron 3.5 Rerank 8B on all nine supported GPUs. For details, refer to Performance and Memory for NVIDIA NeMo Retriever Reranking NIM.

For details, refer to Support Matrix.

All Known Issues#

The known issues for NeMo Retriever Reranking NIM are the following:

Release Notes for Previous Versions#