Skip to content

Release Notes for NeMo Retriever Library

This documentation contains the release notes for NeMo Retriever Library.

26.08.1 Release Notes (26.8.1)

NVIDIA® NeMo Retriever Library version 26.08.1 includes a shared text-generation task API, configurable large language model (LLM) settings, grounded answer-generation model paths, agentic retrieval, and updated Helm NIM defaults. It builds on the graph ingest, multimodal extraction, and Helm-first deployment foundation.

To upgrade the Helm charts for this release, refer to the NeMo Retriever Library Helm Charts.

The following sections summarize user-visible changes included in 26.08.1 and foundational capabilities that remain current.

Upgrade notes

  • Nemotron OCR v2 is now the default OCR engine for local Hugging Face and hosted CPU actors. For Helm NIM deployments, Nemotron OCR v2 is the default. The previous release kept Helm on OCR v1. Refer to Default Helm NIMs for the chart image repository and tag.
  • Helm replaces separate page-elements and table-structure NIMs with the combined nemotron-object-detection:2.0.1 image. Development Compose uses the same combined object-detection image and OCR v2, but still defaults to 2.0.0 tags unless you override NIM_*_TAG.
  • Helm default VL embed and VL rerank NIM images bump to 2.3.0. The previous release used 1.12.0 and 1.11.0. Development Compose still defaults to 1.12.0 and 1.11.0 unless you override NIM_EMBED_TAG and NIM_RERANK_TAG.
  • Default VLM image captioning is Nemotron 3 Nano Omni for local and hosted paths. Chart-classified PDF regions remain on the layout and OCR path.
  • Hosted Nemotron Parse and self-hosted Nemotron Parse use distinct HTTP contracts. Select the matching client path for your endpoint.
  • macOS Intel (x86_64) is no longer supported for package installs. Use Apple Silicon (arm64) macOS, Windows x64, or Linux. Refer to Packaging and platform.
  • Legacy nv-ingest and compatibility pipeline CLI code paths are removed. Use retriever ingest and the graph stage registry.
  • Self-hosted Parakeet on Helm requires both nimOperator.audio.enabled=true and serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051. Enabling the audio NIM alone does not wire the service ASR endpoint.
  • Changing a Helm NIM image repository or tag on an existing release cannot patch NIMCache spec.source.ngc.modelPuller. Delete the NIMCache and its PVC, then upgrade. The affected NIM is unavailable while the operator re-caches weights. Refer to Changing a NIM image repository or tag.
  • A document whose VectorDB write is not acknowledged now fails instead of reporting completed with a positive row count. Earlier builds failed only collection-managed writes and logged a legacy fixed-table failure as a warning. The worker acknowledgement timeout is configurable through serviceConfig.vectordb.writeTimeoutSeconds (rendered as vectordb.write_timeout_s) and defaults to 300 seconds. Refer to Ingest fails with a VectorDB write error.
  • Retriever Service OpenAPI info.version no longer reports a stale package-version value. The service reports the package version, and Helm sets RETRIEVER_SERVICE_VERSION from the running service image tag so /openapi.json matches the deployed release.

Text generation and LLM configuration

  • 26.08.1 includes a shared one-request-per-row text-generation abstraction: TextGenerationTask plus TextGenerationOperator. GenericGenerationOperator accepts a validated custom prompt. Refer to One-shot text generation.
  • SummarizeTask inherits from TextGenerationTask. SummarizationOperator provides the built-in summarization behavior with the default prompt or a custom prompt.
  • Configure those operators with TextGenerationParams.from_kwargs(...). Supported fields include model, api_base, api_key, temperature, top_p, max_tokens, extra_params, num_retries, timeout, prompt, system_prompt, rag_system_prompt, rag_system_prompt_prefix, reasoning_enabled, and max_workers. Refer to TextGenerationParams configuration.

Answer generation

  • Retriever.answer() and optional POST /v1/answer remain the grounded answer-generation path. The default LLM is nvidia/llama-3.3-nemotron-super-49b-v1.5 (Helm nimOperator.answer_llm image nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5:2.0.5). The generic slot also accepts another OpenAI-compatible LLM or vision-language model (VLM), including nvidia/nemotron-3-nano-omni-30b-a3b-reasoning.
  • Enabling the Omni caption Helm key does not enable /v1/answer. Use Omni as the answer backend by overriding the generic answer_llm slot or by pointing serviceConfig.llm at an Omni chat-completions endpoint. Refer to Answer generation and Answer generation (operator-managed LLM).

Agentic retrieval

  • Agentic retrieval is available in 26.08.1. An LLM agent issues multiple searches, fuses candidates, and returns a document-level ranking. The CLI, Python query workflow, REST, and MCP surfaces share this path. Refer to Agentic retrieval (concept) and Workflow: Agentic retrieval.
  • retriever query --agentic runs that ReAct loop over the same LanceDB table as one-pass retrieval. Local CLI and harness runs default to in-process vLLM (nemotron-8b). Remote OpenAI-compatible NIM or NVIDIA-hosted endpoints use --agentic-invoke-url.
  • Retriever Service exposes agentic retrieval on POST /v1/query with agentic=true and an agentic_query MCP tool when agentic.enabled is true. Service mode requires a remote OpenAI-compatible LLM endpoint. Agentic remains opt-in through serviceConfig.agentic.enabled.
  • The Helm answer_llm Super-49B NIM auto-wires /v1/answer only. Self-hosted agentic retrieval against that NIM requires --enable-auto-tool-choice --tool-call-parser llama3_json on NIM_PASSTHROUGH_ARGS and explicit serviceConfig.agentic wiring. Refer to Self-hosted Helm Super-49B.
  • Configurable auto-retrieval is available on the service query path, with evidence and coverage output formats on /v1/query. MCP query-method selection and rerank tools are available.

Models, OCR, and NIM artifacts

  • Nemotron OCR v2 is unified across library, hosted, and Helm defaults. Hosted OCR uses its own language behavior. Refer to Default Helm NIMs and OCR NIM configuration for the chart image.
  • Local OCR crop batching runs across page rows for throughput. Helm extraction NIMs (OCR and object detection) enable performance mode by default. The VL embed NIM does not.
  • 26.08.1 Helm default and optional NIM images that affect mirroring, allowlisting, and troubleshooting include the following:
    • Combined object detection for page elements and table structure: nvcr.io/nim/nvidia/nemotron-object-detection:2.0.1
    • Image OCR: nvcr.io/nim/nvidia/nemotron-ocr-v2:2.0.1
    • VL embedding: nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:2.3.0
    • VL reranking (optional): nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:2.3.0
    • Optional Omni caption and configurable answer VLM: nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:2.0.4-variant
    • Optional answer-generation LLM: nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5:2.0.5
  • Optional Nemotron-3-Embed-1B is available in 26.08.1. It is not enabled by default and is not a Helm NIM.
    • Optional NIM: nvcr.io/nim/nvidia/nemotron-3-embed-1b:2.2.2
    • Optional Hugging Face checkpoint: nvidia/Nemotron-3-Embed-1B-BF16 (revision 9e0b24858b1195815ecb1188ffa1b73bcea7b30a)
  • The CLI lists nvidia/Nemotron-3-Embed-1B-BF16 among tested official local checkpoints. For local Hugging Face inference, pass --embed-model-name nvidia/Nemotron-3-Embed-1B-BF16. For a self-hosted or hosted embedding NIM, pass --embed-invoke-url with --embed-model-name. Refer to Dense Nemotron embedding checkpoints for local checkpoint usage. Refer to Route ingest to hosted or self-hosted NIM endpoints and the text-only embedding NIM note in Multimodal embeddings for external endpoints.
  • The page_elements and table_structure services share the combined object-detection image and select distinct models. Pull that image once for air-gapped or allowlisted deployments.
  • Nemotron 3 Nano Omni is the canonical caption model. It is opt-in on Helm and has a larger GPU footprint than Nano caption profiles.
  • Nemotron Parse endpoint wiring is available in service extraction workers, with documented hosted versus self-hosted contract selection.
  • An embedder model router supports additional Llama Nemotron and Nemotron-3 embedding checkpoints, including 1B, 3B, 8B, local, fine-tuned, and ModelOpt.

Pipeline and ingestion

  • retriever ingest and the graph stage registry are the canonical ingestion paths. The compatibility pipeline CLI is retired.
  • Documented Markdown, JSON, and shell text inputs, plus inline text ingestion support.
  • Service-mode TXT and HTML chunking.
  • return_failures is supported across in-process and batch ingest modes.
  • Tabular ingestion and embedding improvements, including table-type handling.
  • PDF render parameters are forwarded through ingestion graphs.

CLI

  • Root CLI adds first-class retriever ingest and retriever query commands with NIM URL flags, batch tuning, and LanceDB overwrite/append controls
  • retriever query --agentic runs an LLM-driven ReAct retrieval loop over the same LanceDB table as one-pass retrieval. Local CLI and NRB benchmark runs default to in-process vLLM (nemotron-8b). Remote OpenAI-compatible NIM or NVIDIA-hosted endpoints use --agentic-invoke-url. Refer to Workflow: Agentic retrieval.
  • retriever ingest and retriever query replace the retired compatibility pipeline command. Other top-level subcommands—including eval, benchmark, and skill-eval—are development and experimental.

Retriever Service and deployment

  • Helm maps serviceConfig.nimEndpoints.rerankInvokeUrl / rerankModelName into nim_endpoints.rerank_invoke_url / rerank_model_name, and auto-wires those fields when nimOperator.rerankqa.enabled=true, so /v1/query with rerank=true works in split topology.
  • Split topology renders an internal gateway startup Service so realtime and batch init containers can reach GET /v1/live before the gateway passes its deep /v1/health readiness check. This removes the clean-install deadlock that required manually patching publishNotReadyAddresses on the gateway Service.
  • Retriever Service exposes agentic retrieval on POST /v1/query when agentic.enabled is true. Refer to Workflow: Agentic retrieval.
  • Gateway worker pull scheduling replaces push routing.
  • Development Docker Compose deployment is available for local service stacks.
  • Zipkin tracing parity is available alongside OpenTelemetry.
  • Helm maximum upload size configuration and OpenShift deployment follow-ups.
  • Secret-backed Helm authentication for public and internal service tokens. Inline tokens are gated for insecure development only.

Multimodal extraction

  • Fixed an issue that could cause local Hugging Face batch audio extraction to hang in interactive terminals when FFmpeg inherited the parent process's standard input.

Retrieval and RAG

  • Retriever.answer() supports the Super-49B default LLM path and the Omni VLM-capable path documented under Answer generation.
  • Service-mode Retriever.answer support and FastMCP integration are available for local and remote agents.
  • Agentic retrieval is available on the CLI, Python query workflow, REST, and MCP surfaces. Refer to Agentic retrieval.

Vector database and retrieval

  • True LanceDB hybrid retrieval.
  • LanceDB retrieval-mode autodetection and persisted embedding identity for automatic local queries.
  • Local queries warn when an explicit embedding model differs from the model recorded on the LanceDB table. The query continues with the explicit override so intentional model overrides remain available.
  • Dense image-only VDB records are retained where applicable.
  • Scope-isolated collection and document lifecycle APIs (/v1/collections) support create, ingest, replace, query, and cleanup without exposing LanceDB table names.
  • Fixed an issue where concurrent ingests into one VectorDB pod could report success minutes before the rows were durable or queryable. Each write now commits its rows independently, and index maintenance runs in a separate serialized phase where concurrent writers share one coalesced rebuild. An index-readiness wait that expires logs a warning and leaves the committed rows queryable instead of failing the write. Refer to LanceDB index creation fails during concurrent Helm ingestion.

Packaging and platform

  • Public nightlies are published to PyPI while local install extras remain stable.
  • Ray is raised to >=2.56.1 for CVE remediation. The previous release used >=2.49.0. Ray no longer publishes wheels for macOS Intel (x86_64), so pip and uv installs fail on Intel Macs, including in-process library mode. Apple Silicon (arm64) macOS remains supported for slim remote or NIM-only installs, alongside Windows x64.

Helm chart

  • The Helm chart under nemo_retriever/helm/ defaults to OCR v2, the combined object-detection NIM, and VL embedder 2.3.0. Optional NIMs include VL rerank 2.3.0, Omni 2.0.4-variant, Nemotron Parse, and the Super-49B answer_llm slot. Nemotron-3-Embed-1B is optional and is not a chart NIM. Refer to Default Helm NIMs and Models, OCR, and NIM artifacts.

Documentation

Current foundational capabilities

The following foundational capabilities remain current. They are not new 26.08.1 highlights.

  • Text splitting for graph and library ingest uses .extract(split_config=...) instead of standalone .split() on the graph ingest path. The service ingestor API can still expose .split() separately.
  • Direct Retriever(...) construction uses vdb_kwargs, embed_kwargs, and rerank instead of flat lancedb_uri, lancedb_table, embedder, embedding_endpoint, local_query_embed_backend, and reranker arguments.
  • For Helm audio and video extraction, set service.installFfmpeg: true in values.yaml (or pass --set service.installFfmpeg=true) when images no longer bundle ffmpeg and ffprobe by default.
  • nemo_retriever requires Python 3.12.
  • Manifest-based ingest routing replaces input-type routing. retriever ingest is input-aware for PDF, image, audio, video, text, HTML, DOCX, PPTX, SVG, and related types.
  • allow_no_gpu skips the GPU requirement during ingest for CPU-only experimentation.
  • Root CLI includes retriever ingest and retriever query with NIM URL flags, batch tuning, and LanceDB overwrite or append controls.
  • Retriever Service v2 provides a scalable multi-pod architecture with gateway, process isolation, and VectorDB integration.
  • OpenTelemetry provides basic pipeline and service observability.
  • Air-gapped deployment guidance is in deployment options and the Helm chart README.
  • Nemotron Parse is an alternate PDF extraction method (v1.2 HTTP interface, optional Helm NIM, and local inference through vLLM where configured).
  • VLM image captioning through vLLM, including Omni caption model profiles, is available.
  • vLLM-backed text and vision-language embedders, a multimodal VL reranker, and torch 2.11 are available for local GPU installs.
  • The video retrieval pipeline includes frame extraction, OCR, audio-visual fusion, and text deduplication.
  • Long-audio Parakeet chunking provides time-aligned segments, punctuation-based audio segmenting, and ASR batch and streaming improvements.
  • The live RAG SDK includes Retriever.retrieve(), reference answer generation through Retriever.answer(), and optional batch operator graphs through LiteLLM ([llm] extra).
  • Vector database operators are integrated in the pipeline, with custom metadata support and updated LanceDB hybrid search guidance.
  • LanceDB is the first-party vector path for new deployments. Milvus and MinIO guidance is removed from the primary extraction doc set.
  • Evaluation includes a BEIR-centric overhaul and the experimental retriever skill-eval benchmark CLI.
  • Text-to-SQL agent graph and tabular tooling support structured data retrieval, including tabular data ingestion.
  • Optional install extras include [local], [multimedia], [llm], [tabular], [nemotron-parse], [service], and slim remote or NIM-only installs on Mac and Windows.
  • Documentation is aligned to a Helm-first supported path and consolidates extraction concepts, ingest workflow, embeddings, audio and video guides, prerequisites and support matrix, and UDF or custom stages in the graph README.

Release Notes for Previous Versions

Release notes for 24.12.1 and 24.12.0 are on the 25.3.0 archived release notes.