Is this page helpful?

Overview of NVIDIA NIM for Nemotron Parse#

NVIDIA NeMo™ Retriever NIM APIs provide easy access to state-of-the-art models designed to be core building blocks for enterprise semantic search applications – delivering accuracy, scalability, and real-time information retrieval. These NIM microservices enable developers to build powerful AI-driven extraction and retrieval pipelines that parse, process, and connect multimodal data to generative applications. Built on the NVIDIA software platform, NeMo Retriever NIM microservices use NVIDIA® CUDA® for out-of-the-box GPU acceleration and high-performance inference.

NeMo Retriever includes NIM microservices for creating advanced large-scale, multimodal extraction and retrieval pipelines, which is critical for generative AI applications like retrieval-augmented generation (RAG).

Extraction pipelines retrieve documents from external sources beyond the foundational model’s scope. NVIDIA NIM for Nemotron Parse is a document parsing microservice. You send it one rendered document page as an image, and it returns extracted page content with Markdown formatting and coordinate/class control tokens. The NIM owns the prompt, parsing mode, and generation settings, so you don’t need to supply prompts, special tokens, or sampling parameters.

Use Nemotron Parse NIM when you want a single model to extract text and layout annotations from a page. As an alternative, you can orchestrate the Object Detection NIM and the Image OCR NIM (NeMo Retriever OCR) to extract content from tables, charts, and infographics. With these microservices, downstream retrieval-augmented generation applications are able to retrieve across text as well as other modalities.

The retrieval pipeline fetches relevant document data and generates responses during inference. The following NeMo Retriever microservices provide superior natural language processing and understanding, boosting retrieval performance.

  • Embedding NIM - Boosts question-answering retrieval performance, providing high quality embeddings for many downstream NLP tasks.

  • Reranking NIM - Enhances the retrieval performance further with a fine-tuned reranking model, finding the most relevant passages to provide as context when querying an LLM.

The Nemotron Parse NIM release 2.4 includes the following:

  • Content and layout together: Extract document-page text with semantic layout classes, bounding boxes, and reading order, including tables and chart-aware output. The NIM returns model predictions for downstream parsing and indexing.

  • Multilingual document understanding: Bring Nemotron 3.5 Parse to English and non-English document workflows through one NIM service, rather than maintaining separate parsing paths solely for different scripts.

  • Optimized serving stack: An inference service with multi-token prediction enabled by default and documented BF16 deployment across GB200, RTX PRO 6000 Blackwell Server Edition, H100 NVL, A100, L40S, and A10G.

  • Deployment that fits existing pipelines: Run with Docker or Helm, keep model files in /model/parse, and reuse pre-populated weights for offline startup without provider keys. Follow the documented cache and permissions settings.

For details, refer to Release Notes for NVIDIA NIM for Nemotron Parse.

The following diagram shows how NeMo Retriever NIM microservices are used to create advanced extraction and retrieval pipelines for a question-answering RAG application in enterprise settings.

NeMo Retriever pipeline showing document extraction, embedding, and indexing, followed by retrieval and reranking to provide context for question answering.

Migrate from the VLM NIM#

Before version 2.4, the Nemotron Parse model was served by the NVIDIA NIM for Vision Language Models (VLM NIM) through an OpenAI-compatible chat-completions API. Starting with version 2.4, Nemotron Parse is a standalone NIM with a document-oriented API. This is a breaking change. If you used Nemotron Parse through the VLM NIM, update your client code as follows:

  • Send requests to POST /v1/parse as multipart/form-data instead of POST /v1/chat/completions.

  • Upload the page image as the document file part instead of a base64 data URL in a chat message.

  • Remove prompts and special tokens such as <predict_bbox>. The NIM always performs full-page extraction.

  • Remove sampling parameters such as temperature, top_k, and repetition_penalty. The only per-request setting is options.decoding.max_output_tokens.

  • Read the extracted content from the content field of the response instead of choices[0].message.content.

  • Coordinate/class control tokens are expected in the content response. Parsing-mode selection, including detection-only requests, is not supported.

For details, refer to API Usage Examples (OpenAPI) for NVIDIA NIM for Nemotron Parse.