Release Notes

Release 2.0.8-variant

This release contains model updates outlined in the following sections.

Nemotron Parse v2.0

This is the initial release of Nemotron Parse v2.0. Nemotron Parse v2.0 is a purpose-built OCR and document parsing model. This NIM is part of the NIM Certified offering. For more information on this model, see the model card.

For GPU support, refer to the support matrix for Nemotron Parse v2.0.

Note the following limitations:

• Nemotron Parse v2.0

  • Input consists of an image and a task prompt (using control tokens such as <predict_bbox><predict_classes><output_markdown>). Text-only requests are not supported.

  • The /v1/completions endpoint is not supported. Use /v1/chat/completions with an image and a task prompt instead.

  • The /v1/responses endpoint’s stateful lifecycle (retrieve, cancel, previous_response_id chaining, and background create) is not supported. Unsupported operations return a structured 4xx error response, consistent with stateless models.

  • The /tokenize endpoint returns an empty token list for text-only input. The model has no text-only pathway; tokenization requires an image and task prompt through /v1/chat/completions.

  • The following capabilities are not supported:

    • Tool calling and function calling

    • Reasoning output and reasoning-control parameters

    • Structured output (including response_format and structured_outputs)

    • Request-level chat_template and chat_template_kwargs overrides

    • General conversational instruction following (including system-prompt personas)

    • Multi-turn conversational memory

    • General-purpose document grounding or semantic question answering

  • Request validation and error semantics are inherited from the upstream vLLM inference engine and are preserved intentionally. As a result, the API does not guarantee strict or consistent parameter validation, and clients should validate requests on their side rather than relying on the server to reject malformed input. Note the following:

    • Malformed or out-of-range request parameters (for example, invalid or corrupt image data, or an out-of-range temperature or top_p value) may be surfaced as an HTTP 500 internal server error rather than a 4xx client error.

    • Some unrecognized or out-of-range parameters (for example, an unknown role, an unexpected user field, or out-of-range top_k, best_of, or n value) are silently accepted and ignored, returning HTTP 200 rather than a validation error.

    • Certain empty or degenerate inputs (for example, an empty text field or an empty structured-output schema) can cause the affected engine worker to fail. The worker does not recover automatically, and the service remains unavailable until the affected worker or service is restarted.

    • For streaming requests, an error detected after the response has begun is delivered as an error event within the Server-Sent Events (SSE) stream rather than as a top-level HTTP status code.

    These behaviors match conventional vLLM and are not specific to this NIM.

  • On A10G and L40S GPUs, the NIM runs with a reduced KV-cache budget (max_model_len=4096) to avoid out-of-memory errors during engine initialization. As a result, the maximum supported output sequence length (OSL) on these GPUs is 1,024 tokens.

Inkling

This is the initial release of Inkling. This NIM is part of the NIM offering. This NIM serves the model with an SGLang backend. For more information on this model, see the model card.

For GPU support, refer to the support matrix for Inkling.

Note the following limitations:

• Inkling

  • The first inference request incurs a cold-start latency of approximately 5–10 minutes while flashinfer JIT-compiles kernels. Subsequent requests are fast.

  • LoRA adapters are not supported.

  • FFmpeg is intentionally removed from the base image for licensing compliance. As a result, torchcodec-dependent processors (for example, mimo_v2 and qwen_omni) fail to load at startup with libtorchcodec errors. These errors are cosmetic and do not affect Inkling.

For information about past updates and older versions, refer to the previous release notes.