Is this page helpful?

Notes on NIM Container Variants#

Some NIMs are built with packages that vary from the standard base Docker container. These NIMs can better access features specific to a particular model or can run on GPUs before they are fully supported in the main source code branch. These NIMs are also known as NIM container variants.

These NIM container variants have important, underlying differences from NIMs built with the standard base container. These differences vary according to model. This page documents these differences with respect to the features and functionality of VLM NIM container version 1.7.0.

Kimi-K2.5#

Environment Variables#

Not Supported#

The following environment variables are not currently supported:

  • NIM_SDK_USE_NATIVE_TLS

  • NIM_PER_REQ_METRICS_ENABLE

  • NIM_GUIDED_DECODING_BACKEND

  • NIM_DISABLE_LOG_REQUESTS

  • SSL_CERT_FILE

Usage Changes and Features#

The container docker run command doesn’t support the -u $(id -u) parameter.

Known Issues#

Input Validation Limitations

This NIM container variant uses a specialized inference backend that may exhibit relaxed input validation for certain API request parameters. In some cases, invalid or out-of-range parameter values may not produce precise error messages or may be silently accepted. Examples of observed behaviors include:

  • Error messages may not accurately reflect the invalid parameter name.

  • Generation parameters exceeding documented limits may be accepted without error.

  • Required fields may accept empty or invalid values without proper validation.

These validation behaviors do not affect inference correctness when valid parameters are provided. Ensure request parameters conform to the documented API specification.

Qwen3.5-397B-A17B#

Environment Variables#

Not Supported#

The following environment variables are not currently supported:

  • NIM_DISABLE_LOG_REQUESTS

  • NIM_GUIDED_DECODING_BACKEND

  • NIM_PER_REQ_METRICS_ENABLE

  • NIM_SDK_USE_NATIVE_TLS

  • SSL_CERT_FILE

Usage Changes and Features#

The container docker run command doesn’t support the -u $(id -u) parameter.

The request parameter best_of is not supported.

Known Issues#

Input Validation Limitations

This NIM container variant uses a specialized inference backend that may exhibit relaxed input validation for certain API request parameters. In some cases, invalid or out-of-range parameter values may not produce precise error messages or may be silently accepted. Examples of observed behaviors include the following:

  • Error messages may not accurately reflect the invalid parameter name.

  • Generation parameters exceeding documented limits may be accepted without error.

  • Required fields may accept empty or invalid values without proper validation.

These validation behaviors do not affect inference correctness when valid parameters are provided. You should ensure that request parameters conform to the documented API specification.

Structured Generation (Guided Decoding)#

This NIM container variant uses a specialized inference backend with a different guided decoding implementation than the standard for NIM. As a result, behavior differs from NIMs that use the default guided decoding backend (that is, vLLM without any additional configuration).

Limitations#

The following guided decoding parameters are not supported in this container variant:

  • guided_regex

  • guided_choice

  • guided_grammar

  • guided_whitespace_pattern

Simple schema (single string field)#

For a single field such as an image answer or category, use a minimal JSON schema. The response content will be valid JSON (for example, {"answer": "cat"}). Parse the JSON and read the field for a normalized value.

curl -X 'POST' \
'http://0.0.0.0:8000/v1/chat/completions' \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
        "model": "qwen/qwen3.5-397b-a17b",
        "messages": [
            {
                "role": "user",
                "content": "What animal is in this image? Reply with one word only."
            }
        ],
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "image_answer",
                "schema": {
                    "type": "object",
                    "properties": {
                        "answer": { "type": "string" }
                    },
                    "required": ["answer"],
                    "additionalProperties": false
                }
            }
        }
    }'

Rating and sentiment (structured fields)#

To get a numeric rating (for example, 1–5) or a fixed set of labels without relying on guided_regex or guided_choice, use a small JSON schema and parse the result. You can validate or normalize rating and sentiment in your application (for example, map “four” to “4” if needed).

curl -X 'POST' \
'http://0.0.0.0:8000/v1/chat/completions' \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
        "model": "qwen/qwen3.5-397b-a17b",
        "messages": [
            {
                "role": "user",
                "content": "Return the rating (1-5) and sentiment (Good/Bad/Neutral) for this review: This movie exceeds expectations. I rate it four stars out of five."
            }
        ],
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "review_result",
                "schema": {
                    "type": "object",
                    "properties": {
                        "rating": { "type": "string", "description": "One of 1, 2, 3, 4, 5" },
                        "sentiment": { "type": "string", "enum": ["Good", "Bad", "Neutral"] }
                    },
                    "required": ["rating", "sentiment"],
                    "additionalProperties": false
                }
            }
        }
    }'