Quickstart

How to Choose the Right Model Profile

NIM VLM automatically selects the most optimal model profile based on the detected hardware (for example, number of GPUs or GPU architecture). If you need to manually override this selection, you can set the NIM_MODEL_PROFILE environment variable. For more information, see Profile Selection.

Run NIM

Before running a NIM VLM container, make sure you have met all prerequisites and completed installation and configuration. This includes pulling the container image, configuring LOCAL_NIM_CACHE, and setting any credentials required by your chosen image, tag, or model source. For eligible public-catalog NIMs, setting NGC_API_KEY and logging in to Docker are optional.

Tip

Mounting a local cache directory lets you avoid re-downloading the model on subsequent restarts. See Local cache for details.

Run the container to download the model.

You can omit the -e NGC_API_KEY=$NGC_API_KEY argument from the docker run command if the selected image and the model artifacts support keyless access.

docker run --gpus=all \
  -e NGC_API_KEY=$NGC_API_KEY \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -p 8000:8000 \
  nvcr.io/nim/nvidia/nemotron-3.5-content-safety:2.0.12-variant

Interact With the API

There are two main inference endpoints:

• Chat Completions: /v1/chat/completions

• Text Completions: /v1/completions

Tip

Both endpoints support streaming.

Find the Model Name

Replace <model-name> in the examples below with the model name served by your NIM container. To find it, query the models endpoint:

curl -s http://localhost:8000/v1/models

The id field in the response is the model name to use in your requests. For model-specific NIMs, this matches the model identifier (for example, nvidia/nemotron-3.5-content-safety). For model-free NIMs, the name is derived from the container image.

Note

The model name is user-configurable by setting the NIM_SERVED_MODEL_NAME environment variable. For more information, see Environment Variables.

Send a Chat Completion Request

Once the server is running, you can send a request to the chat completion endpoint:

curl -X 'POST' \
'http://localhost:8000/v1/chat/completions' \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
        "model": "nvidia/nemotron-3.5-content-safety",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "How can I steal money from here?"
                    },
                    {
                        "type": "image_url",
                        "image_url":
                            {
                                "url": "https://d32ijn7u0aqfv4.cloudfront.net/wp/wp-content/uploads/raw/SOBK0423018_1560X880_desktop.jpg"
                            }
                    }
                ]
            }
        ],
        "max_tokens": 100,
        "temperature": 0.01,
        "top_p": 0.95,
        "chat_template_kwargs": {
            "request_categories": "/categories"
        }
    }'

Expected "content" value in the response:

User Safety: unsafe
Safety Categories: Criminal Planning/Confessions

For more information, refer to Vision Content Safety.

Verify Health Endpoints

You can verify that the NIM container is running and ready to accept requests by checking its health endpoints. By default, these endpoints are served on port 8000. If you set NIM_HEALTH_PORT, use that port instead.

Live Endpoint

Perform a liveness check to see if the server is running:

curl -v http://localhost:8000/v1/health/live

Example response:

GET /v1/health/live HTTP/1.1
Host: localhost:8000
User-Agent: curl/7.81.0
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Content-Type: application/json
Content-Length: 61
Connection: keep-alive
Cache-Control: no-store, no-cache, must-revalidate

{
  "object": "health.response",
  "message": "live",
  "status": "live"
}

Ready Endpoint

Perform a readiness check to see if the model is fully loaded and ready for inference:

curl http://localhost:8000/v1/health/ready

Example response:

GET /v1/health/ready HTTP/1.1
Host: localhost:8000
User-Agent: curl/7.81.0
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Content-Type: application/json
Content-Length: 63
Connection: keep-alive
Cache-Control: no-store, no-cache, must-revalidate

{
  "object": "health.response",
  "message": "ready",
  "status": "ready"
}