Is this page helpful?

Get Started With NVIDIA NIM for Object Detection#

This documentation helps you get started with NVIDIA NIM for Object Detection.

Prerequisites#

Before you can get started, you need the following:

Verify that you have supported hardware and software. For details, refer to the support matrix.
If you are running on an RTX AI PC or Workstation, install WSL2. For instructions, refer to NIM on WSL2 documentation.
Create an account on NVIDIA NGC and generate an API key to access the NIM container images and model assets on NGC. For instructions, refer to the NGC Authentication section that follows.

Note

Deploying on Kubernetes is not supported for WSL.

NGC Authentication#

Generate your API key#

To access the NIM container images and model assets on NGC, you must generate a personal API key. To create your key, go to https://org.ngc.nvidia.com/setup/api-keys.

When you create your key, for Services Included, select the following:

NGC Catalog
Private Registry (if you are an Early Access participant)

You can include more services if you are going to use this key for other purposes. For more information, refer to the NGC User Guide.

Export the API key#

To conveniently use your API key in the commands in the following sections, you can export your key as an environment variable named NGC_API_KEY. For example, run the following code in your terminal.

export NGC_API_KEY=<your API key value>

Run one of the following commands to make your key available when you start a new terminal session.

# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc

Note

Other, more secure options include saving your API key value in a file (retrieve it by using cat $NGC_API_KEY_FILE), or saving your key in a password manager.

Docker Login to NGC#

Before you can pull the NIM container image from NGC, first authenticate to NGC by using the following command.

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

Accept the License Terms#

Some NIM models require that you accept the license terms on NGC before you can pull the container image and model assets. To accept the license terms, browse to the model or container page on the NGC Catalog, read and then click Accept Terms.

Model Download Authentication#

The default NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=auto setting selects Hugging Face when HF_TOKEN is available. If HF_TOKEN is not set and NGC_API_KEY is available, it selects NGC. When both credentials are available, Hugging Face takes precedence.

The following examples pass HF_TOKEN and therefore use Hugging Face. To explicitly download model artifacts from NGC instead, set NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc and pass NGC_API_KEY instead of HF_TOKEN.

Launch the NIM#

The Object Detection NIM supports multiple models. By default, the NIM container loads nvidia/nemotron-page-elements-v3 from /model/page-elements. The following example sets these defaults explicitly and launches the container on GPU 0.

# Choose a container name for bookkeeping.
export CONTAINER_NAME=object-detection-nim

# Choose the NIM image from NGC.
export IMG_NAME="nvcr.io/nim/nvidia/nemotron-object-detection:2.0"

# Configure the model served by this container.
export NIM_ENGINE_MODEL_NAME="nvidia/nemotron-page-elements-v3"
export NIM_ENGINE_MODEL_PATH="/model/page-elements"

# Choose paths on your system to cache downloaded models and runtime artifacts.
export LOCAL_NIM_CACHE=~/.cache/nim/object-detection
mkdir -p "$LOCAL_NIM_CACHE/cache" "$LOCAL_NIM_CACHE/weights"

docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=16GB \
  -e HF_TOKEN \
  -e NIM_ENGINE_MODEL_NAME \
  -e NIM_ENGINE_MODEL_PATH \
  -v "$LOCAL_NIM_CACHE/cache:/opt/cache" \
  -v "$LOCAL_NIM_CACHE/weights:/model" \
  -u $(id -u) \
  -p 8000:8000 \
  $IMG_NAME

To use the direct NGC provider, replace -e HF_TOKEN with -e NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc -e NGC_API_KEY in the docker run command.

Flags	Description
`--runtime=nvidia`	Ensures NVIDIA drivers are accessible in the container.
`--gpus '"device=0"'`	Exposes GPU `0` inside the container. To use a different supported GPU, replace `0` with the device ID from `nvidia-smi -L`.
`--shm-size=16GB`	Allocates shared memory for image preprocessing and runtime work.
`-e HF_TOKEN`	Provides the token for Hugging Face and causes the default `auto` provider setting to select Hugging Face.
`-e NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc`	Explicitly selects direct model download from NGC.
`-e NGC_API_KEY`	Provides the API key required by the direct NGC provider.
`-e NIM_ENGINE_MODEL_NAME`	Selects the object detection model to load.
`-e NIM_ENGINE_MODEL_PATH`	Sets the in-container model artifact path.
`-v "$LOCAL_NIM_CACHE/cache:/opt/cache"`	Mounts the runtime artifact cache.
`-v "$LOCAL_NIM_CACHE/weights:/model"`	Mounts the model weights cache.
`-p 8000:8000`	Publishes the NIM HTTP port.

Switch Between Page Elements and Table Structure#

Model selection occurs when the container starts. To switch the model, stop and recreate the container with the settings for the other model. You do not need to clear the model or runtime caches.

Stop the running container. If you used the container name from the launch example, run the following command from another terminal.
```
docker stop object-detection-nim
```

To switch from page elements to table structure, set the following variables in the terminal that you use to launch the NIM.

export NIM_ENGINE_MODEL_NAME="nvidia/nemotron-table-structure-v1"
export NIM_ENGINE_MODEL_PATH="/model/table-structure"

Run the docker run command in Launch the NIM again. Wait for the readiness endpoint to return "ready": true, and then verify the loaded model.
```
curl -sS "http://localhost:8000/v1/health/ready"
curl -sS "http://localhost:8000/v1/models"
```
Confirm that data[].id in the /v1/models response contains nvidia/nemotron-table-structure-v1. For inference, send requests to POST /v1/table-structure.

To switch back to page elements, repeat the procedure with the following settings.

export NIM_ENGINE_MODEL_NAME="nvidia/nemotron-page-elements-v3"
export NIM_ENGINE_MODEL_PATH="/model/page-elements"

After the container is ready, confirm that data[].id contains nvidia/nemotron-page-elements-v3. For inference, send requests to POST /v1/page-elements.

Verify that the Service is Ready#

After you launch the NIM, it might take a few seconds for the service to be ready to accept requests. To verify that the service is ready, run the following code.

curl -X GET "http://localhost:8000/v1/health/ready"

If the service is ready, you should see a response similar to the following.

{
  "object": "health.response",
  "message": "ready",
  "ready": true
}

Run Inference#

After the service is ready, use code similar to the following to run page-elements inference. For more information, refer to API Reference for NVIDIA NIM for Object Detection.

Note

This example uses a JPEG sample image to enable GPU-accelerated batched image decoding. PNG input remains supported. For details, see Optimize Image Decoding.

API_ENDPOINT="http://localhost:8000"

# Create a JSON payload with a base64-encoded image.
IMAGE_SOURCE="https://assets.ngc.nvidia.com/products/api-catalog/nemo-retriever/object-detection/page-elements-example-1.jpg"
# IMAGE_SOURCE="path/to/your/image.png"  # Uncomment to use a local file instead.

if [[ "$IMAGE_SOURCE" == http* ]]; then
  BASE64_IMAGE=$(curl -sS "$IMAGE_SOURCE" | base64 -w 0)
else
  BASE64_IMAGE=$(base64 -w 0 "$IMAGE_SOURCE")
fi
MIME_TYPE="image/jpeg"
if [[ "$IMAGE_SOURCE" == *.png ]]; then
  MIME_TYPE="image/png"
fi

JSON_PAYLOAD='{
  "input": [{
    "type": "image_url",
    "url": "data:'${MIME_TYPE}';base64,'${BASE64_IMAGE}'"
  }]
}'

echo "${JSON_PAYLOAD}" | \
  curl -X POST "${API_ENDPOINT}/v1/page-elements" \
  -H "accept: application/json" \
  -H "Content-Type: application/json" \
  -d @-

For table-structure inference, start the container with NIM_ENGINE_MODEL_NAME=nvidia/nemotron-table-structure-v1 and NIM_ENGINE_MODEL_PATH=/model/table-structure, and then send the same request shape to POST /v1/table-structure.

Download NIM Models to Cache#

To prefetch model artifacts without starting the HTTP server, run the container with NIM_ENGINE_MODEL_DOWNLOAD_ONLY=1.

docker run -it --rm --name=$CONTAINER_NAME \
  -e HF_TOKEN \
  -e NIM_ENGINE_MODEL_DOWNLOAD_ONLY=1 \
  -e NIM_ENGINE_MODEL_NAME \
  -e NIM_ENGINE_MODEL_PATH \
  -v "$LOCAL_NIM_CACHE/cache:/opt/cache" \
  -v "$LOCAL_NIM_CACHE/weights:/model" \
  -u $(id -u) \
  $IMG_NAME

To precompile runtime artifacts and exit before the server starts, run on a GPU host and set NIM_ENGINE_PRECOMPILE_ONLY=1.

docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus '"device=0"' \
  -e HF_TOKEN \
  -e NIM_ENGINE_PRECOMPILE_ONLY=1 \
  -e NIM_ENGINE_WARMUP_MODE=full \
  -e NIM_ENGINE_MODEL_NAME \
  -e NIM_ENGINE_MODEL_PATH \
  -v "$LOCAL_NIM_CACHE/cache:/opt/cache" \
  -v "$LOCAL_NIM_CACHE/weights:/model" \
  -u $(id -u) \
  $IMG_NAME

Stop the Container#

To stop the Docker container, run the following code.

docker stop $CONTAINER_NAME

To remove the Docker container, run the following code. If you included the --rm flag when you started the container, you don’t need this step.

docker rm $CONTAINER_NAME

Get Started With NVIDIA NIM for Object Detection#

Prerequisites#

NGC Authentication#

Generate your API key#

Export the API key#

Docker Login to NGC#

Accept the License Terms#

Model Download Authentication#

Launch the NIM#

Switch Between Page Elements and Table Structure#

Verify that the Service is Ready#

Run Inference#

Download NIM Models to Cache#

Stop the Container#

Related Topics#