Is this page helpful?

Troubleshoot NVIDIA NIM for Object Detection#

Use this documentation to troubleshoot issues that arise when you use NVIDIA NIM for Object Detection.

NIM Fails to Start#

If the NIM fails to start when you run the docker run command, verify the following:

  • You accepted the license terms on NGC for the container and model assets.

  • You authenticated to nvcr.io before pulling the container image.

  • If you use NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=auto and the model artifacts are not already present at NIM_ENGINE_MODEL_PATH, you passed HF_TOKEN for Hugging Face or NGC_API_KEY for NGC into the container.

  • If you explicitly set NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=hf, you passed HF_TOKEN.

  • If you explicitly set NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc, you passed NGC_API_KEY.

With the default auto setting, Object Detection NIM selects Hugging Face when HF_TOKEN is available. If HF_TOKEN is not set and NGC_API_KEY is available, it selects NGC. When both credentials are available, Hugging Face takes precedence.

For details, refer to Get Started.

Endpoint Returns 404#

The /v1/page-elements and /v1/table-structure routes are part of the API. If the requested endpoint’s model is not loaded, the route returns a 404 response that identifies the unloaded model and lists the available models.

Use /v1/models to confirm the model loaded by the running container.

curl -sS "http://localhost:8000/v1/models"

Invalid Image Request#

The inference API accepts JPEG and PNG images encoded as data URLs. The following inputs are rejected:

  • External HTTP image URLs in the request body.

  • Plain base64 strings without a data:image/...;base64, prefix.

  • GIF or other unsupported image formats.

  • Data URLs whose MIME type does not match the image bytes.

For performance-sensitive workloads, prefer JPEG input to enable GPU-accelerated batched image decoding. PNG input remains supported through the CPU image decode path. For details, see Optimize Image Decoding.

For details, refer to API Reference for NVIDIA NIM for Object Detection.

Requests Timeout During Cold Start#

The first run on a cold runtime cache can spend extra time compiling runtime artifacts. If requests time out during this period, wait for readiness to return ready: true and retry.

For cold-cache environments, you can also increase NIM_SERVER_REQUEST_TIMEOUT_S or precompile runtime artifacts by running the container with NIM_ENGINE_PRECOMPILE_ONLY=1.

Warmup or CUDA Graph Startup Errors#

Use the current warmup and CUDA graph variables:

  • NIM_ENGINE_WARMUP_MODE=full, partial, or none

  • NIM_ENGINE_CUDA_GRAPH_CAPTURE=full, warmup, or none

  • NIM_ENGINE_WARMUP_SHAPES=<comma-separated-batch-sizes>

  • NIM_ENGINE_CUDA_GRAPH_CAPTURE_SHAPES=<comma-separated-batch-sizes>

Values such as EXPLICIT are not supported. If you need explicit batch sizes, use the *_SHAPES variables. Do not set NIM_ENGINE_WARMUP_SHAPES or NIM_ENGINE_CUDA_GRAPH_CAPTURE_SHAPES together with NIM_ENGINE_WARMUP_MODE=none.

High GPU Memory Usage#

NIM_PIPELINE_MAX_BATCH_SIZE controls the preallocated GPU tensor buffer size. Reducing warmup scope or CUDA graph capture scope does not reduce the tensor buffer allocation. If the NIM runs out of GPU memory at startup, reduce NIM_PIPELINE_MAX_BATCH_SIZE or NIM_ENGINE_COUNT.