Is this page helpful?

Troubleshoot NVIDIA NeMo Retriever Embedding NIM#

Use this documentation to troubleshoot issues that arise when you use NVIDIA NeMo Retriever Embedding NIM.

list-model-profiles command is not available#

Starting in version 2.0, NeMo Retriever Embedding NIM automatically selects an optimized inference pipeline at startup. No user action is required to select profiles, and the list-model-profiles command is not available. For details, refer to Automatic Pipeline Selection.

To request a specific precision, use NIM_ENGINE_PRECISION. For details, refer to Precision Override.

NIM fails to start#

If you attempt to start a NIM, in some cases it fails to start when you run the docker run command. Some NIM models require that you accept the license terms on NGC before you can pull the container image and model assets. To resolve this issue, browse to the model page on the NGC Catalog, read and then click Accept Terms. For details, refer to Get Started.

NIM fails to start with out-of-memory error#

If you attempt to start a NIM, in some cases it fails to start with an out-of-memory error.

NIMs pre-allocate memory in accordance with the maximum input size that may be operated on. Additional engine replicas increase GPU memory requirements, and different models require different amounts of GPU memory.

To resolve this issue, use one of the following options:

  • When you run a NIM on a small VRAM card, adjust the NIM_PIPELINE_MAX_BATCH_SIZE and NIM_PIPELINE_MAX_SEQ_LEN environment variables.

  • For Nemotron 3.5 Embed in the release 2.4 nemotron-embed container, set both NIM_ENGINE_DEVICES=0 and NIM_ENGINE_STREAM_COUNT=1 to request exactly one engine.

For this container, NIM_ENGINE_COUNT=1 alone does not suppress stream replication. The deployment sets NIM_ENGINE_STREAM_COUNT=4, so one selected GPU still receives four engine replicas unless you override the stream count. NIM_ENGINE_DEVICES=0 selects the first container-visible GPU and overrides COUNT-based placement. These settings reduce replica memory use but do not guarantee that the model fits. For placement rules and examples, see Engine Placement and Replicas.

Pod is not Ready after you set nim.httpPort in the Helm chart#

In the Helm chart, setting nim.httpPort alone leaves the pod in a not-Ready state. The server ignores NIM_SERVER_PORT and listens on port 8000 unless NIM_SERVER_BIND_ADDR is set.

Workaround: To change the port that clients use, set service.httpPort; the container still listens on port 8000. To change the container’s port, set both envVars.NIM_SERVER_BIND_ADDR=0.0.0.0:<port> and nim.httpPort=<port>. For more information, refer to Deploy NVIDIA NeMo Retriever Embedding NIM on Kubernetes.