Troubleshoot NeMo Retriever Text Reranking NIM#

Use this documentation to troubleshoot issues that arise when you use NeMo Retriever Text Reranking NIM.

list-model-profiles command fails#

Some older NIMs don’t support the list-model-profiles command, including the following:

  • nv-rerankqa-mistral-4b-v3

  • nv-embedqa-mistral-7b-v2

  • arctic-embed-l

NIM fails to start with out-of-memory error#

If you attempt to start a NIM, in some cases it fails to start with an out-of-memory error.

TensorRT pre-allocates memory in accordance with the maximum input size based on the loaded TensorRT profiles. Model instances are a multiplier on VRAM requirements, and different NIMs require widely different amounts of VRAM.

To resolve this issue, use one of the following options:

  • Ensure your GPU has sufficient VRAM for the model. Refer to the support matrix for approximate memory requirements by compute capability.

  • On GPUs without enough VRAM for multiple model instances, run only a single instance of the reranker.