Support Matrix
About Model Profiles
NVIDIA NIM microservices use model engines tuned for specific GPU models, GPU counts, precision, and other settings. These combinations are called model profiles. Each profile has a unique 64-character hexadecimal profile ID. The NIM selects a profile automatically based on available GPUs or can use a profile ID that you specify.
The model manifest is stored at /opt/nim/etc/default/model_manifest.yaml in the container.
NVIDIA Llama 3.1 NemoGuard 8B TopicGuard Model Profiles
The model requires 48 GB of GPU memory. NVIDIA developed and tested it with these GPU and tensor-parallelism configurations:
- B200: 1 or 2
- GH200 480 GB: 1
- H200: 1 or 2
- H200 NVLink: 1 or 2
- H100: 1 or 2
- H100 NVLink: 1 or 2
- A100: 1 or 2
- A100 SXM4 40 GB: 1 or 2
- A10G: 4 or 8
- L40S: 2 or 4
For details about locally buildable and generic model profiles, see NIM for LLMs model profiles.