> This page is for version 1.14.0 (default). > For other versions, use one of these documentation indexes: > - 1.14.0 (default): https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/1.14.0/llms.txt > - 1.10.1: https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/1.10.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/_mcp/server. # Support Matrix > Software requirements and model profiles for Nemotron Safety Guard 8B NIM. ## Software * CUDA 12.9 is configured in the microservice container. * NVIDIA GPU Driver version 525 or higher. NVIDIA verified the documentation using 570.133.20. * A container runtime environment such as Docker or Kubernetes. For Docker, see the [Docker installation instructions](https://docs.docker.com/engine/install/). * NVIDIA Container Toolkit installed and configured. See the [installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). * An active NVIDIA AI Enterprise subscription or membership in the [NVIDIA Developer Program](https://build.nvidia.com/explore/discover?integrate_nim=true\&developer_enroll=true\&self_hosted_api=true\&signin=true). Access to the container and model is restricted. * An NGC API key to download the container and model. ## About Model Profiles NVIDIA NIM microservices use model engines tuned for specific GPU models, GPU counts, precision, and other settings. These combinations are called *model profiles*. Each profile has a unique 64-character hexadecimal profile ID. The NIM can select a profile automatically based on available GPUs or use a profile ID that you specify. The model manifest is stored at `/opt/nim/etc/default/model_manifest.yaml` in the container. ## NVIDIA Llama 3.1 Nemotron Safety Guard 8B NIM Model Profiles The model requires 48 GB of GPU memory. NVIDIA developed and tested it with these GPUs: * B200 SXM * H200 SXM * H100 SXM * A100 SXM 40 GB and 80 GB * A100 PCIe 40 GB and 80 GB (generic profiles only) * A10G * L40S PCIe * H100 NVL * H200 NVL * GB200 NVL72 * GH200 NVL2 * GH200 96 GB * RTX PRO 6000 Blackwell Server Edition You can use one GPU with the required memory capacity or multiple GPUs that meet the capacity. For details about locally buildable and generic profiles, see [NIM for LLMs model profiles](https://docs.nvidia.com/nim/large-language-models/latest/profiles.html). The table lists the model profiles in the 1.14.0 manifest. Profile IDs select an exact profile. | GPU | Precision | Engine | Profile | GPUs | Tensor parallelism | Profile ID | | ------------------------------------- | --------- | ------------ | ---------- | ---: | -----------------: | ------------------------------------------------------------------ | | B200 | FP8 | TensorRT-LLM | throughput | 1 | 1 | `8258db301f5fb0a5b5a5b9c263af9334ffc0714c6ae82d2a0687b95c11f5834e` | | B200 | FP8 | TensorRT-LLM | latency | 2 | 2 | `1c67491281ac66f32ca917bc566808bf4657ad20dec137f2b40c78d95b3a40dd` | | B200 | BF16 | TensorRT-LLM | throughput | 1 | 1 | `d9bab56236ea000a0e0760e76d2530d7e460dde85cb6487b496009cbbb9ea7b2` | | B200 | BF16 | TensorRT-LLM | latency | 2 | 2 | `7b2460a744467de931be74543f3b1be6a4540edd8f5d3523c31aca253d3ee714` | | GB200 | FP8 | TensorRT-LLM | throughput | 1 | 1 | `45999aa05eca849f11beb20bbf70dd48f2970a01dcf4c2617302b8038fca1662` | | GB200 | FP8 | TensorRT-LLM | latency | 2 | 2 | `cfca3a90be399e2fc6b91dfe19aa141fe7db0ad114df365cf43d77c675711049` | | GB200 | BF16 | TensorRT-LLM | throughput | 1 | 1 | `20ac8ad6d3cb6b083facc320669292ab06972015f432839c64c841494beb4177` | | GB200 | BF16 | TensorRT-LLM | latency | 2 | 2 | `3b86e1c4eafac6dd59612ed5cea6878177867773d567fcc0e0127ad5b2b1221b` | | GH200 144GB | FP8 | TensorRT-LLM | throughput | 1 | 1 | `c53b2566c86bf5ca53d25883b3f305f7fcfa1b8a854a52bf21d6901fec27fd3f` | | GH200 144GB | FP8 | TensorRT-LLM | latency | 2 | 2 | `052a14156d375521d64c584a0197a00ab3c54ae742b55145f1df091072656de7` | | GH200 144GB | BF16 | TensorRT-LLM | throughput | 1 | 1 | `8f0e0665f4f53a02e2e68ab690ba990d5d9a3a6b49cdcd4ff679031757de30ed` | | GH200 144GB | BF16 | TensorRT-LLM | latency | 2 | 2 | `542edd6068b3fee7bb9431ba988f167dfc9f6e5b6dbf005b2e208d42bd17d705` | | GH200 480GB | FP8 | TensorRT-LLM | throughput | 1 | 1 | `8534f70d957abfcbb4353276dcb754f0a0820e90f555afe233be835f8d5c9743` | | GH200 480GB | FP8 | TensorRT-LLM | latency | 1 | 1 | `9784974c7d8bd2e2081fc751d02a70bd619b1b16d535d3ff764622c1b7e73ff2` | | GH200 480GB | BF16 | TensorRT-LLM | throughput | 1 | 1 | `076e5a3eec39670d90ef75725966fabc895074660b42a0ba3edce65c0eb15756` | | GH200 480GB | BF16 | TensorRT-LLM | latency | 1 | 1 | `f41c136a67469ae0bda89f96d78cb8c2b9c01c27d0ac618112248025320817c3` | | H200 NVL | FP8 | TensorRT-LLM | throughput | 1 | 1 | `7a4c46a6911fff7e11e99df87161eeae73b08c48cc75f1928e8289fbacc188fc` | | H200 NVL | FP8 | TensorRT-LLM | latency | 2 | 2 | `373b366ba1d0bf15e9c7087967fadcc00db30c079943feee2ae0e2489aee9d66` | | H200 NVL | BF16 | TensorRT-LLM | throughput | 1 | 1 | `58041a8daba4de24b75c86456d256fbc34e881de58ebdab6135aad18466d8130` | | H200 NVL | BF16 | TensorRT-LLM | latency | 2 | 2 | `dbb457d9b5a45d0a6976c0ba1a8ee6072deb8fe64c49a12e47ba9c71863618d2` | | H200 | FP8 | TensorRT-LLM | throughput | 1 | 1 | `908a7d5dbabe5e1a7956b66d90747b34d44edb357546ab098474efc28a1f95f2` | | H200 | FP8 | TensorRT-LLM | latency | 2 | 2 | `5a708fe91514e2aa44438158af955b35d51fab4ca1fb7268e35930e67fce6e08` | | H200 | BF16 | TensorRT-LLM | throughput | 1 | 1 | `1b953829736de1f2ffc00c8d25316a24d43ed398b5f8599957c9547007fd5471` | | H200 | BF16 | TensorRT-LLM | latency | 2 | 2 | `158a13eff79873eb73689daf87c365fa06946f74856646e54edc69728ef59a8e` | | H100 NVL | FP8 | TensorRT-LLM | throughput | 1 | 1 | `f3fd46462fe31ea44e5207d9f97ea190d8928b5688f5752445bb7dfd8afc999c` | | H100 NVL | FP8 | TensorRT-LLM | latency | 2 | 2 | `5e8a78e4d0c9e2e513466ec23ac181ae8d75ce05bda5c4653eddf8f3a99f2d58` | | H100 NVL | BF16 | TensorRT-LLM | throughput | 1 | 1 | `dbf1e0e5f43bcabfdd0791b0ffa83036fd7cf6be28fd5266990cf94bb8ffc338` | | H100 NVL | BF16 | TensorRT-LLM | latency | 2 | 2 | `a91cc87e8e98c7e88967319c273392e447fab72dd22aa8231630b573284525b2` | | H100 | FP8 | TensorRT-LLM | throughput | 1 | 1 | `21879fb81cb016c598cb4324a7cfa93acac1904ebc8cbfd9fff42d79a929b19d` | | H100 | FP8 | TensorRT-LLM | latency | 2 | 2 | `3a8d53337b395483663f3d96e4528a759d1474c5afa8b68d74a39e99fad7aebf` | | H100 | BF16 | TensorRT-LLM | throughput | 1 | 1 | `dfe2b2518ce82cb404a639f9039d956ec1b8f35703e7594d066ef080d1172156` | | H100 | BF16 | TensorRT-LLM | latency | 2 | 2 | `2f69689bf8fef4118bb018bb07869fc2d4b6eb3185115b2117ad62150f5d0006` | | A100 | BF16 | TensorRT-LLM | throughput | 1 | 1 | `8125bb2032b5649b5152192498543b77543de174d39f18dfb7473a6332c9b3fd` | | A100 | BF16 | TensorRT-LLM | latency | 2 | 2 | `f5a41523d4dd3de5340276b18dfcbce17bb047b2457a36c1310e61589cc28935` | | A100 SXM4 40GB | BF16 | TensorRT-LLM | throughput | 1 | 1 | `3f24e5c0ba016486c057ff9997caa9341f1f974f5dbf53e96fdbf9a0558bc9ab` | | A100 SXM4 40GB | BF16 | TensorRT-LLM | latency | 2 | 2 | `bd225e66799ed1e826ae5e2d7db017e1ba9a73855807e4f362967f22b2d7fecb` | | A10G | BF16 | TensorRT-LLM | throughput | 4 | 4 | `105f6654f47c50206ac6dc9662aa774e1d8bad4b7268e1db3358ac0e446d9abc` | | A10G | BF16 | TensorRT-LLM | latency | 8 | 8 | `80b75e35feecbcc4f3cc33ea634318bff503afcea1924a140f9fe9c3a848b99c` | | L40S | FP8 | TensorRT-LLM | throughput | 1 | 1 | `032ae037e7cddf2e43f37341027d8060e5eb5082797ed3e39d803f1c6af4eb84` | | L40S | FP8 | TensorRT-LLM | latency | 2 | 2 | `f282d4039fc42e3ab8a69854daf1a3a9e0fdce7974d06c3924969e3196e4ac08` | | L40S | BF16 | TensorRT-LLM | throughput | 1 | 1 | `7dedab87edfe691a1eb95d7bdc3fa74cc3e7fb6eeb6b56ab933f2ea3561d4c8d` | | L40S | BF16 | TensorRT-LLM | latency | 2 | 2 | `260a963986f4b26d55e2af38ef19fd45341314564cc55f6b3f968e0323a8daea` | | RTX PRO 6000 Blackwell Server Edition | FP8 | TensorRT-LLM | latency | 1 | 1 | `9325fdaeebf47da133de2bffcbfdcfb7537de5f0cf72adb091fb5bb8429047ad` | | RTX PRO 6000 Blackwell Server Edition | FP8 | TensorRT-LLM | throughput | 1 | 1 | `b5938999f25208fbf5ef64ad1ce0d3978ac52d5b18d32f9b7e1551cbc97737db` | | RTX PRO 6000 Blackwell Server Edition | BF16 | TensorRT-LLM | throughput | 1 | 1 | `86edc198bd7a53981bd95f612e07e204727e116bdf779f1405e0c8ff92e0f067` | | RTX PRO 6000 Blackwell Server Edition | BF16 | TensorRT-LLM | latency | 2 | 2 | `5f2aea268787b1aea54e98d370f99685f2d6066310615ff946cf4f76242dcbfc` | | Generic | BF16 | vLLM | Generic | 1 | 1 | `47804570e8db67fdc60d9cb518b0f60316ac4eb0f3a55af81aafefe124505a8f` | | Generic | BF16 | vLLM | Generic | 2 | 2 | `60cc64256f69a6242aa2b7320a1070ae84e88fe5416674cbbe9a2cb6ecb8e992` | | Generic | BF16 | vLLM | Generic | 4 | 4 | `511022fb9a93b2c8b1eb84f4c472f05833e6904e51de7fd676be168a0aa9d63d` | | Generic | BF16 | vLLM | Generic | 8 | 8 | `a98cb8c244d2fd95e65a102758e4b039c94864f322c5b64bc9ee5749ddac59ab` |