> This page is for version 1.14.0 (default).
> For other versions, use one of these documentation indexes:
> - 1.14.0 (default): https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/1.14.0/llms.txt
> - 1.10.1: https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/1.10.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/_mcp/server.

# Release Notes

> Nemotron Safety Guard 8B NIM 1.14.0 features and known issues.

## v1.14.0

### Features

* The microservice serves a GPU-accelerated LLM model for multilingual content moderation and detects harmful content in user messages or bot responses.
* KV Cache reuse, controlled by `NIM_ENABLE_KV_CACHE_REUSE`, is enabled by default to improve performance. In multi-tenant environments, caching can introduce information leaks. See [Structuring Applications to Secure the KV Cache](https://developer.nvidia.com/blog/structuring-applications-to-secure-the-kv-cache/) for security considerations.

### Known Issues

TensorRT-LLM profiles can experience lower throughput under high load compared to generic profiles that use vLLM. The performance impact varies by GPU and workload, including input and output sequence lengths and concurrency.

The following TensorRT-LLM profiles show reduced throughput compared to vLLM:

* **BF16 precision profiles:**
  * B200, GB200, GH200 (144 GB), H100, H100 NVL, and H200 NVL: lower throughput at high concurrency (100–250 or more concurrent requests) with longer sequences (1000/1000 and 500/2000 tokens).
  * GH200 (480 GB) and H100 NVL: lower throughput at very high concurrency (250 or more concurrent requests) with longer output sequences (500/2000 tokens).
  * H200 BF16 profiles are not recommended for production use in this release.
* **FP8 precision profiles:**
  * B200 and GH200 (144 GB): 25–35% lower throughput at high concurrency (250 or more concurrent requests) with longer sequences (1000/1000 and 500/2000 tokens).
  * GB200, H100 NVL, and H200: lower throughput at high concurrency (100–250 or more concurrent requests) with longer sequences.
  * L40S: lower throughput at low concurrency with very long input sequences (5000 tokens).

For workloads with high concurrency and long sequences, consider vLLM-based generic model profiles for optimal throughput.

The following table lists the affected TensorRT-LLM profile IDs:

| Profile name                              | Profile ID                                                         |
| ----------------------------------------- | ------------------------------------------------------------------ |
| `tensorrt_llm-b200-bf16-2-latency`        | `7b2460a744467de931be74543f3b1be6a4540edd8f5d3523c31aca253d3ee714` |
| `tensorrt_llm-gb200-bf16-2-latency`       | `3b86e1c4eafac6dd59612ed5cea6878177867773d567fcc0e0127ad5b2b1221b` |
| `tensorrt_llm-gh200_144gb-bf16-2-latency` | `542edd6068b3fee7bb9431ba988f167dfc9f6e5b6dbf005b2e208d42bd17d705` |
| `tensorrt_llm-gh200_480gb-bf16-1-latency` | `f41c136a67469ae0bda89f96d78cb8c2b9c01c27d0ac618112248025320817c3` |
| `tensorrt_llm-h100_nvl-bf16-2-latency`    | `a91cc87e8e98c7e88967319c273392e447fab72dd22aa8231630b573284525b2` |
| `tensorrt_llm-h100-bf16-2-latency`        | `2f69689bf8fef4118bb018bb07869fc2d4b6eb3185115b2117ad62150f5d0006` |
| `tensorrt_llm-h200_nvl-bf16-2-latency`    | `dbb457d9b5a45d0a6976c0ba1a8ee6072deb8fe64c49a12e47ba9c71863618d2` |
| `tensorrt_llm-h200-bf16-2-latency`        | `158a13eff79873eb73689daf87c365fa06946f74856646e54edc69728ef59a8e` |
| `tensorrt_llm-b200-fp8-2-latency`         | `1c67491281ac66f32ca917bc566808bf4657ad20dec137f2b40c78d95b3a40dd` |
| `tensorrt_llm-gb200-fp8-2-latency`        | `cfca3a90be399e2fc6b91dfe19aa141fe7db0ad114df365cf43d77c675711049` |
| `tensorrt_llm-gh200_144gb-fp8-2-latency`  | `052a14156d375521d64c584a0197a00ab3c54ae742b55145f1df091072656de7` |
| `tensorrt_llm-h100_nvl-fp8-2-latency`     | `5e8a78e4d0c9e2e513466ec23ac181ae8d75ce05bda5c4653eddf8f3a99f2d58` |
| `tensorrt_llm-h200-fp8-2-latency`         | `5a708fe91514e2aa44438158af955b35d51fab4ca1fb7268e35930e67fce6e08` |
| `tensorrt_llm-l40s-fp8-2-latency`         | `f282d4039fc42e3ab8a69854daf1a3a9e0fdce7974d06c3924969e3196e4ac08` |

These issues from NVIDIA NIM for LLMs v1.14.0 are also present:

* With TensorRT-LLM, setting `temperature=0` enforces greedy decoding and makes `repetition_penalty` ineffective.
* In `/v1/metadata`, `modelInfo.repository_override` and `modelInfo.selectedModelProfileId` are missing.
* If `NIM_GUIDED_DECODING_BACKEND` is `lm-format-enforcer` or a custom backend, set `NIM_ENABLE_KV_CACHE_REUSE=0`; otherwise, an incorrect backend name is treated as a custom backend.

## v1.10.1

### Features

This is the first release of Llama 3.1 Nemotron Safety Guard Multilingual 8B V1 NIM.
The microservice serves a GPU-accelerated LLM model that performs multilingual content moderation for building trustworthy LLM applications.
The LLM model detects harmful content in user messages or bot responses.

### Known Issues

* The locally-buildable profiles are not supported with A100 PCIe GPUs.
  You can workaround the issue by specifying a generic model profile that matches the number of GPUs on the host.