> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# TensorRT-LLM

## Docs

- [TensorRT-LLM](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/overview.md): TensorRT-LLM engines run in Dynamo's distributed runtime with disaggregated serving, KV-aware routing, and multinode support.
- [TensorRT-LLM Sidecar](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/sidecar.md): Run Dynamo beside a stock TensorRT-LLM engine through native gRPC.
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/reference-guide.md): Features, configuration, and operational details for the TensorRT-LLM backend
- [TensorRT-LLM Observability](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/observability.md): TensorRT-LLM exposes request, latency, and KV cache transfer metrics through Dynamo's Prometheus /metrics endpoint.
- [Known Issues and Mitigations](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/known-issues.md): Known issues for TensorRT-LLM in Dynamo, including disaggregated KV cache exhaustion and driver mismatch errors, with mitigations.
- [TensorRT-LLM Multimodal](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal.md)