This page is for version v0.8.1.
For other versions, use one of these documentation indexes:
- Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
- dev: https://docs.nvidia.com/dynamo/dev/llms.txt
- v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
- v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
- v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
- v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
- v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
- v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
- v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
- v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
- v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
- v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
- v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
- v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
- v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
- v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
- v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
- v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
- v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
- v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
- v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt
The Dynamo KV Block Manager (KVBM) is a scalable runtime component designed to handle memory allocation, management, and remote sharing of Key-Value (KV) blocks for inference tasks across heterogeneous and distributed environments. It acts as a unified memory layer for frameworks like vLLM, SGLang, and TRT-LLM.
It offers:
- A unified memory API that spans GPU memory (in future), pinned host memory, remote RDMA-accessible memory, local or distributed pool of SSDs and remote file/object/cloud storage systems.
- Support for evolving block lifecycles (allocate → register → match) with event-based state transitions that storage can subscribe to.
- Integration with NIXL, a dynamic memory exchange layer used for remote registration, sharing, and access of memory blocks over RDMA/NVLink.
The Dynamo KV Block Manager serves as a reference implementation that emphasizes modularity and extensibility. Its pluggable design enables developers to customize components and optimize for specific performance, memory, and deployment needs.
Feature Support Matrix
