Component Interaction Matrix#

Component

Interacts With

Integration Type

Dynamo

KV Block Manager, Router, Planner, NIXL

Runtime

Router

Dynamo, Triton, KV Block Manager

Request routing

KV Block Manager

NIXL, Dynamo, Router

Memory management

NIXL

KV Block Manager, Model Express, GPUs

Data transfer

Model Express

NIXL, Triton, Dynamo

Model loading

Planner

Dynamo, Grove

Performance tuning

Grove

KAI Scheduler, Planner

Orchestration

KAI Scheduler

Grove, GPU Operator

Scheduling

GPU Operator

Network Operator, KAI Scheduler

Infrastructure

Network Operator

GPU Operator

Infrastructure

AIConfigurator

Planning

AIPerf

Triton, Dynamo

Benchmarking

Model Optimizer

TensorRT, TensorRT-LLM

Optimization

TensorRT-LLM

Dynamo, Model Optimizer

Optimization

nvcr.io

All components

Distribution