Component Interaction Matrix#
Component |
Interacts With |
Integration Type |
|---|---|---|
KV Block Manager, Router, Planner, NIXL |
Runtime |
|
Dynamo, Triton, KV Block Manager |
Request routing |
|
NIXL, Dynamo, Router |
Memory management |
|
KV Block Manager, Model Express, GPUs |
Data transfer |
|
NIXL, Triton, Dynamo |
Model loading |
|
Dynamo, Grove |
Performance tuning |
|
KAI Scheduler, Planner |
Orchestration |
|
Grove, GPU Operator |
Scheduling |
|
Network Operator, KAI Scheduler |
Infrastructure |
|
GPU Operator |
Infrastructure |
|
— |
Planning |
|
Triton, Dynamo |
Benchmarking |
|
TensorRT, TensorRT-LLM |
Optimization |
|
Dynamo, Model Optimizer |
Optimization |
|
All components |
Distribution |