Agents | NVIDIA Dynamo Documentation

NVIDIA Dynamo adds agent-aware serving features without taking ownership of the agent loop: your harness still manages prompts, tools, subagents, and reasoning state, while Dynamo uses metadata attached to each LLM request to correlate work, improve routing and scheduling, manage KV cache behavior, and produce traces for replay and analysis.

Send Your First Agent Request

Chat completions with session ID and agent hints

$ curl http://localhost:8000/v1/chat/completions \
>   -H 'Content-Type: application/json' \
>   -H 'Authorization: Bearer sk-dummy' \
>   -H 'x-dynamo-session-id: research-run-42:researcher' \
>   -d '{
>     "model": "my-model",
>     "messages": [{"role": "user", "content": "..."}],
>     "nvext": {
>       "agent_hints": {
>         "priority": 5,
>         "osl": 1024
>       }
>     }
>   }'

Session IDs identify work for tracing and opt-in consumers. Agent hints influence serving behavior. Neither enables sticky placement unless a separate routing policy is configured.

Choose Your Metadata

Metadata	Where	Role	Use when
Session ID	Request header	Passive identity	You need traceability across LLM calls, tool spans, and replay
Agent hints	`nvext.agent_hints` in the request body	Active serving intent	You want to influence queue order, scheduling, or cache behavior

See Session IDs and Agent Hints for the full contract.

Implementation Checklist

Configure a packaged harness when available

Point Codex, Pi, Claude Code, or another supported CLI at Dynamo. Supported harnesses emit native session headers, so you do not need to add X-Dynamo-Session-ID yourself. See Agent Harnesses.

Identify agent sessions from a custom client

If you are building a custom client instead of using a packaged harness, send X-Dynamo-Session-ID on every request in a reasoning chain. See Session IDs.

Add agent hints when serving intent matters

Set nvext.agent_hints only when you want router or engine behavior to change for that request. See Agent Hints.

Enable tracing if you need measurements

Set DYN_REQUEST_TRACE=1 on the frontend to capture timing, tool calls, and session identity. See Agent Tracing.

Where Signals Are Used

Layer	Signal	Optimization
Frontend API	Session headers and `nvext` request extensions	Normalize agent identity and serving intent across APIs.
Router	Priority, expected output length, and cache-overlap signals	Place requests for KV reuse and order queued work.
KV cache management	Priority and session metadata forwarded to the backend runtime	Influence engine scheduling, cache eviction, and subagent KV isolation where the backend supports it.

Next Steps

Session IDs

Stable agent identity for tracing and opt-in consumers.

Agent Hints

Per-request priority, expected output length, and speculative prefill.

Agent Harnesses

Connect Codex, Pi, Claude Code, and other supported coding agents.

Agent Tracing

Request traces, inferred tool calls, and Perfetto conversion.

Agent Simulation

Convert agent traces into replay and simulation inputs.

ThunderAgent Program Scheduler

Experimental tool-boundary pause/resume scheduler on top of KV-aware routing.