Skip to navigation
Agents

Run an Agent on Dynamo

Point a coding agent at a local Dynamo endpoint and see its requests in a trace
View as Markdown

By the end of this guide, a coding agent is sending requests to a local Dynamo deployment, and you can see those requests grouped by session in a request trace.

You need a Dynamo source checkout (run commands from the repository root) and 2 GPUs.

Start the Endpoint

bash examples/backends/sglang/launch/agg_agent.sh

This serves zai-org/GLM-4.7-Flash with SGLang at TP2 on port 8000. It also turns on the settings that matter for agent workloads:

SettingWhat it does for agents
--router-mode kvSends each request to the worker that already holds the most reusable KV cache.
--kv-events-configLets the worker report its cached blocks so the router can find reuse.
--radix-eviction-policy priorityWhen GPU cache fills, evicts low-priority entries first. It only has an effect once you send priorities.
--dyn-tool-call-parser, --dyn-reasoning-parserTurn model output into structured tool calls and reasoning. The values (glm47, glm45) are specific to GLM; change them with the model.
--enable-anthropic-apiServes /v1/messages so Claude Code can connect.
DYN_REQUEST_TRACE=1Records every request so you can analyze the run.

With one worker, KV routing has nothing to choose between. It starts to matter when you add workers.

The script enables request tracing with the default gzip sink. It writes segments named /tmp/dynamo-request-trace.NNNNNN.jsonl.gz, or uses DYN_REQUEST_TRACE_FILE_PATH as the prefix if you set it, and prints the prefix at startup.

Check that the endpoint is up:

curl http://localhost:8000/v1/models

Connect a Harness

Point your agent at http://localhost:8000. For Claude Code:

export ANTHROPIC_BASE_URL=http://localhost:8000
export ANTHROPIC_MODEL=zai-org/GLM-4.7-Flash
export ANTHROPIC_SMALL_FAST_MODEL=zai-org/GLM-4.7-Flash
export CLAUDE_CODE_ATTRIBUTION_HEADER=0 # preserve KV cache hits
export ANTHROPIC_API_KEY=
claude

Setup for Codex, Pi, OpenCode, OpenClaw, and Hermes is in Connect an Agent Harness. These harnesses send their own session headers, so you do not add x-dynamo-session-id yourself.

Send a Labeled Request

To see the same behavior without a harness, send a request with a session ID:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'x-dynamo-session-id: quickstart:main' \
  -d '{"model":"zai-org/GLM-4.7-Flash","messages":[{"role":"user","content":"Say hello."}]}'

Check the Trace

List the requests by session:

gzip -cd "${DYN_REQUEST_TRACE_FILE_PATH:-/tmp/dynamo-request-trace}".*.jsonl.gz \
  | jq -c '.event | select(.event_type=="request_end")
       | {session: .agent_context.session_id,
          input_tokens: .request.input_tokens,
          output_tokens: .request.output_tokens,
          cached_tokens: .request.cached_tokens}'

Each line is one LLM request. A coding agent produces many lines per session. cached_tokens is the number of input tokens served from KV cache, so it is the number to watch as a session grows. If you see requests with a null session, the harness is not sending a session header. See Identify agent sessions.

Next

Identify agent sessions explains the headers and how subagents are linked.