Admission Control | NVIDIA Dynamo Documentation

Backend admission control limits requests inside each engine and queues excess work outside it. Configure the gate on the worker containers in your DynamoGraphDeployment (DGD). It supports both Dynamo Python and Dynamo Sidecar backends.

For behavior and tuning guidance, see Admission Control Architecture.

Configure the Worker

Start with a working DynamoGraphDeployment. These three settings apply per data-parallel (DP) rank:

Environment variable	What it limits	Default per rank
`DYN_BACKEND_ADMISSION_ENGINE_REQUEST_LIMIT`	Admitted requests that have not finished.	Reported `max_num_seqs`; `10000` until reported.
`DYN_BACKEND_ADMISSION_ENGINE_REQUEST_WAITING_LIMIT`	Admitted requests awaiting their first generated token or completion.	Reported `max_num_seqs`; `10000` until reported.
`DYN_BACKEND_ADMISSION_REQUEST_QUEUE_LIMIT`	Requests waiting outside the engine. Set `0` to disable queueing.	Twice reported `max_num_seqs`; `20000` until reported.

A request needs room under both engine limits. Otherwise, it waits in the gate queue. If the queue is also full, the worker rejects it; overload reaches the client as HTTP 529 by default.

The following nvidia.com/v1beta1 fragment overrides all three settings. Add these entries to your worker container’s existing env list. Omit an entry to keep its default.

spec:
  components:
    - name: worker
      type: worker
      podTemplate:
        spec:
          containers:
            - name: main
              env:
                - name: DYN_BACKEND_ADMISSION_ENGINE_REQUEST_LIMIT
                  value: "256"
                - name: DYN_BACKEND_ADMISSION_ENGINE_REQUEST_WAITING_LIMIT
                  value: "32"
                - name: DYN_BACKEND_ADMISSION_REQUEST_QUEUE_LIMIT
                  value: "512"

For a disaggregated deployment, configure prefill and decode workers separately. Set these variables on the workers, not the frontend. See Configuration for CLI precedence and aliases, and Best Practices for sizing guidance.

Verify the Configuration

Check the worker logs for the effective limits. Under load, use the dynamo_backend_admission_* metrics to observe engine occupancy, queued requests, rejections, and cancellations. The dp_rank label identifies the rank; unspecified covers unranked requests. Sum over dp_rank for worker totals.

See Metrics for metric names and labels.