> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Profiler Configuration Reference

The Dynamo profiler has two entry points, each with its own arguments:

* **`python -m dynamo.profiler`** — the SLA profiler. Sweeps a model across tensor-parallel sizes and interpolates prefill/decode performance to recommend a deployment that meets your SLA. Takes a DynamoGraphDeploymentRequest (DGDR) spec plus a handful of operational flags.
* **`python -m dynamo.profiler.profile_endpoint`** — the low-level per-endpoint profiler. Profiles a single already-running prefill or decode endpoint.

For the profiling workflow and DGDR spec structure, see the [Profiler Guide](/dynamo/dev/knowledge-base/modular-components/profiler/overview). Field names, types, and defaults on this page come from [`__main__.py`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/profiler/__main__.py) and [`profile_endpoint.py`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/profiler/profile_endpoint.py).

## SLA profiler (`dynamo.profiler`)

**`--config`** `string` — required

DynamoGraphDeploymentRequestSpec as an inline JSON string or a path to a JSON/YAML file. This defines the model, backend, and SLA targets to profile against. Startup fails if it is neither a valid file path nor valid JSON.

---

**`--output-dir`** `string` — default: profiling\_results

Path to the output results directory. The profiler writes plots, the interpolation data, `profile_sla.log`, and a `profiler_status.yaml` status file here.

---

**`--deployment-timeout`** `integer` — default: 3600

Maximum seconds to wait for each trial deployment to become ready before failing.

---

**`--prefill-interpolation-granularity`** `integer` — default: 16

Number of input-sequence-length (ISL) samples used to interpolate prefill performance. Higher values give a finer prefill curve at the cost of more profiling runs.

---

**`--decode-interpolation-granularity`** `integer` — default: 6

Number of samples used to interpolate decode performance across the KV-cache load range. Higher values give a finer decode curve at the cost of more profiling runs.

---

**`--dry-run`** `boolean` — default: false

Skip deployments and benchmarking. Use this to validate the config and preview the planned sweep without provisioning GPUs.

---

## Per-endpoint profiler (`dynamo.profiler.profile_endpoint`)

This entry point profiles a single running endpoint. It is normally invoked by the SLA profiler, but you can run it directly against an existing deployment.

**`--mode`** `string` — required

Which phase to profile.

Allowed values:

prefill

decode

---

**`--model_name`** `string` — required

Model name to send with requests. Must match the model the endpoint serves.

---

**`--url`** `string` — required

Base URL of the endpoint to profile, for example `http://localhost:8000`.

---

**`--num_gpus`** `integer` — required

Number of GPUs backing the endpoint. Used to normalize the reported per-GPU performance.

---

**`--tokenizer_path`** `string`

Path to the tokenizer. When empty, the tokenizer is resolved from `--model_name`.

---

**`--max_kv_tokens`** `integer` — default: 0

Maximum KV-cache tokens for the endpoint. Used only in `decode` mode to bound the KV-load sweep. Decode profiling requires an explicit positive value; the default `0` is valid only for prefill mode and causes decode mode to fail validation.

---

**`--work_dir`** `string` — default: endpoint\_profiling\_results/

Directory to save this endpoint's profiling results.

---

**`--max_context_length`** `integer` — default: 16384

Maximum context length to probe when generating request payloads.

---

**`--interpolation_granularity`** `integer` — default: 8

Number of samples used to interpolate this endpoint's results.

---

**`--attention_dp_size`** `integer` — default: 1

Attention data-parallel size of the endpoint. Set this for MoE models that run attention with data parallelism.

---

## Related pages

#### [Profiler Guide](/dynamo/dev/knowledge-base/modular-components/profiler/overview)

Profiling workflow, DGDR spec structure, and how the SLA recommendation is produced.

#### [Profiler Examples](/dynamo/dev/knowledge-base/modular-components/profiler/profiler-examples)

End-to-end profiling runs and how to read the output.

#### [Planner Configuration](/dynamo/dev/reference/components/planner-configuration)

Field reference for the Planner that consumes profiler output.

#### [Runtime Configuration](/dynamo/dev/reference/components/runtime-configuration)

Shared Dynamo runtime flags parsed by every component.