> This page is for version v1.4.1. > For other versions, use one of these documentation indexes: > - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt > - dev: https://docs.nvidia.com/dynamo/dev/llms.txt > - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt > - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt > - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt > - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt > - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt > - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt > - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt > - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt > - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt > - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt > - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt > - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt > - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt > - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt > - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt > - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt > - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt > - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt > - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt > For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Profiler Configuration Reference The Dynamo profiler has two entry points, each with its own arguments: * **`python -m dynamo.profiler`** — the SLA profiler. Sweeps a model across tensor-parallel sizes and interpolates prefill/decode performance to recommend a deployment that meets your SLA. Takes a DynamoGraphDeploymentRequest (DGDR) spec plus a handful of operational flags. * **`python -m dynamo.profiler.profile_endpoint`** — the low-level per-endpoint profiler. Profiles a single already-running prefill or decode endpoint. For the profiling workflow and DGDR spec structure, see the [Profiler Guide](/dynamo/v1.4.1/knowledge-base/modular-components/profiler/overview). Field names, types, and defaults on this page come from [`__main__.py`](https://github.com/ai-dynamo/dynamo/blob/v1.4.1/components/src/dynamo/profiler/__main__.py) and [`profile_endpoint.py`](https://github.com/ai-dynamo/dynamo/blob/v1.4.1/components/src/dynamo/profiler/profile_endpoint.py). ## SLA profiler (`dynamo.profiler`) **`--config`** `string` — required DynamoGraphDeploymentRequestSpec as an inline JSON string or a path to a JSON/YAML file. This defines the model, backend, and SLA targets to profile against. Startup fails if it is neither a valid file path nor valid JSON. --- **`--output-dir`** `string` — default: profiling\_results Path to the output results directory. The profiler writes plots, the interpolation data, `profile_sla.log`, and a `profiler_status.yaml` status file here. --- **`--deployment-timeout`** `integer` — default: 3600 Maximum seconds to wait for each trial deployment to become ready before failing. --- **`--prefill-interpolation-granularity`** `integer` — default: 16 Number of input-sequence-length (ISL) samples used to interpolate prefill performance. Higher values give a finer prefill curve at the cost of more profiling runs. --- **`--decode-interpolation-granularity`** `integer` — default: 6 Number of samples used to interpolate decode performance across the KV-cache load range. Higher values give a finer decode curve at the cost of more profiling runs. --- **`--dry-run`** `boolean` — default: false Skip deployments and benchmarking. Use this to validate the config and preview the planned sweep without provisioning GPUs. --- ## Per-endpoint profiler (`dynamo.profiler.profile_endpoint`) This entry point profiles a single running endpoint. It is normally invoked by the SLA profiler, but you can run it directly against an existing deployment. **`--mode`** `string` — required Which phase to profile. Allowed values: prefill decode --- **`--model_name`** `string` — required Model name to send with requests. Must match the model the endpoint serves. --- **`--url`** `string` — required Base URL of the endpoint to profile, for example `http://localhost:8000`. --- **`--num_gpus`** `integer` — required Number of GPUs backing the endpoint. Used to normalize the reported per-GPU performance. --- **`--tokenizer_path`** `string` Path to the tokenizer. When empty, the tokenizer is resolved from `--model_name`. --- **`--max_kv_tokens`** `integer` — default: 0 Maximum KV-cache tokens for the endpoint. Used only in `decode` mode to bound the KV-load sweep. Decode profiling requires an explicit positive value; the default `0` is valid only for prefill mode and causes decode mode to fail validation. --- **`--work_dir`** `string` — default: endpoint\_profiling\_results/ Directory to save this endpoint's profiling results. --- **`--max_context_length`** `integer` — default: 16384 Maximum context length to probe when generating request payloads. --- **`--interpolation_granularity`** `integer` — default: 8 Number of samples used to interpolate this endpoint's results. --- **`--attention_dp_size`** `integer` — default: 1 Attention data-parallel size of the endpoint. Set this for MoE models that run attention with data parallelism. --- ## Related pages #### [Profiler Guide](/dynamo/v1.4.1/knowledge-base/modular-components/profiler/overview) Profiling workflow, DGDR spec structure, and how the SLA recommendation is produced. #### [Profiler Examples](/dynamo/v1.4.1/knowledge-base/modular-components/profiler/profiler-examples) End-to-end profiling runs and how to read the output. #### [Planner Configuration](/dynamo/v1.4.1/reference/components/planner-configuration) Field reference for the Planner that consumes profiler output. #### [Runtime Configuration](/dynamo/v1.4.1/reference/components/runtime-configuration) Shared Dynamo runtime flags parsed by every component.