> This page is for version 1.8.1 (default).
> For other versions, use one of these documentation indexes:
> - 1.8.1 (default): https://docs.nvidia.com/networking/cloudai/1.8.1/llms.txt
> - 1.7.1: https://docs.nvidia.com/networking/cloudai/1.7.1/llms.txt
> - 1.6.1: https://docs.nvidia.com/networking/cloudai/1.6.1/llms.txt
> - dev: https://docs.nvidia.com/networking/cloudai/dev/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/networking/cloudai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/networking/cloudai/_mcp/server.

DeepEP Benchmark workload (`test_template_name` is `DeepEP`) allows users to execute DeepEP (Deep Expert Parallelism) MoE (Mixture of Experts) benchmarks within the CloudAI framework.

## Overview

DeepEP is a benchmark for measuring the performance of MoE models with distributed expert parallelism. It supports:

* **Two operation modes**: Standard and Low-Latency

* **Multiple data types**: bfloat16 and FP8

* **Flexible network configurations**: With or without NVLink

* **Configurable model parameters**: Experts, tokens, hidden size, top-k

* **Performance profiling**: Kineto profiler support

## Usage Example

Test TOML example (Standard Mode):

```toml
name = "deepep_standard"
description = "DeepEP MoE Benchmark - Standard Mode"
test_template_name = "DeepEP"

[cmd_args]
docker_image_url = "<docker container url here>"
mode = "standard"
tokens = 1024
num_experts = 256
num_topk = 8
hidden_size = 7168
data_type = "bfloat16"
num_warmups = 20
num_iterations = 50
```

Test TOML example (Low-Latency Mode):

```toml
name = "deepep_low_latency"
description = "DeepEP MoE Benchmark - Low Latency Mode"
test_template_name = "DeepEP"

[cmd_args]
docker_image_url = "<docker container url here>"
mode = "low_latency"
tokens = 128
num_experts = 256
num_topk = 1
hidden_size = 7168
data_type = "bfloat16"
allow_nvlink_for_low_latency = false
allow_mnnvl = false
```

Test Scenario example:

```toml
name = "deepep-benchmark"

[[Tests]]
id = "Tests.1"
test_name = "deepep_standard"
num_nodes = 2
time_limit = "00:30:00"
```

Test-in-Scenario example:

```toml
name = "deepep-benchmark"

[[Tests]]
id = "Tests.1"
num_nodes = 2
time_limit = "00:30:00"

name = "deepep_standard"
description = "DeepEP MoE Benchmark"
test_template_name = "DeepEP"

  [Tests.cmd_args]
  docker_image_url = "<docker container url here>"
  mode = "standard"
  tokens = 1024
  num_experts = 256
  num_topk = 8
```

## API Documentation

### Command Arguments

*class* cloudai.workloads.deepep.deepep.DeepEPCmdArgs(

*\**,

*docker_image_url: str*,

*subtest_name: Literal\['test_internode', 'test_intranode', 'test_low_latency', 'test_ep'] = 'test_internode'*,

*deep_ep_root: str = '/workspace/DeepEP'*,

*legacy_tests_root: str | None = None*,

*elastic_tests_root: str | None = None*,

*python_executable: str = 'python'*,

*num_processes: int = 8*,

*num_tokens: int = 4096*,

*hidden: int = 7168*,

*num_topk: int = 8*,

*num_experts: int = 256*,

*num_topk_groups: int | None = None*,

*allow_mnnvl: bool = False*,

*test_ll_compatibility: bool = False*,

*pressure_test_mode: int = 0*,

*pressure_test: bool = False*,

*shrink_test: bool = False*,

*disable_nvlink: bool = False*,

*use_logfmt: bool = False*,

*num_sms: int = 0*,

*num_qps: int = 0*,

*num_allocated_qps: int = 0*,

*num_gpu_timeout_secs: int = 100*,

*num_cpu_timeout_secs: int = 100*,

*sl_idx: int = 0*,

*do_cpu_sync: int = 1*,

*allow_hybrid_mode: int = 1*,

*allow_multiple_reduction: int = 1*,

*prefer_overlap_with_compute: int = 0*,

*deterministic: bool = False*,

*seed: int = 0*,

*skip_check: bool = False*,

*skip_perf_test: bool = False*,

*do_pressure_test: bool = False*,

*reuse_elastic_buffer: bool = False*,

*test_first_only: bool = False*,

*unbalanced_ratio: float = 1.0*,

*precise_unbalanced_ratio: bool = False*,

*masked_ratio: float = 0.0*,

*dump_profile_traces: str = ''*,

*ignore_local_traffic: bool = False*,

*\*\*extra_data: Any*,

)

Bases: `CmdArgs`

Command arguments for the official DeepEP test scripts.

### Test Definition

*class* cloudai.workloads.deepep.deepep.DeepEPTestDefinition(*\*, name: str, description: str, test_template_name: str, cmd_args: \~cloudai.workloads.deepep.deepep.DeepEPCmdArgs, dse_excluded_args: list\[str] = \<factory>, extra_env_vars: dict\[str, str | \~typing.List\[str]] = \{}, extra_cmd_args: dict\[str, str] = \{}, extra_container_mounts: list\[str] = \[], git_repos: list\[\~cloudai.\_core.installables.git_repo.GitRepo] = \[], nsys: \~cloudai.models.workload.NsysConfiguration | None = None, predictor: \~cloudai.models.workload.PredictorConfig | None = None, training_report: \~cloudai.models.workload.TrainingReportConfig | None = None, agent: str = 'grid_search', agent_steps: int = 1, agent_metrics: list\[str] = \['default'], agent_reward_function: str = 'inverse', agent_config: dict\[str, \~typing.Any] | None = None, env_params: dict\[str, \~cloudai.configurator.env_params.EnvParamSpec] = \<factory>*)

Bases: `TestDefinition`

Test object for official DeepEP v1/v2 test scripts.

**&#x20;*property* is_domain_randomization_enabled*: bool*\*\*

at least one `env_params` annotation.

**Type:**

Whether the config declares domain randomization

**&#x20;is_dse_excluded_arg(*path: str*) → bool**

Return whether a dot-separated cmd_args path should be ignored by DSE.

**&#x20;is_env_sampled(*cmd_args_path: str*) → bool**

Whether a cmd_args field is env-sampled (env draws it per trial, not the agent).

metric_observations(

*system: System*,

*tr: TestRun*,

) → list\[MetricObservation]

Return canonical metric observations produced by this test run, if supported.

**&#x20;*validator* validate_env_params*  » \* *all fields**\*

Validate env_params annotations against cmd_args.

`env_params` is an annotation: each key names a `cmd_args` field whose value is the candidate set (the single source of truth), and the entry carries only *how* to sample. So each key must name a real `cmd_args` field whose value is a candidate list; a scalar is already fixed, so annotating it is a meaningless label and is rejected here. When `weights` are declared, the list needs >= 2 values and the weights must align 1:1 with it. Sampling, persistence, the per-trial cmd_args overlay, and the cache key all live in `CloudAIGymEnv`; keeping this shape check in core lets the overlay stay agent- and workload-agnostic rather than re-implemented per workload.