This page is for version dev.
For other versions, use one of these documentation indexes:
- 1.8.1 (default): https://docs.nvidia.com/networking/cloudai/1.8.1/llms.txt
- 1.7.1: https://docs.nvidia.com/networking/cloudai/1.7.1/llms.txt
- 1.6.1: https://docs.nvidia.com/networking/cloudai/1.6.1/llms.txt
- dev: https://docs.nvidia.com/networking/cloudai/dev/llms.txt
This workload (test_template_name is NixlPerftest) runs NIXL performance testing suite for comprehensive network performance evaluation.
Usage Examples
Test TOML example:
name = "my_nixl_perftest_test"description = "Example NIXL Perftest test"test_template_name = "NixlPerftest"[cmd_args]docker_image_url = "<docker container url here>"subtest = "sequential-ct-perftest"num_user_requests = 1batch_size = 1num_prefill_nodes = 1num_decode_nodes = 1prefill_tp = 4decode_tp = 4isl_mean = 10000isl_scale = 3000model = "deepseek-r1-distill-llama-70b"
Test Scenario example:
name = "nixl-perftest-test"[[Tests]]id = "perftest.1"num_nodes = 2time_limit = "00:20:00"test_name = "my_nixl_perftest_test"
Test-in-Scenario example:
name = "nixl-perftest-test"[[Tests]]id = "perftest.1"num_nodes = 2time_limit = "00:20:00"name = "my_nixl_perftest_test"description = "Example NIXL Perftest test"test_template_name = "NixlPerftest"[Tests.cmd_args]docker_image_url = "<docker container url here>"subtest = "sequential-ct-perftest"num_user_requests = 100batch_size = 1num_prefill_nodes = 1num_decode_nodes = 1prefill_tp = 8decode_tp = 8model = "deepseek-r1-distill-llama-70b"[Tests.extra_env_vars]CUDA_VISIBLE_DEVICES = "$SLURM_LOCALID"
API Documentation
Command Arguments
NixlPerftestCmdArgs
CmdArgs for NIXL Perftest.
| Field | Type | Default | Description |
|---|---|---|---|
docker_image_url | str | Required | URL of the Docker image to use for the benchmark. |
etcd_path | str | 'etcd' | Path to the etcd executable. |
wait_etcd_for | int | 60 | Number of seconds to wait for etcd to become healthy. |
etcd_image_url | str | None | None | Optional URL of the Docker image to use for etcd, by default etcd will be run from the same image as the benchmark. |
subtest | Literal['sequential-ct-perftest'] | Required | |
perftest_script | str | '/workspace/nixl/benchmark/kvbench/main.py' | |
matgen_script | str | '/workspace/nixl/benchmark/kvbench/test/inference_workload_matgen.py' | |
python_executable | str | 'python' | |
num_user_requests | int | list[int] | Required | |
batch_size | int | list[int] | Required | |
num_prefill_nodes | int | list[int] | Required | |
num_decode_nodes | int | list[int] | Required | |
isl_mean | int | list[int] | None | None | |
isl_scale | int | list[int] | None | None | |
prefill_tp | int | list[int] | 1 | |
prefill_pp | int | list[int] | 1 | |
prefill_cp | int | list[int] | 1 | |
decode_tp | int | list[int] | 1 | |
decode_pp | int | list[int] | 1 | |
decode_cp | int | list[int] | 1 | |
model | str | list[str] | None | None | |
hidden_size | int | None | None | |
num_layers | int | None | None | |
num_heads | int | None | None | |
num_kv_heads | int | None | None | |
dtype_size | int | None | None | |
matgen_args | cloudai.workloads.nixl_perftest.nixl_perftest.MatgenCmdArgs | Factory: MatgenCmdArgs |
Test Definition
NixlPerftestTestDefinition
TestDefinition for NixlPerftest.
| Field | Type | Default | Description |
|---|---|---|---|
name | str | Required | |
description | str | Required | |
test_template_name | str | Required | |
cmd_args | cloudai.workloads.nixl_perftest.nixl_perftest.NixlPerftestCmdArgs | Required | |
dse_excluded_args | list[str] | Factory: list | |
extra_env_vars | dict[str, str | typing.List[str]] | {} | |
extra_cmd_args | dict[str, str] | {} | |
extra_container_mounts | list[str] | [] | |
git_repos | list[cloudai._core.installables.git_repo.GitRepo] | [] | |
nsys | cloudai.models.workload.NsysConfiguration | None | None | |
predictor | cloudai.models.workload.PredictorConfig | None | None | |
training_report | cloudai.models.workload.TrainingReportConfig | None | None | |
agent | str | 'grid_search' | |
agent_steps | int | 1 | |
agent_metrics | list[str] | ['default'] | |
agent_reward_function | str | 'inverse' | |
agent_config | dict[str, Any] | None | None | Agent configuration. |
env_params | dict[str, cloudai.configurator.env_params.EnvParamSpec] | Factory: dict | Environment parameters sampled by the env per trial. Sibling to cmd_args; not part of the agent’s action space. CloudAIGymEnv samples, persists them in the trajectory output, and includes them in the trajectory cache key. |