Skip to navigation

AIConfigurator

View as Markdown

AIConfigurator workload (test_template_name is Aiconfigurator) runs the AIConfigurator predictor using the installed aiconfigurator Python package. It is a Standalone workload (no Slurm/Kubernetes/RunAI required).

Outputs

Each test run produces:

  • report.json: Predictor output (JSON dict of metrics and metadata)

  • stdout.txt / stderr.txt: Predictor logs

  • run_simple_predictor.sh: Repro script containing the exact executed command (useful for debugging)

Usage Example

Test TOML example (Disaggregated mode):

name = "aiconfigurator_disagg_demo"
description = "Example AIConfigurator disaggregated predictor"
test_template_name = "Aiconfigurator"
[cmd_args]
model_name = "LLAMA3.1_70B"
system = "h200_sxm"
backend = "trtllm"
version = "0.20.0"
isl = 4000
osl = 500
[cmd_args.disagg]
p_tp = 1
p_pp = 1
p_dp = 1
p_bs = 1
p_workers = 1
d_tp = 1
d_pp = 1
d_dp = 1
d_bs = 8
d_workers = 2
prefill_correction_scale = 1.0
decode_correction_scale = 1.0

Test TOML example (Aggregated/IFB mode):

name = "aiconfigurator_agg_demo"
description = "Example AIConfigurator aggregated predictor"
test_template_name = "Aiconfigurator"
[cmd_args]
model_name = "LLAMA3.1_70B"
system = "h200_sxm"
backend = "trtllm"
version = "0.20.0"
isl = 4000
osl = 500
[cmd_args.agg]
batch_size = 8
ctx_tokens = 16
tp = 1
pp = 1
dp = 1

Running

uv run cloudai run --system-config conf/common/system/standalone_system.toml \
--tests-dir conf/experimental/aiconfigurator/test \
--test-scenario conf/experimental/aiconfigurator/test_scenario/aiconfigurator_disagg.toml

API Documentation

Command Arguments

class cloudai.workloads.aiconfig.aiconfigurator.AiconfiguratorCmdArgs(

*,

python_executable: str = ‘python’,

model_name: str,

system: str,

backend: str = ‘trtllm’,

version: str = ‘0.20.0’,

isl: int | List[int],

osl: int | List[int],

agg: Agg | None = None,

disagg: Disagg | None = None,

**extra_data: Any,

)

Bases: CmdArgs

Command arguments for Aiconfigurator workload with nested agg/disagg configs.

Test Definition

class cloudai.workloads.aiconfig.aiconfigurator.AiconfiguratorTestDefinition(

*,

name: str,

description: str,

test_template_name: str,

cmd_args: AiconfiguratorCmdArgs,

extra_env_vars: dict[str, str | List[str]] = {},

extra_cmd_args: dict[str, str] = {},

extra_container_mounts: list[str] = [],

git_repos: list[GitRepo] = [],

nsys: NsysConfiguration | None = None,

predictor: PredictorConfig | None = None,

agent: str = ‘grid_search’,

agent_steps: int = 1,

agent_metrics: list[str] = [‘default’],

agent_reward_function: str = ‘inverse’,

agent_config: dict[str, Any] | None = None,

)

Bases: TestDefinition

Test object for running Aiconfigurator predictor as a workload.

Command Generation Strategy (Standalone)

class cloudai.workloads.aiconfig.standalone_command_gen_strategy.AiconfiguratorStandaloneCommandGenStrategy(

system: System,

test_run: TestRun,

)

Bases: CommandGenStrategy

Generate a standalone command that invokes the Aiconfigurator predictor and writes JSON output.

store_test_run() → None

Store the test run information in output folder.

Only at command generation time, CloudAI has all the information to store the test run.

gen_exec_command() → str

Generate the execution command for a test based on the given parameters.

Returns:

The generated execution command.

Return type:

str

__init__(

system: System,

test_run: TestRun,

) → None

cleanup_job_artifacts() → None

Best-effort cleanup hook run after the job has fully completed.

Report Generation Strategy

class cloudai.workloads.aiconfig.report_generation_strategy.AiconfiguratorReportGenerationStrategy(

system: System,

tr: TestRun,

)

Bases: ReportGenerationStrategy

Generate metrics from Aiconfigurator predictor outputs.

__init__(

system: System,

tr: TestRun,

) → None