Skip to navigation

AIConfigurator

View as Markdown

AIConfigurator workload (test_template_name is Aiconfigurator) runs the AIConfigurator predictor using the installed aiconfigurator Python package. It is a Standalone workload (no Slurm/Kubernetes/RunAI required).

Outputs

Each test run produces:

  • report.json: Predictor output (JSON dict of metrics and metadata)
  • stdout.txt / stderr.txt: Predictor logs
  • run_simple_predictor.sh: Repro script containing the exact executed command (useful for debugging)

Usage Example

Test TOML example (Disaggregated mode):

name = "aiconfigurator_disagg_demo"
description = "Example AIConfigurator disaggregated predictor"
test_template_name = "Aiconfigurator"
[cmd_args]
model_name = "LLAMA3.1_70B"
system = "h200_sxm"
backend = "trtllm"
version = "0.20.0"
isl = 4000
osl = 500
[cmd_args.disagg]
p_tp = 1
p_pp = 1
p_dp = 1
p_bs = 1
p_workers = 1
d_tp = 1
d_pp = 1
d_dp = 1
d_bs = 8
d_workers = 2
prefill_correction_scale = 1.0
decode_correction_scale = 1.0

Test TOML example (Aggregated/IFB mode):

name = "aiconfigurator_agg_demo"
description = "Example AIConfigurator aggregated predictor"
test_template_name = "Aiconfigurator"
[cmd_args]
model_name = "LLAMA3.1_70B"
system = "h200_sxm"
backend = "trtllm"
version = "0.20.0"
isl = 4000
osl = 500
[cmd_args.agg]
batch_size = 8
ctx_tokens = 16
tp = 1
pp = 1
dp = 1

Running

uv run cloudai run --system-config conf/common/system/standalone_system.toml \
--tests-dir conf/experimental/aiconfigurator/test \
--test-scenario conf/experimental/aiconfigurator/test_scenario/aiconfigurator_disagg.toml

API Documentation

Command Arguments

AiconfiguratorCmdArgs

Command arguments for Aiconfigurator workload with nested agg/disagg configs.

FieldTypeDefaultDescription
requirementsstr'aiconfigurator~=0.5.0'Space-separated Python requirements installed into the Aiconfigurator environment.
model_namestrRequired
systemstrRequired
backendstr'trtllm'
versionstr'0.20.0'
islint | List[int]Required
oslint | List[int]Required
aggcloudai.workloads.aiconfig.aiconfigurator.Agg | NoneNone
disaggcloudai.workloads.aiconfig.aiconfigurator.Disagg | NoneNone

Test Definition

AiconfiguratorTestDefinition

Test object for running Aiconfigurator predictor as a workload.

FieldTypeDefaultDescription
namestrRequired
descriptionstrRequired
test_template_namestrRequired
cmd_argscloudai.workloads.aiconfig.aiconfigurator.AiconfiguratorCmdArgsRequired
dse_excluded_argslist[str]Factory: list
extra_env_varsdict[str, str | typing.List[str]]{}
extra_cmd_argsdict[str, str]{}
extra_container_mountslist[str][]
git_reposlist[cloudai._core.installables.git_repo.GitRepo][]
nsyscloudai.models.workload.NsysConfiguration | NoneNone
predictorcloudai.models.workload.PredictorConfig | NoneNone
training_reportcloudai.models.workload.TrainingReportConfig | NoneNone
agentstr'grid_search'
agent_stepsint1
agent_metricslist[str]['default']
agent_reward_functionstr'inverse'
agent_configdict[str, Any] | NoneNoneAgent configuration.
env_paramsdict[str, cloudai.configurator.env_params.EnvParamSpec]Factory: dictEnvironment parameters sampled by the env per trial. Sibling to cmd_args; not part of the agent’s action space. CloudAIGymEnv samples, persists them in the trajectory output, and includes them in the trajectory cache key.

Command Generation Strategy (Standalone)

cloudai.workloads.aiconfig.standalone_command_gen_strategy.AiconfiguratorStandaloneCommandGenStrategy

Generate a standalone command that invokes the Aiconfigurator predictor and writes JSON output.

cleanup_job_artifacts

cleanup_job_artifacts(self) -> None

Best-effort cleanup hook run after the job has fully completed.

Report Generation Strategy

cloudai.workloads.aiconfig.report_generation_strategy.AiconfiguratorReportGenerationStrategy

Generate metrics from Aiconfigurator predictor outputs.