OSU
This workload (test_template_name is OSUBench) allows you to execute OSU Micro Benchmarks within the CloudAI framework.
Usage Example
Test example:
Test Scenario example:
Test-in-Scenario example:
Comparison Reports
The OSU comparison v2 report labels the X axis with the measured message sizes in bytes. Sizes are displayed at equally spaced positions, including zero-byte messages, rather than using default logarithmic ticks. When runs measure different size ranges, charts and tables use the sorted union of measured sizes. Missing measurements appear as chart gaps and n/a table cells.
API Documentation
Command Arguments
class cloudai.workloads.osu_bench.osu_bench.OSUBenchCmdArgs(
*,
docker_image_url: str,
benchmarks_dir: str,
benchmark: str | List[str],
message_size: str | List[str] | None = None,
iterations: int | None = None,
warmup: int | None = None,
mem_limit: int | None = None,
full: bool = True,
**extra_data: Any,
)
Bases: CmdArgs
Command line arguments for a OSU Benchmark test.
field docker_image_url: str [Required]**
URL of the Docker image to use for the test.
field benchmarks_dir: str [Required]**
Directory with the OSU Benchmark binaries inside the container.
field benchmark: str | List[str] [Required]**
Name of the benchmark to run.
field message_size: str | List[str] | None = None**
Message size for the benchmark.
Examples:
field iterations: int | None = None**
Number of iterations for the benchmark.
field warmup: int | None = None**
Number of warmup iterations to skip before timing.
field mem_limit: int | None = None**
Per-process maximum memory consumption in bytes.
field full: bool = True**
Print full format listing of results.
Test Definition
class cloudai.workloads.osu_bench.osu_bench.OSUBenchTestDefinition(*, name: str, description: str, test_template_name: str, cmd_args: ~cloudai.workloads.osu_bench.osu_bench.OSUBenchCmdArgs, dse_excluded_args: list[str] = <factory>, extra_env_vars: dict[str, str | ~typing.List[str]] = {}, extra_cmd_args: dict[str, str] = {}, extra_container_mounts: list[str] = [], git_repos: list[~cloudai._core.installables.git_repo.GitRepo] = [], nsys: ~cloudai.models.workload.NsysConfiguration | None = None, predictor: ~cloudai.models.workload.PredictorConfig | None = None, training_report: ~cloudai.models.workload.TrainingReportConfig | None = None, agent: str = ‘grid_search’, agent_steps: int = 1, agent_metrics: list[str] = [‘default’], agent_reward_function: str = ‘inverse’, agent_config: dict[str, ~typing.Any] | None = None, env_params: dict[str, ~cloudai.configurator.env_params.EnvParamSpec] = <factory>)
Bases: TestDefinition
Test definition for OSU Benchmark test.
property is_domain_randomization_enabled: bool**
at least one env_params annotation.
Type:
Whether the config declares domain randomization
is_dse_excluded_arg(path: str) → bool
Return whether a dot-separated cmd_args path should be ignored by DSE.
is_env_sampled(cmd_args_path: str) → bool
Whether a cmd_args field is env-sampled (env draws it per trial, not the agent).
metric_observations(
system: System,
tr: TestRun,
) → list[MetricObservation]
Return canonical metric observations produced by this test run, if supported.
validator validate_env_params » * all fields*
Validate env_params annotations against cmd_args.
env_params is an annotation: each key names a cmd_args field whose value is the candidate set (the single source of truth), and the entry carries only how to sample. So each key must name a real cmd_args field whose value is a candidate list; a scalar is already fixed, so annotating it is a meaningless label and is rejected here. When weights are declared, the list needs >= 2 values and the weights must align 1:1 with it. Sampling, persistence, the per-trial cmd_args overlay, and the cache key all live in CloudAIGymEnv; keeping this shape check in core lets the overlay stay agent- and workload-agnostic rather than re-implemented per workload.