Skip to navigation

Nemo Run

View as Markdown

This workload (test_template_name is NemoRun) executes NeMo training and fine-tuning tasks using the NeMo Run framework.

Usage Examples

Test TOML example:

name = "my_nemo_test"
description = "Example NeMo Run test"
test_template_name = "NemoRun"
[cmd_args]
recipe = "llama3_8b"
task = "pretrain"

Test Scenario example:

name = "nemo-run-test"
[[Tests]]
id = "nemo.1"
num_nodes = 4
time_limit = "02:00:00"
test_name = "my_nemo_test"

Test-in-Scenario example:

name = "nemo-run-test"
[[Tests]]
id = "nemo.1"
num_nodes = 4
time_limit = "02:00:00"
name = "my_nemo_test"
description = "Example NeMo Run test"
test_template_name = "NemoRun"
[Tests.cmd_args]
recipe = "llama3_8b"
task = "pretrain"

API Documentation

Command Arguments

NeMoRunCmdArgs

NeMoRun test command arguments.

FieldTypeDefaultDescription
docker_image_urlstrRequired
taskstrRequired
recipe_namestrRequired
num_layersint | NoneNone
trainercloudai.workloads.nemo_run.nemo_run.TrainerFactory: Trainer
logcloudai.workloads.nemo_run.nemo_run.LogFactory: Log
datacloudai.workloads.nemo_run.nemo_run.DataFactory: Data
optimcloudai.workloads.nemo_run.nemo_run.OptimFactory: Optim

Test Definition

NeMoRunTestDefinition

NeMoRun test definition.

FieldTypeDefaultDescription
namestrRequired
descriptionstrRequired
test_template_namestrRequired
cmd_argscloudai.workloads.nemo_run.nemo_run.NeMoRunCmdArgsRequired
dse_excluded_argslist[str]Factory: list
extra_env_varsdict[str, str | typing.List[str]]{}
extra_cmd_argsdict[str, str]{}
extra_container_mountslist[str][]
git_reposlist[cloudai._core.installables.git_repo.GitRepo][]
nsyscloudai.models.workload.NsysConfiguration | NoneNone
predictorcloudai.models.workload.PredictorConfig | NoneNone
training_reportcloudai.models.workload.TrainingReportConfig | NoneNone
agentstr'grid_search'
agent_stepsint1
agent_metricslist[str]['default']
agent_reward_functionstr'inverse'
agent_configdict[str, Any] | NoneNoneAgent configuration.
env_paramsdict[str, cloudai.configurator.env_params.EnvParamSpec]Factory: dictEnvironment parameters sampled by the env per trial. Sibling to cmd_args; not part of the agent’s action space. CloudAIGymEnv samples, persists them in the trajectory output, and includes them in the trajectory cache key.
scriptcloudai._core.installables.file.FileFile(src=PosixPath('src/cloudai/workloads/nemo_run/cloudai_nemorun.py'))