> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.orchestration.executors.resume_script

The auto-resume prologue: a self-chaining Slurm job.

Rendered at the top of a benchmark's sbatch script, ahead of every
service/driver command, when `BenchmarkRunConfig.resumable` is set. Each job
in the chain inspects the PREVIOUS job (passed as `$1`) via `sacct`, decides
whether to continue, and unconditionally queues its own successor with an
`afternotok` dependency before doing any real work. A job that COMPLETEs
leaves its queued successor with a dependency that can never be satisfied, so
Slurm cancels it on its own -- the chain needs no explicit "we're done" step
on the success path.

## Module Contents

### Functions

| Name                                                                                               | Description                                                        |
| -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| [`_walltime_to_seconds`](#nemo_gym-orchestration-executors-resume_script-_walltime_to_seconds)     | Convert a Slurm-style duration string to seconds.                  |
| [`render_resume_prologue`](#nemo_gym-orchestration-executors-resume_script-render_resume_prologue) | Render the auto-resume prologue for one benchmark's sbatch script. |

### Data

[`_JOB_SCRIPT_NAME`](#nemo_gym-orchestration-executors-resume_script-_JOB_SCRIPT_NAME)

[`_MAX_WALLTIME_CHECK_TEMPLATE`](#nemo_gym-orchestration-executors-resume_script-_MAX_WALLTIME_CHECK_TEMPLATE)

[`_PROLOGUE_TEMPLATE`](#nemo_gym-orchestration-executors-resume_script-_PROLOGUE_TEMPLATE)

### API

```python
nemo_gym.orchestration.executors.resume_script._walltime_to_seconds(
    duration: str
) -> int
```

Convert a Slurm-style duration string to seconds.

Covers the subset of Slurm's `--time` grammar that `max_walltime` accepts:
`MM`, `MM:SS`, `HH:MM:SS`, `D-HH`, `D-HH:MM`, `D-HH:MM:SS` -- the same
spellings Slurm itself takes, so a value can be copied straight from
`compute.walltime`.

```python
nemo_gym.orchestration.executors.resume_script.render_resume_prologue(
    resume: nemo_gym.orchestration.api.ResumeConfig
) -> str
```

Render the auto-resume prologue for one benchmark's sbatch script.

```python
nemo_gym.orchestration.executors.resume_script._JOB_SCRIPT_NAME = 'job.sh'
```

```python
nemo_gym.orchestration.executors.resume_script._MAX_WALLTIME_CHECK_TEMPLATE = '        if (( _gym_accumulated >= {seconds} )); then\n            echo "Accumul...
```

```python
nemo_gym.orchestration.executors.resume_script._PROLOGUE_TEMPLATE = '# --- Auto-resume chain ---\nOUTPUT_DIR="$PWD"\n_this_script="$OUTPUT_DIR/{job_...
```