Run on Any Cloud with SkyPilot

In this guide, you will learn how to launch NeMo AutoModel training jobs with SkyPilot. SkyPilot can target public clouds such as AWS, GCP, Azure, and Lambda, and it can also submit jobs to Kubernetes clusters. For a beginner-friendly Kubernetes walkthrough, see SkyPilot + Kubernetes tutorial. For on-premises cluster usage without SkyPilot, see Run on a Cluster (Slurm). For single-node workstation usage, see Run on Your Local Workstation.

SkyPilot is an open-source framework that abstracts cloud infrastructure so you can train on whichever cloud is cheapest or most available at launch time — including automatic spot-instance handling for significant cost savings.

Before You Begin

Complete the following setup steps before launching your first AutoModel job on a cloud provider.

1. Create and activate a virtual environment, then install SkyPilot with the connector for your target infrastructure. Choose one connector install command:

$
uv venv
$
source .venv/bin/activate
$
$
uv pip install "skypilot[gcp]==0.8.1"         # Google Cloud
$
# uv pip install "skypilot[aws]==0.8.1"         # Amazon Web Services
$
# uv pip install "skypilot[lambda]==0.8.1"      # Lambda Cloud
$
# uv pip install "skypilot[kubernetes]==0.8.1"  # Any Kubernetes cluster

NeMo AutoModel’s current launcher passes detach_run and stream_logs to sky.launch(). These SDK arguments are supported by SkyPilot 0.8.1 and were removed in later releases, so keep this version pin until the launcher is migrated to the newer SkyPilot API. Azure is not currently usable with this integration: skypilot[azure]==0.8.1 no longer resolves because its azure-cli dependency requires unavailable azure-batch versions.

2. Configure access for your target infrastructure, then verify:

$
sky check

You should see at least one cloud listed as OK.

3. Set required environment variables:

$
export HF_TOKEN=hf_...          # Required for gated models (e.g. Llama)
$
export WANDB_API_KEY=...        # Optional: Weights & Biases logging

Quickstart

Add a skypilot: section to any existing config YAML, then run the same automodel command you already know:

$
automodel your_config_with_skypilot.yaml

The CLI detects the skypilot: key, strips it from the training config, uploads the code and config to a cloud VM, and launches training — all in one command.

Configuration Reference

Below is an annotated example for fine-tuning Llama-3.2-1B on SQuAD on a GCP spot T4. The checked-in examples/llm_finetune/llama3_2/llama3_2_1b_squad_skypilot.yaml is a configuration reference, but its commented custom setup: example predates the replacement semantics below. If you enable setup:, use the complete override shown here.

1
# ── SkyPilot launcher section ─────────────────────────────────────────────
2
# Removed before the training config reaches the remote VM.
3
skypilot:
4
  cloud: gcp                  # aws | gcp | lambda | kubernetes
5
  accelerators: T4:1          # GPU type:count per node, e.g. A100:8
6
  use_spot: true              # ~80 % cost reduction vs on-demand
7
  disk_size: 100              # Remote VM disk size in GB
8
  num_nodes: 1                # Increase for multi-node distributed training
9
  region: us-central1         # Optional — SkyPilot picks cheapest if omitted
10
  job_name: llama3_2_finetune # Also used as the SkyPilot cluster name
11
12
  # Use env-var placeholders so secrets are never stored in YAML
13
  hf_token: ${HF_TOKEN}
14
  # wandb_key: ${WANDB_API_KEY}
15
16
  # Optional: replace the default remote setup command.
17
  # Include the editable install and any extra dependencies in the override:
18
  # setup: |
19
  #   cd ~/sky_workdir && pip install -e . --quiet
20
  #   pip install some-extra-dependency
21
22
  # Optional: override the default output directory (default: ./skypilot_jobs)
23
  # job_dir: /path/to/skypilot/jobs
24
25
# ── Training config (forwarded to the VM unchanged) ───────────────────────
26
step_scheduler:
27
  global_batch_size: 64
28
  local_batch_size: 8
29
  num_epochs: 1
30
31
model:
32
  _target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
33
  pretrained_model_name_or_path: meta-llama/Llama-3.2-1B
34
35
# ... rest of your training config ...

All skypilot: Fields

Cloud Examples

AWS — On-Demand A10G

1
skypilot:
2
  cloud: aws
3
  accelerators: A10G:1
4
  use_spot: false
5
  region: us-east-1
6
  job_name: llm_aws_finetune
7
  hf_token: ${HF_TOKEN}

GCP — Spot V100, 8 GPUs (Single Node)

1
skypilot:
2
  cloud: gcp
3
  accelerators: V100:8
4
  use_spot: true
5
  region: us-west1
6
  job_name: llm_gcp_v100_8gpu
7
  hf_token: ${HF_TOKEN}

Multi-Node Distributed Training (2 x 8 x A100)

1
skypilot:
2
  cloud: gcp
3
  accelerators: A100:8
4
  num_nodes: 2
5
  use_spot: false
6
  job_name: llm_multinode_a100
7
  hf_token: ${HF_TOKEN}

For multi-node jobs, the launcher builds the torchrun rendezvous arguments from the $SKYPILOT_NODE_RANK, $SKYPILOT_NUM_NODES, and $SKYPILOT_NODE_IPS environment variables provided by SkyPilot.

Monitor and Manage Jobs

After submitting, use standard SkyPilot commands:

$
CLUSTER_NAME="llm_gcp_v100_8gpu"
$
JOB_ID="1"  # Find job IDs with: sky queue "$CLUSTER_NAME"
$
$
sky status                                      # List running clusters and their status
$
sky logs "$CLUSTER_NAME" "$JOB_ID"              # Stream training logs
$
sky logs --sync-down "$CLUSTER_NAME" "$JOB_ID"  # Download training logs
$
ssh "$CLUSTER_NAME"                             # SSH into the VM for debugging
$
sky cancel "$CLUSTER_NAME" "$JOB_ID"            # Cancel a running job
$
sky down "$CLUSTER_NAME"                        # Terminate the cluster and stop billing

The launcher’s local job_dir contains only the timestamped job_config.yaml snapshot. sky logs --sync-down downloads SkyPilot’s logs to the local ~/sky_logs tree, not to job_dir.

How It Works

1. The automodel CLI detects the skypilot: key in the YAML and calls SkyPilotLauncher().launch().
2. The training config (with skypilot: removed) is written to a local skypilot_jobs/<timestamp>/job_config.yaml.
3. A sky.Task is created with:
   • workdir — the current directory synced to ~/sky_workdir on the remote VM.
   • file_mounts — the job config uploaded to /tmp/automodel_job_config.yaml.
   • setup — defaults to cd ~/sky_workdir && pip install -e . --quiet. A non-empty custom setup: value replaces this command rather than being appended, so the override must include any required package installation.
   • run — a torchrun command pointing at the recipe script and config.
4. sky.launch() provisions the VM, runs setup, then executes training. With detach_run=True, it returns after job submission without streaming execution logs; use sky logs to follow progress.

Customize Configuration

Override any training parameter from the command line, same as with local runs:

$
automodel config_with_skypilot.yaml \
>
  --model.pretrained_model_name_or_path meta-llama/Llama-3.2-3B

Kubernetes Users

If you want to run on a Kubernetes cluster, use cloud: kubernetes and follow the dedicated SkyPilot + Kubernetes tutorial. That guide includes:

• a copy-paste single-node config
• a two-node example
• sample sky and kubectl output to help you sanity-check your setup
• a short troubleshooting section for common first-run issues

When to Use SkyPilot vs. Slurm