> This page is for version Release 2.3.
> For other versions, use one of these documentation indexes:
> - Release 3.0 (default): https://docs.nvidia.com/isaaclab/3.0/llms.txt
> - Release 2.3: https://docs.nvidia.com/isaaclab/2.3/llms.txt
> - Release 2.2: https://docs.nvidia.com/isaaclab/2.2/llms.txt
> - Release 2.1: https://docs.nvidia.com/isaaclab/2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/isaaclab/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/isaaclab/_mcp/server.

# Reinforcement Learning Scripts

We provide wrappers to different reinforcement libraries. These wrappers convert the data
from the environments into the respective libraries function argument and return types.

## RL-Games

> **Warning**
>
> When using RL-Games with the Ray workflow for distributed training or hyperparameter tuning,
> please be aware that due to security risks associated with Ray, this workflow is not intended
> for use outside of a strictly controlled network environment.

* Training an agent with
  [RL-Games](https://github.com/Denys88/rl_games) on `Isaac-Ant-v0`:

#### Linux

```bash
# install python module (for rl-games)
./isaaclab.sh -i rl_games
# run script for training
./isaaclab.sh -p scripts/reinforcement_learning/rl_games/train.py --task Isaac-Ant-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth
# run script for playing a pre-trained checkpoint with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py --task Isaac-Ant-v0 --num_envs 32 --use_pretrained_checkpoint
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py --task Isaac-Ant-v0 --headless --video --video_length 200
```

#### Windows

```batch
:: install python module (for rl-games)
isaaclab.bat -i rl_games
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\rl_games\train.py --task Isaac-Ant-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rl_games\play.py --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rl_games\play.py --task Isaac-Ant-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p scripts\reinforcement_learning\rl_games\play.py --task Isaac-Ant-v0 --headless --video --video_length 200
```

## RSL-RL

* Training an agent with
  [RSL-RL](https://github.com/leggedrobotics/rsl_rl) on `Isaac-Reach-Franka-v0`:

#### Linux

```bash
# install python module (for rsl-rl)
./isaaclab.sh -i rsl_rl
# run script for training
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Reach-Franka-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint /PATH/TO/model.pt
# run script for playing a pre-trained checkpoint with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
```

#### Windows

```batch
:: install python module (for rsl-rl)
isaaclab.bat -i rsl_rl
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\train.py --task Isaac-Reach-Franka-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint /PATH/TO/model.pt
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
```

* Training and distilling an agent with
  [RSL-RL](https://github.com/leggedrobotics/rsl_rl) on `Isaac-Velocity-Flat-Anymal-D-v0`:

#### Linux

```bash
# install python module (for rsl-rl)
./isaaclab.sh -i rsl_rl
# run script for rl training of the teacher agent
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Flat-Anymal-D-v0 --headless
# run script for distilling the teacher agent into a student agent
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Flat-Anymal-D-v0 --headless --agent rsl_rl_distillation_cfg_entry_point --load_run teacher_run_folder_name
# run script for playing the student with 64 environments
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Velocity-Flat-Anymal-D-v0 --num_envs 64 --agent rsl_rl_distillation_cfg_entry_point
```

#### Windows

```batch
:: install python module (for rsl-rl)
isaaclab.bat -i rsl_rl
:: run script for rl training of the teacher agent
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\train.py --task Isaac-Velocity-Flat-Anymal-D-v0 --headless
:: run script for distilling the teacher agent into a student agent
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\train.py --task Isaac-Velocity-Flat-Anymal-D-v0 --headless --agent rsl_rl_distillation_cfg_entry_point --load_run teacher_run_folder_name
:: run script for playing the student with 64 environments
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Velocity-Flat-Anymal-D-v0 --num_envs 64 --agent rsl_rl_distillation_cfg_entry_point
```

## SKRL

* Training an agent with
  [SKRL](https://skrl.readthedocs.io) on `Isaac-Reach-Franka-v0`:

#### PyTorch

#### \`fa-brands fa-linux\` Linux

```bash
# install python module (for skrl)
./isaaclab.sh -i skrl
# run script for training
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Reach-Franka-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/model.pt
# run script for playing a pre-trained checkpoint with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
```

#### \`fa-brands fa-windows\` Windows

```batch
:: install python module (for skrl)
isaaclab.bat -i skrl
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\skrl\train.py --task Isaac-Reach-Franka-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/model.pt
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
```

#### JAX

> **Warning**
>
> It is recommended to [install JAX](https://jax.readthedocs.io/en/latest/installation.html) manually before proceeding to install skrl and its dependencies, as JAX installs its CPU version by default.
> Visit the **skrl** [installation](https://skrl.readthedocs.io/en/latest/intro/installation.html) page for more details.
> Note that JAX GPU support is only available on Linux.
>
> JAX 0.6.0 or higher (built on CuDNN v9.8) is incompatible with Isaac Lab's PyTorch 2.7 (built on CuDNN v9.7), and therefore not supported.
> To install a compatible version of JAX for CUDA 12 use `pip install "jax[cuda12]<0.6.0" "flax<0.10.7"`, for example.

```bash
# install python module (for skrl)
./isaaclab.sh -i skrl
# install jax<0.6.0 for torch 2.7
./isaaclab.sh -p -m pip install "jax[cuda12]<0.6.0" "flax<0.10.7"
# install skrl dependencies for JAX
./isaaclab.sh -p -m pip install skrl["jax"]
# run script for training
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax
# run script for playing with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32  --ml_framework jax --checkpoint /PATH/TO/model.pt
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax --video --video_length 200
```

* Training the multi-agent environment `Isaac-Shadow-Hand-Over-Direct-v0` with skrl:

#### Linux

```bash
# install python module (for skrl)
./isaaclab.sh -i skrl
# run script for training with the MAPPO algorithm (IPPO is also supported)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Shadow-Hand-Over-Direct-v0 --headless --algorithm MAPPO
# run script for playing with 32 environments with the MAPPO algorithm (IPPO is also supported)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Shadow-Hand-Over-Direct-v0 --num_envs 32 --algorithm MAPPO --checkpoint /PATH/TO/model.pt
```

#### Windows

```batch
:: install python module (for skrl)
isaaclab.bat -i skrl
:: run script for training with the MAPPO algorithm (IPPO is also supported)
isaaclab.bat -p scripts\reinforcement_learning\skrl\train.py --task Isaac-Shadow-Hand-Over-Direct-v0 --headless --algorithm MAPPO
:: run script for playing with 32 environments with the MAPPO algorithm (IPPO is also supported)
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Shadow-Hand-Over-Direct-v0 --num_envs 32 --algorithm MAPPO --checkpoint /PATH/TO/model.pt
```

## Stable-Baselines3

* Training an agent with
  [Stable-Baselines3](https://stable-baselines3.readthedocs.io/en/master/index.html)
  on `Isaac-Velocity-Flat-Unitree-A1-v0`:

#### Linux

```bash
# install python module (for stable-baselines3)
./isaaclab.sh -i sb3
# run script for training
./isaaclab.sh -p scripts/reinforcement_learning/sb3/train.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/sb3/play.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --num_envs 32 --checkpoint /PATH/TO/model.zip
# run script for playing a pre-trained checkpoint with 32 environments
./isaaclab.sh -p scripts/reinforcement_learning/sb3/play.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --num_envs 32 --use_pretrained_checkpoint
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p scripts/reinforcement_learning/sb3/play.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --headless --video --video_length 200
```

#### Windows

```batch
:: install python module (for stable-baselines3)
isaaclab.bat -i sb3
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\sb3\train.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\sb3\play.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --num_envs 32 --checkpoint /PATH/TO/model.zip
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\sb3\play.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p scripts\reinforcement_learning\sb3\play.py --task Isaac-Velocity-Flat-Unitree-A1-v0 --headless --video --video_length 200
```

All the scripts above log the training progress to [Tensorboard](https://www.tensorflow.org/tensorboard) in the `logs` directory in the root of
the repository. The logs directory follows the pattern `logs/<library>/<task>/<date-time>`, where `<library>`
is the name of the learning framework, `<task>` is the task name, and `<date-time>` is the timestamp at
which the training script was executed.

To view the logs, run:

#### Linux

```bash
# execute from the root directory of the repository
./isaaclab.sh -p -m tensorboard.main --logdir=logs
```

#### Windows

```batch
:: execute from the root directory of the repository
isaaclab.bat -p -m tensorboard.main --logdir=logs
```