> This page is for version Release 2.2.
> For other versions, use one of these documentation indexes:
> - Release 3.0 (default): https://docs.nvidia.com/isaaclab/3.0/llms.txt
> - Release 2.3: https://docs.nvidia.com/isaaclab/2.3/llms.txt
> - Release 2.2: https://docs.nvidia.com/isaaclab/2.2/llms.txt
> - Release 2.1: https://docs.nvidia.com/isaaclab/2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/isaaclab/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/isaaclab/_mcp/server.

# Reinforcement Learning Library Comparison

In this section, we provide an overview of the supported reinforcement learning libraries in Isaac Lab,
along with performance benchmarks across the libraries.

The supported libraries are:

* [SKRL](https://skrl.readthedocs.io)
* [RSL-RL](https://github.com/leggedrobotics/rsl_rl)
* [RL-Games](https://github.com/Denys88/rl_games)
* [Stable-Baselines3](https://stable-baselines3.readthedocs.io/en/master/index.html)

## Feature Comparison

| Feature                         | RL-Games        | RSL RL            | SKRL                                                            | Stable Baselines3                                                                                       |
| ------------------------------- | --------------- | ----------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| Algorithms Included             | PPO, SAC, A2C   | PPO, Distillation | [Extensive List](https://skrl.readthedocs.io/en/latest/#agents) | [Extensive List](https://github.com/DLR-RM/stable-baselines3?tab=readme-ov-file#implemented-algorithms) |
| Vectorized Training             | Yes             | Yes               | Yes                                                             | No                                                                                                      |
| Distributed Training            | Yes             | Yes               | Yes                                                             | No                                                                                                      |
| ML Frameworks Supported         | PyTorch         | PyTorch           | PyTorch, JAX                                                    | PyTorch                                                                                                 |
| Multi-Agent Support             | PPO             | PPO               | PPO + Multi-Agent algorithms                                    | External projects support                                                                               |
| Documentation                   | Low             | Low               | Comprehensive                                                   | Extensive                                                                                               |
| Community Support               | Small Community | Small Community   | Small Community                                                 | Large Community                                                                                         |
| Available Examples in Isaac Lab | Large           | Large             | Large                                                           | Small                                                                                                   |

## Training Performance

We performed training with each RL library on the same `Isaac-Humanoid-v0` environment
with `--headless` on a single RTX PRO 6000 GPU using 4096 environments
and logged the total training time for 65.5M steps for each RL library.

| RL Library        | Time in seconds |
| ----------------- | --------------- |
| RL-Games          | 207             |
| SKRL              | 208             |
| RSL RL            | 199             |
| Stable-Baselines3 | 322             |