RL Training with Unsloth

This tutorial demonstrates how to use Unsloth to fine-tune models with NeMo Gym environments.

Unsloth is a fast, memory-efficient library for fine-tuning large language models. It provides optimized implementations that significantly reduce memory usage and training time, making it possible to fine-tune larger models on consumer hardware.

Prerequisites

• A Google account (for Colab) or a local GPU with 16GB+ VRAM

• Familiarity with NeMo Gym concepts (Quickstart)

---

Getting Started

Follow these interactive notebooks to train models with Unsloth and NeMo Gym:

Sudoku

Multi-Environment Training

Check out Unsloth’s documentation for more details.