Training TutorialsNeMo RL

Gym Configuration

View as Markdown

Before running GRPO training, you need to configure how NeMo RL connects to NeMo Gym. The training config file contains Gym-specific parameters that control data loading, environment interaction, and validation.

Goal: Understand the Gym configuration parameters for RL training.

Time: ~10 minutes (read)

In this section, you will learn:

  1. How to configure data paths for training and validation
  2. How to enable and configure NeMo Gym in NeMo RL
← Previous: About Workplace Assistant

Prerequisites

Configuration File Location

The full training configuration file is located at:

examples/nemo_gym/grpo_workplace_assistant_nemotron_nano_v2_9b.yaml

Gym Configuration Sections

There are two Gym-specific sections in the NeMo RL training config: data and env.

Data Section

1data:
2  train_jsonl_fpath: 3rdparty/Gym-workspace/Gym/data/workplace_assistant/train.jsonl
3  validation_jsonl_fpath: 3rdparty/Gym-workspace/Gym/data/workplace_assistant/validation.jsonl
ParameterDescription
train_jsonl_fpathPath to training dataset (prepared in Setup)
validation_jsonl_fpathPath to validation dataset

Environment Section

1env:
2  should_use_nemo_gym: true
3  nemo_gym:
4    config_paths:
5    - responses_api_models/vllm_model/configs/vllm_model_for_training.yaml
6    - resources_servers/workplace_assistant/configs/workplace_assistant.yaml
7    workplace_assistant_simple_agent:
8      responses_api_agents:
9        simple_agent:
10          max_steps: 6
ParameterDescription
should_use_nemo_gymSet to true to enable Gym
nemo_gymEverything under this key is a Gym config
nemo_gym.config_pathsGym config files: vLLM model config and Workplace Assistant agent/resources config
max_stepsMaximum tool-calling steps per task (6 for Workplace Assistant)

The vllm_model_for_training.yaml config is required for NeMo RL training integration.

Next Steps

With the Gym configuration understood, learn about the GRPO training parameters:

Continue to NeMo RL Configuration →