> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# Development Setup

This guide covers everything you need to set up your development environment and submit contributions to NeMo Gym.

## Quick Start

```bash
# Clone and set up development environment
git clone git@github.com:NVIDIA-NeMo/Gym.git
cd Gym
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv venv --python 3.12
source .venv/bin/activate
uv sync --extra dev

# Install pre-commit hooks (required for contributors)
pre-commit install
```

## Development Commands

**Run NeMo Gym Tests**:

```bash
gym dev test                                   # Run all NeMo Gym core tests
gym env test                                   # Run all server tests
gym env test --resources-server example_single_tool_call    # Test a single resources server
```

**View Test Coverage**:

```bash
coverage html                                  # Generate HTML coverage report
```

**Configuration Debugging**:

```bash
gym env resolve                                # Dump config as NeMo Gym sees it
```

---

## CI/CD Requirements

All contributions must pass these automated checks:

**Required Checks**:

* **Unit Tests**: All existing tests must pass
* **Build Docs**: Documentation must build without errors
* **Copyright Check**: All files must have proper copyright headers
* **DCO Signing**: All commits must be signed off
* **Pre-commit Hooks**: Code formatting and linting

**Test Requirements**:

* At least one test per server you contribute
* Tests must run using `gym env test --resources-server your_server`
* Use pytest for async testing patterns

### Build Docs CI Failures

If the build-docs check fails:

1. **Test documentation locally**: Follow the [Fern docs quickstart](https://github.com/NVIDIA-NeMo/Gym/blob/main/fern/README.md).
   * For local documentation previews, ensure Node.js 22+ is installed.
   * From the repository root, run `make docs-login` once to provision and authenticate Fern.
   * From the repository root, run `make docs` to generate the API reference and start the local preview.
   * After the initial generation, prose-only changes can use `cd fern && npm run dev`.

2. **Common issues**:
   * Missing docstrings in public functions
   * Broken markdown links in README or tutorials
   * Invalid MDX syntax

### Copyright Header Errors

**Error**: "Found files with missing copyright"

**Solution**: Add this header to all new Python files:

```python
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

---

## DCO and Commit Signing

All NeMo Gym contributions require **commit signing** and **DCO sign-off**.

### Quick Setup

**1. Configure Git Signing (Required)**

```bash
# Set your identity (use your GitHub email)
git config --global user.name "Your Name"
git config --global user.email "your@github-email.com"

# Enable commit signing
git config --global commit.gpgsign true

# Use SSH signing (recommended - simpler than GPG)
git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/id_ed25519.pub
```

**2. Add Signing Key to GitHub**

1. Go to [GitHub Settings → SSH and GPG keys](https://github.com/settings/keys)
2. Under "SSH keys" section, find "Signing keys" subsection
3. Click "New SSH key" → Set type to "Signing Key"
4. Copy your public key: `cat ~/.ssh/id_ed25519.pub`
5. Paste the **entire output** (including `ssh-ed25519` prefix)

**3. Test Your Setup**

```bash
# Test commit signing
git commit --allow-empty -s -m "Test commit signing"

# Verify it's signed (should show "S" for signed)
git log --show-signature -1
```

### IDE Integration

**VSCode Setup**

Create/update `.vscode/settings.json` in your project:

```json
{
    "git.enableCommitSigning": true,
    "git.alwaysSignOff": true
}
```

**Other IDEs**: Enable "Git commit signing" and "Always sign-off" in your Git settings.

### Making Signed Commits

**Every commit must be signed off** using the `-s` flag:

```bash
# Standard workflow
git add .
git commit -s -m "Your commit message"

# The -s flag adds this line automatically:
# Signed-off-by: Your Name [your@email.com](mailto:your@email.com)
```

### Troubleshooting

**Problem**: `error: gpg failed to sign the data`

```bash
# Check if key exists and is correct format
ls -la ~/.ssh/id_ed25519.pub

# Verify git config
git config --list | grep -E "(user|gpg|signingkey)"

# Test SSH key
ssh-add -l
```

**Problem**: "Signing key not found on GitHub"

* Ensure you copied the complete public key including the `ssh-ed25519` prefix
* Add the key as a **Signing Key** (not Authentication Key) on GitHub
* Wait a few minutes for GitHub to process the new key

**Problem**: IDE not signing commits

* Restart your IDE after configuring Git
* Check IDE Git settings match the command line configuration
* Try committing through the command line first to validate setup

**Problem**: "DCO sign-off missing"

**If you have not pushed yet:**

```bash
# Fix the last commit
git commit --amend -s --no-edit

# Fix multiple local commits
git rebase --signoff HEAD~3  # Adjust number as needed
```

**If you already pushed to your PR branch:**

Force-pushing is disallowed on branches in the upstream `NVIDIA-NeMo/Gym` repo. If your PR branch is in a fork and your fork allows force pushes, you can re-sign commits locally and push with `--force-with-lease`; otherwise, push the signed history to a new branch and update the PR.

```bash
# Option 1: Re-sign commits locally, then push to a new branch and update your PR
git rebase --signoff origin/main
git push origin HEAD:your-user/your-feature-dco-fix

# Option 2: Squash into one signed commit on a fresh branch
git checkout -b your-feature-dco-fix origin/main
git merge --squash your-feature-branch
git commit -s -m "Your commit message"
git push -u origin your-feature-dco-fix

# Option 3 (forks only): Re-sign and force-push to the same branch
git rebase --signoff origin/main
git push --force-with-lease
```

**Tip:** Always use `git commit -s` when creating commits to avoid this issue.

### Alternative: GPG Signing

If you prefer GPG over SSH signing:

```bash
# Generate GPG key (if you don't have one)
gpg --full-generate-key

# List keys and copy the key ID
gpg --list-secret-keys --keyid-format=long

# Configure Git to use GPG
git config --global user.signingkey YOUR_GPG_KEY_ID
# Do not set gpg.format (defaults to gpg)

# Add GPG key to GitHub
gpg --armor --export YOUR_GPG_KEY_ID
# Copy output to GitHub Settings → SSH and GPG keys → GPG keys
```

---

## Common Issues

### Testing Issues

**Problem**: Tests fail locally but not in CI

* Check Python version (3.12+ required)
* Ensure all dependencies installed: `uv sync --extra dev`
* Run in clean environment

**Problem**: Async test failures

* Use `pytest-asyncio` for async tests
* Mark async tests with `@pytest.mark.asyncio`
* Ensure proper fixture cleanup

### Pre-commit Hook Failures

**Problem**: Pre-commit hooks fail

```bash
# Fix common issues
pre-commit run --all-files        # Run all hooks manually
pre-commit autoupdate             # Update hook versions
```

**Common fixes**:

* `ruff check --fix .` for linting
* `ruff format .` for formatting
* Add copyright headers to new files

---

## Development Workflow

For a small, self-contained change, open a single PR:

1. **Create a feature branch**: `git checkout -b <username>/<feature>`
2. **Make changes** with tests
3. **Run local checks**: `gym dev test && pre-commit run --all-files`
4. **Commit with signoff**: `git commit -s -S -m "Your message"`
5. **Push and open a PR**: ensure all CI checks pass
6. **Address review feedback** and iterate

### Stacked Pull Requests

For larger changes, NeMo Gym uses [GitHub Stacked PRs](https://github.github.com/gh-stack/): a chain of branches where each PR targets the branch below it, so each layer is reviewed as its own focused diff. See GitHub's [Stacked PRs overview](https://github.github.com/gh-stack/) for the concept and PR UI; the mechanics of the `gh stack` CLI in this repo follow.

#### Prerequisites

Install the [GitHub CLI](https://cli.github.com/) (`gh`) v2.0 or later, the `gh stack` extension, and the agent skill:

```bash
gh extension install github/gh-stack
gh skill install github/gh-stack gh-stack   # vendored under .agents/skills/gh-stack/; this installs it for your agent
```

> **Warning**
>
> `gh stack` requires OAuth — Personal Access Tokens are rejected during the private preview. If `gh auth status` shows a `github_pat_…` token, `submit`/`push`/`sync` fail with exit code 9. Re-authenticate with the web flow:
>
> ```bash
> gh auth login   # GitHub.com → HTTPS → "Login with a web browser"
> ```

A stack is created in the repo you push to, and that repo must have the preview enabled — push your branches to the remote that points at `NVIDIA-NeMo/Gym`. With multiple remotes, set the target once (or pass `--remote <name>` on each command) and enable conflict memory so commands do not prompt:

```bash
git config remote.pushDefault upstream   # the remote pointing at NVIDIA-NeMo/Gym
git config rerere.enabled true
```

#### Create a Stack

Build the stack bottom-up, one logical change per branch, where each branch depends on the one below it. Keep a contribution self-contained — land code together with its tests in a single PR — and stack genuinely dependent follow-ups (for example, documentation) on top.

```bash
# Bottom layer: the contribution (resources server, agent config, and its tests) as one branch.
gh stack init -p <user>/my-env contribution
git add resources_servers/my_env/
git commit -s -m "feat: add my_env environment and tests"

# Next layer: a follow-up that depends on the contribution above.
gh stack add docs
git add fern/versions/latest/pages/environment-tutorials/my-env.mdx
git commit -s -m "docs: document my_env"

# Push every branch and open a draft PR per layer (--auto titles them).
gh stack submit --auto

# Inspect the result (always --json; bare `view` opens a TUI).
gh stack view --json
```

`submit` sets each PR's base to the branch below it and links them as a stack on GitHub. Run every `gh stack` command non-interactively: pass branch names to `init`/`add`/`checkout`, `--auto` to `submit`, and `--json` to `view`. Without these flags the commands prompt or open a TUI and hang.

#### Update a Stack

Make a change on the branch it belongs to, then rebase the layers above it:

```bash
gh stack down              # or: gh stack checkout <branch|pr-number>
git add <files> && git commit -s -m "..."
gh stack rebase --upstack  # replay the layers above onto the change
gh stack push
```

After a PR merges, sync to fast-forward the trunk and rebase the rest of the stack:

```bash
gh stack sync --prune      # fetch, rebase, push, drop merged branches
```

On a rebase conflict (exit code 3), resolve the listed files, `git add` them, then `gh stack rebase --continue` (or `--abort`). Merging is done from the GitHub PR UI; CLI merge is not supported yet.

#### Command Reference

The vendored skill (`.agents/skills/gh-stack/SKILL.md`) carries the full reference, exit codes, and non-interactive rules for agents.

| Task                            | Command                                   |
| ------------------------------- | ----------------------------------------- |
| Start a stack                   | `gh stack init -p <prefix> <branch>`      |
| Add a layer                     | `gh stack add <suffix>`                   |
| Push and open PRs               | `gh stack submit --auto`                  |
| Inspect state                   | `gh stack view --json`                    |
| Navigate                        | `gh stack up` / `down` / `top` / `bottom` |
| Rebase after a lower-layer edit | `gh stack rebase --upstack`               |
| Sync after merges               | `gh stack sync --prune`                   |

See the [gh-stack documentation](https://github.github.com/gh-stack/) for the full feature set.

---

**Questions?** Check existing issues or create a new one for guidance.