Configure Models

View as Markdown

Configure your model with the inference backend of your choice.

NeMo Gym uses the Responses API as its native schema because it natively represents tool calls, multi-turn conversations, and structured outputs. NeMo Gym provides middleware to automatically convert other protocols — Chat Completions and Anthropic Messages — into Responses format.

For training, use a backend that returns token IDs and log probabilities (marked with training below).

Available Backends

OpenAI

Use OpenAI models.

cloud
Azure OpenAI

Use models hosted on Azure OpenAI deployments.

cloud
Inference Providers

Use hosted providers like Fireworks, Together.ai, OpenRouter, and more.

cloud
vLLM

Connect to a vLLM server you start and manage yourself.

self-hostedtraining
Local vLLM

Let NeMo Gym launch and manage the vLLM server for you.

self-hostedgym-managedtraining
Local vLLM Proxy

Serve multiple request-time configs (e.g. reasoning on/off) from one Local vLLM deployment.

self-hostedgym-managedtraining
Switchyard

Route each model call across a fleet of models with a Switchyard routing proxy Gym hosts for you.

routergym-managed

Observe model calls

Model-call capture

Record per-rollout model requests, responses, token usage, latency, and correlation evidence.

observability

Protocols

Every backend above also exposes these conversion dialects on the same model server:

Anthropic Messages

Run Anthropic-native harnesses like Claude Code against any Gym model server via POST /v1/messages.

claudedialect