> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# Configure Models

Configure your model with the inference backend of your choice.

NeMo Gym uses the [Responses API](https://developers.openai.com/api/reference/resources/responses/methods/create) as its native schema because it natively represents tool calls, multi-turn conversations, and structured outputs. NeMo Gym provides middleware to automatically convert other protocols, for example, Chat Completions, into Responses format.

For training, use a backend that returns token IDs and log probabilities (marked with

training

below).

## Available Backends

#### [OpenAI](/model-server/openai)

Use OpenAI models.

cloud

#### [Azure OpenAI](/model-server/azure-openai)

Use models hosted on Azure OpenAI deployments.

cloud

#### [Inference Providers](/model-server/inference-providers)

Use hosted providers like Fireworks, Together.ai, OpenRouter, and more.

cloud

#### [vLLM](/model-server/vllm)

Connect to a vLLM server you start and manage yourself.

self-hosted

training

#### [Local vLLM](/model-server/local-vllm)

Let NeMo Gym launch and manage the vLLM server for you.

self-hosted

gym-managed

training

#### [Local vLLM Proxy](/model-server/local-vllm-proxy)

Serve multiple request-time configs (e.g. reasoning on/off) from one Local vLLM deployment.

self-hosted

gym-managed

training

## Observe model calls

#### [Model-call capture](/model-server/model-call-capture)

Record per-rollout model requests, responses, token usage, latency, and correlation evidence.

observability

## Docs

- [OpenAI](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/openai.md): Use OpenAI (or any endpoint that natively serves the Responses API) with no conversion
- [Azure OpenAI](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/azure-openai.md): Use models hosted on Azure OpenAI deployments
- [Inference Providers](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/inference-providers.md): Use hosted inference providers like Fireworks, Together.ai, OpenRouter, and more for eval workloads
- [Model-call capture](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/model-call-capture.md): Optional per-rollout model-call evidence for evaluation and debugging
- [vLLM](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/vllm.md): Wrapper for an existing, external vLLM server
- [Local vLLM](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/local-vllm.md): Gym-managed vLLM server deployment
- [Local vLLM Proxy](https://docs.nvidia.com/nemo/gym/v0.5.1/model-server/local-vllm-proxy.md): Expose one Local vLLM deployment as multiple model servers