Configure Models

Configure your model with the inference backend of your choice.

NeMo Gym uses the Responses API as its native schema because it natively represents tool calls, multi-turn conversations, and structured outputs. NeMo Gym provides middleware to automatically convert other protocols — Chat Completions and Anthropic Messages — into Responses format.

For training, use a backend that returns token IDs and log probabilities (marked with training below).

Available Backends

Embeddings

Every Model Server exposes an OpenAI-compatible POST /v1/embeddings route, so memory- and retrieval-based agents can route embedding calls through Gym instead of calling a provider directly. The OpenAI and Inference Providers backends implement it as a passthrough to the provider’s /v1/embeddings (this covers OpenAI, NVIDIA’s hosted endpoint, and any other OpenAI-compatible provider); backends without an embeddings model return 501.

The passthrough merges the server’s extra_body under the request (a field the request sets wins), replaces model with the configured one, and returns the provider’s JSON unchanged, so encoding_format: base64 and providers that omit optional fields such as usage both work.

Embeddings calls are not recorded by model-call capture, which covers only /v1/responses, /v1/chat/completions, and /v1/messages. A call under a rollout’s /ng-rollout/<rollout_id> prefix still reaches the embeddings route, and it does not mark a token-capture rollout incomplete, because the route is declared non-generating.

An embedding model is usually distinct from the policy model, so wire it as a separate Model Server instance (referenced via ModelServerRef), following the existing one-server-one-model pattern.

Observe model calls

Protocols

Every backend above also exposes these conversion dialects on the same model server: