Customize Request and Response Policies
Each provider binding defines which fields and values the gateway accepts and which text reaches Rails checks. Use the built-in binding for its documented request and response shapes. Customize it in Python when you need stricter request values or your upstream gateway adds metadata that the binding rejects.
This guide restricts reasoning_effort and allows a diagnostic
gateway_trace_id on buffered Chat Completions responses and streamed chunks.
Assistant text remains guarded; the metadata value is opaque and is not checked.
Changing one binding does not update the others. Accepted payloads retain their original bytes; these models validate and locate text rather than rewrite JSON. The YAML launcher’s native and LiteLLM bindings remain unchanged.
Allowing a previously disabled field does not implement a new guarded capability. Nonempty tool calls, tool-result conversations, multimodal content, and additional generated-content channels need suitable extraction, checking, and protocol support. Declaring their contents opaque would let them pass unchecked. This walkthrough keeps those restrictions in place.
Restrict an Existing Request Field
Use the installed library environment from the
quickstart. Create
custom_binding.py in your application directory:
The override accepts omission, null, or one of the three declared values. Other
values are rejected before upstream dispatch. The inherited stream selector
still chooses buffered or streaming output; the same request restriction applies
to both. All other request restrictions, including the single user-text message
and disabled tools, remain in effect. Your upstream provider and model must also
support the values you allow. A binding does not add provider support.
Allow Custom Buffered Response Metadata
Append this code to custom_binding.py:
This allows the root-level name gateway_trace_id without requiring it or
constraining its value. Use this allowance only for diagnostic metadata.
Assistant text at choices[0].message.content remains the checked
subject, and other unknown root fields remain rejected.
Rebuilding the text location and payload contract binds the full inherited and customized model declarations. An exported YAML file alone cannot change them.
Allow the Metadata on Streamed Chunks
Streaming has its own payload model and classifier. Append the following to
custom_binding.py to allow the same root-level metadata on Chat chunks:
The new classifier uses the extended model for chunk validation and derives its
text path and field coverage from that model. Content at
choices[0].delta.content stays guarded. Metadata-only chunks, errors, keepalives,
and [DONE] retain their existing roles. The unchanged OpenAI lifecycle hooks
still enforce completion and ordering. Reuse them only when your upstream gateway
follows those protocol semantics; other event formats need corresponding rules and hooks.
Select the Customized Endpoint
After copying proxy-example as described in the quickstart, create
custom_app.py beside custom_binding.py:
Replace https://gateway.example/v1 with your upstream gateway’s API base, then run
uvicorn custom_app:app --host 127.0.0.1 --port 8000 from that directory.
This app guards only Chat Completions. Responses requests pass through without
Rails checks.
Send the quickstart’s
buffered Chat Completions request
or streaming Chat Completions request
using your upstream gateway’s model ID and authorization header. The endpoint
accepts the optional gateway_trace_id in both response modes; ordinary OpenAI does not
return that field. NeMo Guardrails Gateway does not require an upstream gateway
SDK; any configured detector services retain their own dependencies and
credentials.
When output checks are disabled, successful upstream response bodies pass through without these response-policy checks.
Export and Review Your Customized Guard Contract
Export from the endpoint selected by your application, using only a trusted local Python reference:
The export includes the restricted request values and opaque metadata allowance
in both buffered and streaming policies. --check fails when the file differs
from the current export. Regenerate it after changing the Python declaration
and review the diff.
The Python bindings remain authoritative. The export does not configure the gateway, enable additional content handling, or serialize arbitrary Python validators, stateful stream hooks, or transport behavior. Refer to guard contracts for the format’s limits and the Chat reference for nested projections and streaming declarations.