nemoguardrails.server.api
Module Contents
Classes
Functions
Data
llm_rails_events_history_cache
API
Bases: BaseModel
A terminal error frame pushed into the stream by the guardrails runtime.
type is restricted to the internal markers so that model output which
merely looks like an OpenAI error object is streamed on as ordinary content
rather than ending the stream. Without output rails nothing else inspects a
chunk before it reaches process_chunk, so this is the last gate.
Bases: BaseModel
Bases: FastAPI
Custom FastAPI subclass with additional attributes for Guardrails server.
Bases: Exception
Exception raised for errors in the configuration.
Build the main model for a request, on top of the configured one when there is one.
Return the model the config declares as “main”, if it declares one.
Format streaming chunks from LLMRails.stream_async() as SSE events.
Args: stream_iterator: AsyncIterator from stream_async() that yields str or dict chunks model_name: The model name to include in the chunks
Yields: SSE-formatted strings (data: {…}
)
Generates a cache key for the given config ids and model name.
Returns the rails instance for the given config id and model.
Parameters:
List of configuration IDs to load
The model name from the request (overrides config’s main model)
Check if a directory (or its ‘config’ subdirectory) contains a config.yml/yaml.
violation with a tool result’s message index counted in the request’s own messages.
Inject the request’s model into a RailsConfig, keeping the configured main model’s fields.
Move reasoning_content into the assistant message as a <think> prefix and clear the field.
Map internal RailStatus to API status string.
Resolve the engine for an injected main model. Priority order:
- MAIN_MODEL_ENGINE environment variable (warns if mismatch with
configured_model.engine) configured_model.engineif configured_model is provided- Fallback to
openai
Resolve the parameters for an injected main model, preferring MAIN_MODEL_BASE_URL.
Register any additional challenges, if available at startup.
Update the main model in the RailsConfig.
If a model with type=“main” exists, it replaces it. Otherwise, adds it.
Build a main model from its fields, reporting a rejected field as a configuration error.
The tool violations with each index pointing into the messages the caller sent.
Chat completion for the provided conversation.
TODO: add support for explicit state object.
Returns the list of available challenges for red teaming.
Returns the list of available rails configurations.
Guardrail check request.
Returns 422 when rail_types includes a type with no configured flows, when
the serving engine cannot run a tool check it was asked for: a tool rail type, or
tools, on a config the IORails engine does not serve, or when tools holds an
entry that isn’t a function tool.
Return HTTP 200 while the server process is running and able to serve requests.
Run the server lifespan inside the metrics exporter’s lifetime.
The exporter starts before anything can construct a rails instance so the
first IORails metric lands on a real MeterProvider (a no-op unless
NEMO_GUARDRAILS_SERVER_METRICS_EXPORTER is set). An exporter already
running is reused: one started directly, as the CLI does, is left for its
starter to stop, and one a lifespan started stops as the last lifespan
holding it exits, even when startup fails inside _server_lifespan,
for example on a malformed challenges.json.
Return the list of models available from the configured provider.
Processes a single chunk from the stream.
Parameters:
A single chunk from the stream (can be str, dict, or other type).
The model name (not used in processing but kept for signature consistency).
Returns: Union[Any, ChunkError , ToolCallChunk , ToolCallDeltaChunk ]
A ChunkError instance for terminal errors, or the original text or structured event.
Register additional challenges
Parameters:
The new challenges to be registered.
Registers a DataStore to be used by the server.
Register an additional logger
Start a thread that monitors the config folder for changes.