nemoguardrails.guardrails.engine_registry
nemoguardrails.guardrails.engine_registry
Engine registry for IORails: one ModelEngine per configured model type.
Module Contents
Classes
Data
API
One ModelEngine per configured model, keyed by model type.
The configured model engines keyed by Model.type.
This is the Dict[str, LLMModel] that library rail actions index by
model type (llms["content_safety"]). API engines are not LLMs and
are excluded.
Async context manager entry: start all engine clients.
Async context manager exit: stop all engine clients.
Look up an engine by name, verifying its type.
Strictly parse the last assistant message’s tool calls with the named model engine.
Group messages into per-turn (tool_calls, tool_results) exchanges.
Delegates to the engine’s extract_tool_exchanges so each tool result is
validated against its own turn’s calls. This keeps call_id linkage
turn-local, which RailsManager.are_tool_results_safe relies on so that ids
reused across turns (spec-allowed) are not flagged as ambiguous duplicates.
Raises:
KeyError: If no engine is registered with the given name.TypeError: If the named engine is not a ModelEngine.
Extract incoming tool results from messages for the named model engine.
Delegates to the engine’s extract_tool_results so the provider’s
tool-result messages are normalized into the ToolResult list the
ToolResultRail consumes.
Raises:
KeyError: If no engine is registered with the given name.TypeError: If the named engine is not a ModelEngine.
Route a chat completion request to the named model engine.
Returns the structured LLMResponse from the engine — content,
reasoning (when the provider exposes it), usage, finish reason.
Callers that only want the assistant text should access .content.
Parameter merging and OTEL instrumentation live in
ModelEngine.generate_from_messages so that rails, which reach the
model through llm_call rather than through this method, emit the
same spans and metrics. messages is already in wire form here — every
IORails entry point normalizes through IORails._convert_to_messages
— so this skips the generate_async protocol adapter.
Raises:
KeyError: If no engine is registered with the given name.TypeError: If the named engine is not a ModelEngine.
Parse the tool block in llm_params for the named model engine.
Delegates to the engine’s parse_tools so the provider-specific shape
(keyed on the engine) is normalized into a Toolset for the tool rails.
Raises:
KeyError: If no engine is registered with the given name.TypeError: If the named engine is not a ModelEngine.
Return the provider/engine name (e.g. ‘nim’, ‘openai’) for a model engine.
Start all engine clients.
Failure or cancellation attempts bounded rollback, including the engine whose start was interrupted. Unfinished cleanup must complete before starting again; it can also be retried explicitly with stop().
Stop all engine clients.
Successful closes are forgotten; failed or interrupted closes remain owned for retry. Ordinary failures do not prevent other engines from closing. Cancellation propagates without claiming cleanup completed.
Stream chat completion chunks from the named model engine.
Yields LLMResponseChunk objects. Parameter merging, the LLM
CLIENT span, and the metrics live in
ModelEngine.stream_from_messages — see that method for the span and
metric contract. As in model_call, messages is already in wire
form, so this skips the stream_async protocol adapter.
Raises:
KeyError: If no engine is registered with the given name.TypeError: If the named engine is not a ModelEngine.