ReferencePython SDK ReferenceNemoguardrailsGuardrails

nemoguardrails.guardrails.engine_registry

View as Markdown

Engine registry for IORails: one ModelEngine per configured model type.

Module Contents

Classes

NameDescription
EngineRegistryOne ModelEngine per configured model, keyed by model type.

Data

_EngineT

log

API

class nemoguardrails.guardrails.engine_registry.EngineRegistry(
    models: list[nemoguardrails.rails.llm.config.Model],
    tracer: typing.Optional[opentelemetry.trace.Tracer] = None,
    metrics_enabled: bool = False,
    content_capture_enabled: bool = False
)

One ModelEngine per configured model, keyed by model type.

_engines
dict[str, BaseEngine] = {}
_llms
dict[str, LLMModel] = {}
llms
dict[str, LLMModel]

The configured model engines keyed by Model.type.

This is the Dict[str, LLMModel] that library rail actions index by model type (llms["content_safety"]). API engines are not LLMs and are excluded.

nemoguardrails.guardrails.engine_registry.EngineRegistry.__aenter__()
async

Async context manager entry: start all engine clients.

nemoguardrails.guardrails.engine_registry.EngineRegistry.__aexit__(
    exc_type,
    exc_val,
    exc_tb
)
async

Async context manager exit: stop all engine clients.

nemoguardrails.guardrails.engine_registry.EngineRegistry._get_engine(
    name: str,
    expected_type: type[nemoguardrails.guardrails.engine_registry._EngineT]
) -> nemoguardrails.guardrails.engine_registry._EngineT

Look up an engine by name, verifying its type.

nemoguardrails.guardrails.engine_registry.EngineRegistry._rollback_start(
    started: list[tuple[str, nemoguardrails.guardrails.base_engine.BaseEngine]]
) -> None
async

Release everything a partial start brought up.

nemoguardrails.guardrails.engine_registry.EngineRegistry.extract_tool_exchanges(
    model_type: str,
    messages: list[dict]
) -> list[nemoguardrails.guardrails.tool_schema.ToolExchange]

Group messages into per-turn (tool_calls, tool_results) exchanges.

Delegates to the engine’s extract_tool_exchanges so each tool result is validated against its own turn’s calls. This keeps call_id linkage turn-local, which RailsManager.are_tool_results_safe relies on so that ids reused across turns (spec-allowed) are not flagged as ambiguous duplicates.

Raises:

  • KeyError: If no engine is registered with the given name.
  • TypeError: If the named engine is not a ModelEngine.
nemoguardrails.guardrails.engine_registry.EngineRegistry.extract_tool_results(
    model_type: str,
    messages: list[dict]
) -> list[nemoguardrails.guardrails.tool_schema.ToolResult]

Extract incoming tool results from messages for the named model engine.

Delegates to the engine’s extract_tool_results so the provider’s tool-result messages are normalized into the ToolResult list the ToolResultRail consumes.

Raises:

  • KeyError: If no engine is registered with the given name.
  • TypeError: If the named engine is not a ModelEngine.
nemoguardrails.guardrails.engine_registry.EngineRegistry.model_call(
    model_type: str,
    messages: list[dict],
    kwargs: typing.Any = {}
) -> nemoguardrails.types.LLMResponse
async

Route a chat completion request to the named model engine.

Returns the structured LLMResponse from the engine — content, reasoning (when the provider exposes it), usage, finish reason. Callers that only want the assistant text should access .content.

Parameter merging and OTEL instrumentation live in ModelEngine.generate_from_messages so that rails, which reach the model through llm_call rather than through this method, emit the same spans and metrics. messages is already in wire form here — every IORails entry point normalizes through IORails._convert_to_messages — so this skips the generate_async protocol adapter.

Raises:

  • KeyError: If no engine is registered with the given name.
  • TypeError: If the named engine is not a ModelEngine.
nemoguardrails.guardrails.engine_registry.EngineRegistry.parse_tools(
    model_type: str,
    llm_params: typing.Optional[dict]
) -> nemoguardrails.guardrails.tool_schema.Toolset

Parse the tool block in llm_params for the named model engine.

Delegates to the engine’s parse_tools so the provider-specific shape (keyed on the engine) is normalized into a Toolset for the tool rails.

Raises:

  • KeyError: If no engine is registered with the given name.
  • TypeError: If the named engine is not a ModelEngine.
nemoguardrails.guardrails.engine_registry.EngineRegistry.provider_name(
    model_type: str
) -> str

Return the provider/engine name (e.g. ‘nim’, ‘openai’) for a model engine.

nemoguardrails.guardrails.engine_registry.EngineRegistry.start() -> None
async

Start all engine clients.

Call this during service startup. A failure part-way through rolls everything already started back, so a failed start leaks nothing.

nemoguardrails.guardrails.engine_registry.EngineRegistry.stop() -> None
async

Stop all engine clients.

Call this during service shutdown. Every component is stopped even if an earlier one fails; the failures are reported together afterwards.

nemoguardrails.guardrails.engine_registry.EngineRegistry.stream_model_call(
    model_type: str,
    messages: list[dict],
    kwargs: typing.Any = {}
) -> collections.abc.AsyncGenerator[nemoguardrails.types.LLMResponseChunk, None]
async

Stream chat completion chunks from the named model engine.

Yields LLMResponseChunk objects. Parameter merging, the LLM CLIENT span, and the metrics live in ModelEngine.stream_from_messages — see that method for the span and metric contract. As in model_call, messages is already in wire form, so this skips the stream_async protocol adapter.

Raises:

  • KeyError: If no engine is registered with the given name.
  • TypeError: If the named engine is not a ModelEngine.
nemoguardrails.guardrails.engine_registry._EngineT = TypeVar('_EngineT', bound=BaseEngine)
 
nemoguardrails.guardrails.engine_registry.log = logging.getLogger(__name__)