Migrating to 0.25
Version 0.25.0 of the NVIDIA NeMo Guardrails library deprecates the LLMRails engine and the Colang 1.0 and Colang 2.x runtimes.
Removal is planned no earlier than 0.27.0, and the release that removes them is not decided yet.
Until then, they keep working.
Use Guardrails, which runs the IORails engine.
IORails reads the same RailsConfig and config.yml as LLMRails.
What changes is the engine that runs your configuration, and the parts of a configuration that only LLMRails uses.
What Is Deprecated
The last row is deprecated in the documentation only: using it emits no Python warning.
For example, Guardrails falling back to LLMRails only logs the reason, as described in Configurations That Fall Back to LLMRails.
The following are unchanged in 0.25.0 and emit no Python warning on a Colang 1.0 configuration, although several of them run on LLMRails:
Guardrails(config, llm=...)andGuardrails(config, use_iorails=False)TestChat- the LangChain integration:
RunnableRails, the LangChain agent middleware, the LangChain model framework, andLangChainLLMAdapter - the
nemoguardrails convert,actions-server, andeval rail topicalcommands nemoguardrails chatwith a Colang 1.0 configuration, andnemoguardrails server
Messages You Can Receive
Find the message you received in the following table, and follow the link in the last column.
LLMRailsRemovalWarning is a subclass of DeprecationWarning, and you can import it from nemoguardrails.
The library attributes it to the first line outside the library in the call stack, so it points at your code, and one construction emits at most one.
Under Python’s default warning filters, a DeprecationWarning shows only when it is attributed to __main__.
You see it when a script or notebook constructs LLMRails, and pytest shows it in its warnings summary.
It stays hidden when a module that your application imports constructs LLMRails, and in the Guardrails API server and the CLI.
To show it everywhere, refer to Silence or Escalate the Warnings.
The UserWarning diagnostic describes what IORails does today and carries no removal date.
It names the configuration’s directory and each part that IORails ignores.
Python shows it by default, once for each line of code and configuration directory that triggers it.
Replace LLMRails with Guardrails
Change the import and the constructor.
For message-based calls, generate() and generate_async() return the same assistant message dictionary on both engines.
The following code uses LLMRails:
The following code uses Guardrails instead:
Guardrails runs IORails when IORails can run the whole configuration.
Otherwise, it falls back to LLMRails and logs the reason, as described in Configurations That Fall Back to LLMRails.
A configuration that IORails cannot run and that also has per-tool rails is the exception: it raises ValueError instead of falling back.
To fail at construction instead of falling back, pass require_iorails=True:
With require_iorails=True, the constructor raises a ValueError with the same text as the fallback log line, before it builds LLMRails.
Keep One Instance Running
The synchronous generate() and check() methods on IORails start a short-lived engine for every call, with tracing and metrics turned off.
They suit scripts.
In a service, keep one Guardrails instance running and call the asynchronous methods:
If your application has its own startup and shutdown hooks, call await rails.startup() and await rails.shutdown() from them instead of using async with.
Methods That IORails Does Not Provide
When IORails runs the configuration, Guardrails raises NotImplementedError for the methods and properties that need the Colang runtime or a model object you supply:
explain()and theexplain_infopropertygenerate_events(),generate_events_async(),process_events(), andprocess_events_async()update_llm()and thellmproperty- the
runtimeandpassthrough_fnproperties register_action(),register_action_param(),register_filter(),register_output_parser(),register_prompt_context(),register_embedding_search_provider(), andregister_embedding_provider()
Code that depends on them has no IORails equivalent.
GenerationOptions also behaves differently on IORails.
For the details, refer to Generation Options: IORails and LLMRails Differences and Engine Feature Support.
Models You Pass with llm
IORails creates every model from models in config.yml and calls it through its own OpenAI-compatible client.
It does not accept a model object, so Guardrails(config, llm=...) runs LLMRails.
The llm argument is not deprecated and keeps working.
To run the configuration on IORails, configure the main model under models and leave out the argument:
IORails knows the default endpoint only for the nim and openai engines.
For a model with any other engine, set parameters.base_url.
Providers that you register with register_provider() and custom LLM frameworks have no effect on IORails.
To call another provider’s OpenAI-compatible endpoint with the openai engine, set api_key_env_var to the environment variable that holds that endpoint’s key:
Without api_key_env_var, the openai engine sends the key in OPENAI_API_KEY to base_url, which hands your OpenAI key to another provider.
For an endpoint that needs no key, leave out api_key_env_var and parameters.api_key, and unset OPENAI_API_KEY for the process.
IORails then sends no Authorization header, unless you set one in parameters.default_headers.
Guardrails(config, use_iorails=False) always runs LLMRails.
Remove the argument so that Guardrails runs IORails when IORails can run the configuration.
What IORails Ignores in a Configuration
IORails runs the input, output, and tool rails in config.yml that it supports.
It ignores the following parts of a configuration, which only LLMRails uses:
Move the model and rail settings that config.py makes into config.yml.
Guardrails looks for config.py in the configuration’s directory and in the directories it imports.
A configuration that RailsConfig.from_content() builds has no directory of its own, so Guardrails checks only the directories that its import_paths setting names.
When a configuration has several of these parts, one warning names all of them.
To keep them working, pass use_iorails=False, which runs LLMRails.
Configurations That Fall Back to LLMRails
When IORails cannot run a configuration and the configuration has no per-tool rails, Guardrails falls back to LLMRails.
It logs the reason at WARNING level on the nemoguardrails.guardrails.guardrails logger:
If your application configures no logging, Python prints the record to standard error.
The fallback emits no Python warning of its own; a Colang 2.x configuration also emits the Colang 2.x LLMRailsRemovalWarning when LLMRails starts.
The following table maps the reason after IORails cannot be used: to the change to make.
A configuration that IORails cannot run and that also has per-tool rails (rails.tool_call.per_tool or rails.tool_result.per_tool) does not fall back.
Per-tool rails alone do not cause this; a configuration that IORails can run, per-tool rails included, runs on IORails.
Guardrails raises a ValueError that names the same reason instead, because LLMRails does not run per-tool rails and falling back would turn them off.
To find fallbacks before you deploy, construct Guardrails(config, require_iorails=True) in a test for each configuration.
You can also call IORails.unsupported_reason(config), which returns the reason, or None when IORails can run the configuration:
The IORails column of Rail Engine Support lists every built-in rail that IORails runs, and the page explains why it refuses the others.
A rail that only LLMRails runs has no IORails equivalent.
Colang Configurations
IORails reads Colang 1.0 configurations but runs none of their Colang flows, and it does not support Colang 2.x.
Dialog Flows in Colang 1.0
IORails runs a Colang 1.0 configuration that contains its own define flow or define user blocks, but it does not run those dialog flows.
It runs only the input, output, and tool rails in config.yml that it supports, and falls back to LLMRails for a configuration that lists others.
Guardrails emits a UserWarning when it runs such a configuration on IORails, because the configuration already behaves differently than it does on LLMRails.
That includes a flow that copies or redefines a built-in rail’s flow, because IORails runs only the built-in rail and ignores the flows in your configuration.
Move the logic of your dialog flows into input and output rails:
- To refuse topics or requests that a flow used to catch, add an input rail from the Guardrail Catalog, such as topic safety or content safety.
- To check or rewrite what the model returns, add an output rail.
The Colang 1.0 runtime is deprecated together with LLMRails, and its removal is planned no earlier than 0.27.0.
Colang 2.x
IORails does not support Colang 2.x and will not, so there is no replacement runtime.
Every construction of LLMRails on a Colang 2.x configuration emits an LLMRailsRemovalWarning, whichever route builds it:
- your code, directly or through a subclass
Guardrails, when it falls back, andGuardrails(config, use_iorails=False)TestChat,RunnableRails, and the LangChain agent middleware- the
evalcommands
nemoguardrails chat prints the notice for Colang 2.x chat and its debugger commands to standard error instead of the Python warning.
The Guardrails API server logs the notice for each Colang 2.x configuration it loads instead of the Python warning, which Python hides there by default.
To keep the input and output checks of a Colang 2.x configuration, write a Colang 1.0 config.yml that lists the same built-in rails under rails.input.flows and rails.output.flows.
Colang 2.x flows that steer the conversation have no IORails equivalent.
The CLI and the Guardrails API Server
nemoguardrails chat with a Colang 1.0 configuration and nemoguardrails server run LLMRails by default and print no deprecation notice.
Set NEMO_GUARDRAILS_IORAILS_ENGINE=true to run them through Guardrails, which runs IORails when IORails can run the configuration:
The variable also accepts 1 and yes, in any case.
The library reads it when it is first imported, and it acts on the whole Python process: from nemoguardrails import LLMRails returns Guardrails instead.
Code in that process that constructs LLMRails through that import gets Guardrails and no LLMRailsRemovalWarning, except that a Colang 2.x configuration still warns when Guardrails falls back.
Code that imports LLMRails from nemoguardrails.rails still gets LLMRails and the warning.
To find every LLMRails call site with a pytest filter, run your tests without the variable.
With the variable set, the UserWarning diagnostics and the fallback log line report the same cases as they do in Python code.
The UserWarning names the configuration’s directory, so each configuration is reported.
A request that names several configurations merges them into one, and the merged configuration always falls back to LLMRails, or raises ValueError if it has per-tool rails.
Silence or Escalate the Warnings
The following methods change how Python reports the warnings:
The following example hides the removal warning in application code:
The following pytest.ini turns the removal warning into an error in your tests:
Python’s own -W option and PYTHONWARNINGS cannot name LLMRailsRemovalWarning.
Python prints Invalid -W option ignored: invalid module name: 'nemoguardrails' and applies the default filters.
The nemoguardrails chat notice is printed text, and the fallback line is a log record, so warning filters do not affect them.