> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/relay/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# nemo-relay

> NeMo Relay Core

Generated from `cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi`.

## NeMo Relay Core

The core runtime library for the NeMo Relay multi-language agent framework. This crate provides execution scope management, lifecycle event tracking, and middleware pipelines (guardrails and intercepts) for tool and LLM calls.

### Architecture

The runtime is organized around a **global context** ([`api::runtime::NemoRelayContextState`](/reference/api/rust-library-reference/nemo-relay/api/runtime/state/struct-nemorelaycontextstate)) that holds all registered middleware (guardrails, intercepts, subscribers) and a **scope stack** ([`api::runtime::ScopeStack`](/reference/api/rust-library-reference/nemo-relay/api/runtime/scope_stack/struct-scopestack)) that tracks the hierarchical execution context via task-local or thread-local storage.

### Primary Entry Points

Most integrations start with the high-level lifecycle helpers in [`api`](/reference/api/rust-library-reference/nemo-relay/api):

* [`api::scope::push_scope`](/reference/api/rust-library-reference/nemo-relay/api/scope/fn-push-scope) / [`api::scope::pop_scope`](/reference/api/rust-library-reference/nemo-relay/api/scope/fn-pop-scope) create nested execution scopes.
* [`api::tool::tool_call_execute`](/reference/api/rust-library-reference/nemo-relay/api/tool/fn-tool-call-execute) runs a complete tool middleware pipeline.
* [`api::llm::llm_call_execute`](/reference/api/rust-library-reference/nemo-relay/api/llm/fn-llm-call-execute) and [`api::llm::llm_stream_call_execute`](/reference/api/rust-library-reference/nemo-relay/api/llm/fn-llm-stream-call-execute) run non-streaming and streaming LLM middleware pipelines.
* [`api::registry`](/reference/api/rust-library-reference/nemo-relay/api/registry) exposes global and scope-local middleware registration APIs.
* [`api::subscriber`](/reference/api/rust-library-reference/nemo-relay/api/subscriber) exposes lifecycle event subscriber registration APIs.

#### Modules

* [`api`](/reference/api/rust-library-reference/nemo-relay/api) - Public API functions, handles, lifecycle event types, runtime helpers, and guardrail/intercept/subscriber registration. These are the primary entry points.
* [`error`](/reference/api/rust-library-reference/nemo-relay/error) - Error types ([`error::FlowError`](/reference/api/rust-library-reference/nemo-relay/error/enum-flowerror)) and the [`error::Result`](/reference/api/rust-library-reference/nemo-relay/error/type-result) type alias.
* [`json`](/reference/api/rust-library-reference/nemo-relay/json) - JSON type alias ([`json::Json`](/reference/api/rust-library-reference/nemo-relay/json/type-json)) and the [`json::merge_json`](/reference/api/rust-library-reference/nemo-relay/json/fn-merge-json) utility.
* [`observability`](/reference/api/rust-library-reference/nemo-relay/observability) - Built-in observability backends including [`atif::AtifExporter`](/reference/api/rust-library-reference/nemo-relay/observability/atif/struct-atifexporter), [`otel::OpenTelemetrySubscriber`](/reference/api/rust-library-reference/nemo-relay/observability/otel/struct-opentelemetrysubscriber), and [`openinference::OpenInferenceSubscriber`](/reference/api/rust-library-reference/nemo-relay/observability/openinference/struct-openinferencesubscriber).
* [`stream`](/reference/api/rust-library-reference/nemo-relay/stream) - [`stream::LlmStreamWrapper`](/reference/api/rust-library-reference/nemo-relay/stream/struct-llmstreamwrapper) - a stream adapter that applies per-chunk intercepts and aggregates streaming LLM responses.

### Middleware Pipeline

Both tool and LLM calls flow through a configurable middleware pipeline:

1. **Request intercepts** - transform the request before execution
2. **Sanitize request guardrails** - sanitize/normalize the request
3. **Conditional execution guardrails** - gate execution (reject if criteria not met)
4. **Execution intercepts** - optionally replace the execution function entirely
5. **Sanitize response guardrails** - sanitize/normalize the response

All middleware is priority-ordered (ascending) and registered by name for easy addition and removal at runtime.

## Modules

* [api](/reference/api/rust-library-reference/nemo-relay/api): Public API for the NeMo Relay runtime.
* [codec](/reference/api/rust-library-reference/nemo-relay/codec): LLM codec types, traits, and built-in implementations.
* [config\_editor](/reference/api/rust-library-reference/nemo-relay/config_editor): Typed configuration editor metadata.
* [error](/reference/api/rust-library-reference/nemo-relay/error): Error types for the NeMo Relay runtime.
* [json](/reference/api/rust-library-reference/nemo-relay/json): JSON utilities for the NeMo Relay runtime.
* [observability](/reference/api/rust-library-reference/nemo-relay/observability): Optional observability integrations for NeMo Relay Core.
* [plugin](/reference/api/rust-library-reference/nemo-relay/plugin): Generic plugin infrastructure for NeMo Relay runtimes.
* [plugins](/reference/api/rust-library-reference/nemo-relay/plugins): First-party plugin implementations for NeMo Relay Core.
* [stream](/reference/api/rust-library-reference/nemo-relay/stream): Streaming LLM response wrapper.

## Macros

* [editor\_config](/reference/api/rust-library-reference/nemo-relay/macro-editor-config): Implements [`EditorConfig`](/reference/api/rust-library-reference/nemo-relay/config_editor/trait-editorconfig) for a configuration type.