For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
  • About NVIDIA NeMo Relay
    • Overview
    • Architecture
    • Ecosystem
    • Concepts
    • Release Notes
  • Getting Started
    • Agent Runtime Primer
    • Prerequisites
    • Installation
    • Configuration / Setup
    • Quick Start
  • NVIDIA NeMo Relay CLI
    • About
    • Basic Usage
    • Claude Code
    • Codex
    • Cursor
    • Hermes Agent
  • Supported Integrations
    • About
    • OpenClaw Plugin Guide
    • LangChain Integration Guide
    • LangGraph Integration Guide
    • Deep Agents Integration Guide
  • Instrument Applications
    • About
    • Adding Scopes and Marks
    • Instrument a Tool Call
    • Instrument an LLM Call
    • Add Middleware
    • Code Examples
  • Observability Plugin
    • About
    • Configuration
    • Agent Trajectory Interchange Format (ATIF)
    • Agent Trajectory Observability Format (ATOF)
    • OpenTelemetry
    • OpenInference
  • Adaptive Plugin
    • About
    • Configuration
    • Adaptive Cache Governor (ACG)
    • Adaptive Hints
  • NeMo Guardrails Plugin
    • About
    • Configuration
  • Integrate into Frameworks
    • About
    • Adding Scopes
    • Wrap Tool Calls
    • Wrap LLM Calls
    • Handle Non-Serializable Data
    • Using Codecs
    • Provider Codecs
    • Provider Response Codecs
    • Code Examples
  • Build Plugins
    • About
    • Define a Plugin
    • Validate Plugin Configuration
    • Plugin Configuration Files
    • Register Plugin Behavior
    • Design Plugin Configuration
    • NeMo Guardrails Example Plugin
    • Code Examples
  • Contribute
    • About
    • Development Setup
    • Workflow and Reviews
    • Testing and Documentation
  • Reference
    • APIs
      • Python Library Reference
      • Node.js Library Reference
      • Rust Library Reference
        • nemo-relay
        • nemo-relay-adaptive
        • nemo-relay-ffi
          • api
          • callable
          • convert
          • error
          • types
    • Performance
  • Resources
    • Support and FAQs
    • Glossary
    • Troubleshooting Guide
    • Community
    • Legal
NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.

LogoLogo
On this page
  • Middleware Pipeline
  • Error Handling
  • Memory Ownership
  • Modules
ReferenceAPIsRust Library Reference

nemo-relay-ffi

||View as Markdown|
Previous

Struct Adaptive Runtime

Next

Module api

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

C FFI layer for NeMo Relay.

This crate exposes the NeMo Relay core runtime as a C-compatible shared library. It is consumed by the Go bindings via CGo and regenerates the committed nemo_relay.h header through cbindgen during Cargo builds. All exported symbols use the nemo_relay_ prefix.

Middleware Pipeline

When a tool or LLM call is executed end-to-end via the _execute functions, the runtime applies the following middleware pipeline in order:

  1. Request intercepts - transform the request before guardrails.
  2. Sanitize-request guardrails - validate/sanitize the request.
  3. Conditional-execution guardrails - gate whether the call proceeds.
  4. Execution intercepts - optionally replace the call implementation.
  5. Sanitize-response guardrails - validate/sanitize the response.

Error Handling

Every extern "C" function returns an error::NemoRelayStatus code. On failure, call error::nemo_relay_last_error on the same thread to retrieve a human-readable error description. The error is stored in thread-local storage and is valid until the next FFI call on that thread.

Memory Ownership

All opaque handles (FfiScopeHandle, FfiToolHandle, FfiLLMHandle, etc.) are heap-allocated and must be freed through their corresponding nemo_relay_*_free functions. C strings returned by accessor functions must be freed with nemo_relay_string_free.

Modules

  • api - Top-level FFI entry points (scope, tool, LLM, guardrail, intercept, subscriber, ATIF exporter). Tool calls accept an optional tool_call_id and LLM calls accept an optional model_name for ATIF trajectory correlation. ATIF exporter functions (nemo_relay_atif_exporter_*) create, register, export, and clear trajectory data.
  • types - C-compatible struct and enum definitions, plus event accessor functions (nemo_relay_event_input, _output, _model_name, _tool_call_id, _parent_uuid, _scope_type) and the FfiAtifExporter opaque handle.
  • error - Status codes and thread-local error storage.
  • callable - C function pointer typedefs and wrapper functions.
  • convert - JSON and C-string conversion utilities.