> This page is for NeMo Relay, version v0.9.4. > For other versions, use one of these documentation indexes: > - Latest (v0.10.0) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt > - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt > - v0.10.0: https://docs.nvidia.com/nemo/relay/v0.10.0/llms.txt > - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt > - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt > - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt > - v0.6.0: https://docs.nvidia.com/nemo/relay/v0.6.0/llms.txt > - v0.5.0: https://docs.nvidia.com/nemo/relay/v0.5.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server. # Plugin Configuration Files > Configure plugins.toml discovery, layering, validation, and interactive editing. Use `plugins.toml` when Relay should activate plugins from file configuration. The runtime discovers and layers this generic plugin document for direct Rust, Python, and Node.js integrations as well as for the `nemo-relay` CLI gateway. The file encodes the same document that binding APIs accept, but uses TOML at the file root. This guide documents runtime file discovery, precedence, merge behavior, and conflict rules. It also identifies gateway-only editor, explicit-config, and discoverable-plugin workflows. Each component guide documents its component-specific fields. > **Note** > > NeMo Relay plugin configuration keys use `snake_case` regardless of language or > file format. Node.js helper APIs can have `camelCase` function names, but the > generic plugin document and component-local `config` objects use canonical > `snake_case` keys. ## Shortest Path Example Use this minimal `plugins.toml` when you want Relay to start with one plugin-managed observability exporter and no extra layering: ```toml version = 1 [[components]] kind = "observability" enabled = true [components.config] version = 4 [components.config.atof] enabled = true [[components.config.atof.sinks]] type = "file" output_directory = "logs" filename = "events.jsonl" mode = "append" ``` For the most deterministic CLI-gateway verification path, keep `plugins.toml` in the same directory as the gateway `config.toml` file for this run, then launch the wrapper with that explicit config path. For example: ```text path/to/ config.toml plugins.toml ``` Run the gateway with the following command: ```bash nemo-relay --config path/to/config.toml run -- codex ``` This uses the colocated `plugins.toml` instead of the ambient user plugin file. The system file still layers on top, so ensure system policy does not override the exporter you want to verify. The plugin file is the configuration being demonstrated here; `--config` only tells the gateway which low plugin layer to use for this run. For implicit discovery, place the file in the XDG user configuration directory. Refer to [CLI Basic Usage](/nemo-relay-cli/basic-usage) for the wrapper command shapes. ## What Success Looks Like The shortest-path setup is working when: * `nemo-relay` starts without plugin validation or activation errors. * The observability component activates the ATOF exporter configured in this file. * An instrumented gateway run writes ATOF JSONL output to `logs/events.jsonl`. After that path works, expand the config with additional components, policies, or higher-precedence files as needed. ## File Shape `plugins.toml` uses the canonical plugin document shape: ```toml version = 1 [[components]] kind = "observability" enabled = true [components.config] version = 4 [components.config.atof] enabled = true [[components.config.atof.sinks]] type = "file" output_directory = "logs" filename = "events.jsonl" mode = "append" [policy] unknown_component = "warn" unknown_field = "warn" unsupported_value = "error" ``` The top-level fields are: | Field | Default | Notes | | ------------ | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | `version` | `1` | Plugin configuration format version. Non-`1` versions fail validation by default. | | `components` | `[]` | Ordered plugin components to validate and activate. | | `policy` | warn unknown components and fields, error on unsupported values | Global host validation policy. Non-default field/value settings override the matching policy for built-in components. | Each built-in component has: | Field | Default | Notes | | --------- | -------- | ------------------------------------------------------------------ | | `kind` | Required | Registered plugin kind, such as `observability` or `adaptive`. | | `enabled` | `true` | Relay validates disabled components but does not initialize them. | | `config` | `{}` | Component-local configuration object. The shape depends on `kind`. | ## Gateway Discoverable Plugin Records Use `[[plugins.dynamic]]` only for a gateway-managed manifest-backed native or worker plugin. These records remain separate from `[[components]]`. During gateway activation, Relay loads each enabled dynamic adapter, synthesizes its internal component, and validates the component configuration. The following record configures a dynamic plugin: ```toml [[plugins.dynamic]] manifest = "./plugins/acme/relay-plugin.toml" [plugins.dynamic.config] mode = "audit" [plugins.dynamic.config.executor] worker_threads = 4 ``` > **Warning** > > For a Python worker, run `nemo-relay plugins add ` instead of adding > this record manually. The command creates and records the Relay-managed Python > environment that the worker requires at startup. `manifest` is required and resolves relative to this file. `config` is optional. For Rust native plugins using the typed SDK, `config.executor.worker_threads` sets the positive worker count for that component's SDK-owned Tokio executor. When the manifest declares a configuration schema, it must include the SDK-owned `executor` object and a positive-integer `worker_threads` property; otherwise static validation and `nemo-relay plugins edit` reject this supported configuration. Run `nemo-relay plugins validate ` to validate it against the manifest’s optional static JSON Schema before you enable or run the plugin. Use `nemo-relay plugins add`, `validate`, `inspect`, and `enable` to manage the dynamic-plugin lifecycle. Refer to [Configure Discoverable Plugins](/configure-plugins/discoverable-plugins) for manifest, trust, and policy requirements. ### Embedded Python Host Initialization Python hosts pass an optional explicit `plugins.toml` directly to the unified host initializer: ```python import asyncio from nemo_relay import plugin async def main() -> None: async with plugin.activate( {}, "path/to/plugins.toml" ) as activation: # Run your host application while dynamic plugins are active. print(activation.report) ... asyncio.run(main()) ``` Core resolves manifest references relative to the declaring file, applies lifecycle state and trust policy, validates all selected plugins before loading code, and owns every resulting runtime through the returned activation handle. Use `plugin.activate(...)` when the host lifetime fits one `async with` block. Use `await plugin.initialize(...)` when the caller needs to retain and close the activation manually. The runtime reads only files named `plugins.toml` during default discovery. ## Runtime Discovery The runtime resolves plugin configuration from `plugins.toml` files and an optional code-driven layer. Direct Rust, Python, and Node.js calls to their plugin-initialization APIs use this same discovery and layering behavior. File configuration comes from `plugins.toml`: | Source | Use case | | -------------- | ------------------------------------------------------------- | | `plugins.toml` | Normal user- and system-managed runtime plugin configuration. | The runtime does not read plugin configuration from `config.toml`. ### Gateway Explicit Config When the CLI gateway receives `--config path/to/config.toml`, it scopes plugin file discovery's low layer to `path/to/plugins.toml`. An explicit `--plugin-config-path` selects that low layer directly. Either explicit form replaces the ambient user plugin file; the system file still applies. ### Default Discovery Locations The runtime checks these `plugins.toml` locations from lowest to highest precedence: 1. Explicit or user: * `--plugin-config-path`, or the `plugins.toml` beside `--config`, when supplied * otherwise `$XDG_CONFIG_HOME/nemo-relay/plugins.toml`, or `~/.config/nemo-relay/plugins.toml` when `XDG_CONFIG_HOME` is not set 2. System: * Unix: `/etc/nemo-relay/plugins.toml` * Windows: `%ProgramData%\nemo-relay\plugins.toml` The runtime skips missing files and loads a physical file only once when multiple paths or symlinks select it. If no plugin config source exists, initialization continues without process-level plugin activation. Repository-local `.nemo-relay/plugins.toml` files are not discovered. ## Gateway Editing Files Use the interactive editor for Observability, Adaptive, PII Redaction, and existing NeMo Guardrails configuration. The built-in `nemo_guardrails` component is deprecated and scheduled for removal in NeMo Relay 0.9; do not use the editor to start a new Guardrails deployment. The editor also updates the configuration of manifest-backed dynamic plugins that `plugins add` has registered: ```bash nemo-relay plugins edit ``` By default, the editor writes the user plugin file: ```text $XDG_CONFIG_HOME/nemo-relay/plugins.toml ``` or: ```text ~/.config/nemo-relay/plugins.toml ``` When the top-level CLI receives `--plugin-config-path`, the editor uses that exact file. Otherwise, `--config path/to/config.toml` makes the editor use the sibling `path/to/plugins.toml`, matching the runtime selection for that configuration. This explicit file replaces the user editor target, so `plugins edit --user` keeps the inherited explicit target. Use `--global` to edit the system layer. These rules only select the file opened by the editor; they do not change runtime discovery, layering, or merge precedence. Use a scope flag to edit another location: ```bash nemo-relay plugins edit --global ``` Scope flags are mutually exclusive. `--global` writes `/etc/nemo-relay/plugins.toml` on Unix or `%ProgramData%\nemo-relay\plugins.toml` on Windows and usually requires elevated filesystem permissions. The editor menus support these controls: | Key | Behavior | | --------------------- | ------------------------------------------------------ | | Arrow keys, `j`, `k` | Move through menu items. | | `Enter`, `Space` | Select or toggle the highlighted item. | | `Backspace`, `Delete` | Clear the highlighted optional field. | | `r` | Reset the highlighted field or section to its default. | | `p` | Preview TOML from the main menu. | | `s` | Save from the main menu. | | `?` | Show help. | | `q`, `Esc` | Go back or cancel without saving. | Text and JSON value prompts use normal line editing. Use the surrounding field menu to reset, clear, preview, or save. ## Precedence and Merge Behavior When more than one `plugins.toml` file is discovered, later files have higher precedence. System config overrides the selected explicit-or-user config. TOML tables merge recursively. Top-level lists inside a component's `config` concatenate, as do the declared observability destination lists. Entries from the higher-precedence layer are placed before entries from the lower-precedence layer. Other nested lists replace. For example, the following user configuration enables one ATOF file sink: ```toml # user plugins.toml [[components]] kind = "observability" [components.config] version = 4 [components.config.atof] enabled = true [[components.config.atof.sinks]] type = "file" output_directory = "~/.local/state/nemo-relay" mode = "append" ``` The system scope can add a fleet sink without repeating the user sink: ```toml # system plugins.toml [[components]] kind = "observability" [components.config.atof] [[components.config.atof.sinks]] type = "file" output_directory = "/var/log/nemo-relay" mode = "overwrite" ``` The effective Agent Trajectory Observability Format (ATOF) configuration keeps `version` and `enabled` from the user file. Its `sinks` list contains the system sink first, followed by the user sink. The top-level `components` array is special. Relay matches enabled components by `kind` across files. A higher-precedence component with the same `kind` merges into the lower-precedence component. Relay adds a component with a different `kind` to the effective configuration. Dynamic plugin records are special as well. Relay matches `[[plugins.dynamic]]` records by the plugin ID declared in their manifests. A higher-precedence record with the same ID replaces the lower-precedence record entirely, including its manifest reference and optional `config`. This lets system configuration select the approved dynamic plugin package and configuration without inheriting values from the user or explicit file. Declare a plugin ID at most once in each individual `plugins.toml` file. A component entry that explicitly sets `enabled = false` is skipped before matching and merging. It does not change a lower-precedence component's enabled state or contribute any `config` fields or list entries. If every layer for a component kind sets `enabled = false`, that kind is absent from the effective configuration. This behavior applies to list fields declared at the top level of a component's `config`. It also applies to the observability destination lists `atof.sinks`, `opentelemetry.endpoints`, `opentelemetry.logs.endpoints`, `opentelemetry.metrics.endpoints`, and `atif.storage`. For example, `pricing.sources` and PII redaction `profiles` are top-level component config lists and concatenate across layers. Higher-precedence pricing sources can therefore override one model while still retaining lower-precedence user pricing sources. Lists nested inside arbitrary structured values are not treated as top-level plugin lists; a higher-precedence value replaces those lists. Declare each `kind` at most once inside one `plugins.toml` file. Duplicate component kinds in the same file fail before merge. Duplicate singleton components that reach plugin validation also fail validation. Outside component `config` objects, higher-precedence arrays and scalar values replace lower-precedence values. Tables continue to merge recursively. ## Configuration Layering Plugin settings come from files and code. Files form the base layer, and code sits on top. When the two conflict, code takes precedence. Layering works as follows: 1. Discover and merge the `plugins.toml` files from lowest to highest precedence (explicit-or-user → system), using the [Precedence And Merge Behavior](#precedence-and-merge-behavior) rules above. 2. Layer the config object you pass to `initialize` over that merged base. Any setting it specifies overrides the file value, and the result is the effective config that Relay validates and activates. Programmatic lists participate in the same concatenation rules. For example, a programmatic `config.opentelemetry.endpoints` list appears before endpoints inherited from system and explicit-or-user files; it does not remove those file entries. The same higher-precedence-first order applies to `config.opentelemetry.logs.endpoints` and `config.opentelemetry.metrics.endpoints`. For a log or metric signal, an explicitly empty higher-precedence `endpoints` list clears inherited explicit signal endpoints. An enabled signal then fails validation because its explicit list is empty. Omit the `endpoints` field to derive destinations from trace endpoints instead. When library initialization discovers `plugins.toml` files, Relay emits one `plugin.configuration_inherited` warning per file. Each warning is written to the operational log and included in the initialization result and active plugin report. It names only the source path and does not include configuration values or credentials. Discovery itself does not block initialization, but validation or activation errors in the effective layered configuration still fail normally. Files and code differ only in how they treat a setting you **omit**: | You omit | In a file | In code | | --------------------------------------------------------------------- | -------------------------------------- | -------------------------------------------------------------- | | `version`, `policy`, or the `enabled` flag of a component you declare | Inherited from a lower-precedence file | **Always taken from code** — its default if you did not set it | | A whole component kind, or a key inside a component's `config` | Inherited from a lower-precedence file | Inherited from the file layer | Lower-precedence files fill fields that higher-precedence files omit. Typed code fields always have values, so they override file values. Only component selection and keys inside component `config` merge with files. When a component declared in code resolves `enabled = true` over a discovered `enabled = false`, initialization and the active plugin report include a `plugin.component_reenabled` warning that names the contributing file. Without filesystem access, no files are read, so the base is empty and only your `initialize` config applies. ## Explicit Defaults and Overrides The editor writes explicit defaults for edited Observability and Adaptive sections. It writes NeMo Guardrails, PII Redaction, and dynamic-plugin fields only when readers configure them. Use the NeMo Guardrails editor only to maintain or remove an existing configuration. In a layered config model, omitting a field means "inherit a lower precedence value"; it does not mean "delete that value." Use the dedicated `nemo-relay model-pricing` commands to manage model-pricing catalog sources. For example, this system file disables ATOF even if an explicit or user file enables it: ```toml [[components]] kind = "observability" [components.config.atof] enabled = false ``` The merged configuration can still contain an inherited ATOF `sinks` array, but the runtime ignores the section because `enabled = false`. To override an inherited non-default scalar field with its default value, write the default explicitly in the higher-precedence file. List entries are not merged item by item: higher-precedence entries are added before inherited entries. To change or remove an inherited sink, profile, source, or other list entry, edit the layer that declares it. There is no tombstone syntax for deleting an inherited nested field while keeping the rest of the lower-precedence component. To remove inherited settings entirely, edit the lower-precedence file or override the behavior with another field such as `enabled = false`. ## Validation Plugin validation runs before activation. Invalid plugin config blocks gateway startup instead of starting with a partially installed plugin set. Common validation failures include: * Unknown component kinds when policy treats them as errors. * Unknown fields when policy treats them as errors. * Unsupported field values, such as an invalid exporter mode or transport. * Duplicate singleton components. * Enabled components whose build-time features are unavailable. * Component-specific semantic failures, such as an Agent Trajectory Interchange Format (ATIF) filename template that does not contain `{session_id}`. * Dynamic manifests that have an incompatible Relay version, unsupported capability, invalid load contract, or failed trust evidence. Run `nemo-relay plugins validate ` to check optional static schema validation for dynamic component config. Use `nemo-relay doctor` to inspect the resolved gateway configuration and plugin diagnostics. For Observability, doctor also reports enabled exporter sections, checks writable file exporter directories, probes configured ATOF stream sinks, and checks reachable OTLP endpoints when those settings are present. For model pricing, doctor validates enabled file and inline sources and fails when a source is unreadable or the catalog schema is invalid. For dynamic-plugin validation, lifecycle state, and trust diagnostics, use `nemo-relay plugins validate`, `inspect`, and `list`; doctor reports only the resolved manifest references and host configuration status. ## Relationship to `config.toml` `config.toml` owns gateway and agent setup, such as upstream provider base URLs and agent command configuration. `plugins.toml` owns reusable runtime behavior installed by the plugin system. Keep all long-lived plugin setup in `plugins.toml`. `config.toml` owns gateway and agent setup only. Legacy observability config sections in `config.toml`, such as `[exporters]`, `[observability]`, and `[export.openinference]`, are not supported. Configure Observability exporters through `plugins.toml`. ## Component Guides Use the component guides for field-level configuration: * [Observability Configuration](/configure-plugins/observability/configuration) * [Adaptive Configuration](/configure-plugins/adaptive/configuration) * [Adaptive Cache Governor (ACG)](/configure-plugins/adaptive/acg) * [Adaptive Hints](/configure-plugins/adaptive/adaptive-hints) * [NeMo Guardrails Configuration](/configure-plugins/nemo-guardrails/configuration) * [PII Redaction Configuration](/configure-plugins/pii-redaction/configuration) * [Model Pricing](/configure-plugins/model-pricing)