> This page is for NeMo Relay, version v0.11.1. > For other versions, use one of these documentation indexes: > - Latest (v0.10.4) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt > - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt > - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt > - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt > - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt > - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt > - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server. # Package Discoverable Plugins > Package a native library or grpc-v1 worker for validated Relay discovery. A discoverable plugin is a package containing one `relay-plugin.toml` manifest, its native library or worker entrypoint, and an optional component JSON Schema. The manifest lets Relay validate compatibility and integrity before code is loaded or a process is started. Declare the package with `[[plugins.dynamic]]` in `plugins.toml` and put its runtime settings under `[plugins.dynamic.config]`. The CLI gateway and SDK initialization read these records and use lifecycle state to select plugins. A manual record can activate at startup when no lifecycle state file exists. Refer to [Dynamic Plugin Configuration](/nemo/relay/v0.11.1/configure-plugins/plugin-configuration-files#gateway-discoverable-plugin-records) for the configuration shape. ## Manifest Responsibilities | Block | What It Controls | | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `[plugin]` | Stable package identity and plugin kind. | | `[compat]` | Supported Relay range plus the native manifest API or worker protocol contract. A typed native plugin built with the current SDK must declare a Relay range beginning at 0.10.0. | | `[defaults]` and `[capabilities]` | The required `enabled = false` declaration and the features the package declares, such as `plugin_native`, `plugin_worker`, and `config_schema`. | | `[config_schema]` | An optional JSON Schema path resolved relative to the manifest. Packages that use it also declare the `config_schema` capability. | | `[source]` | The artifact covered by integrity verification and, for managed Python workers, the package root used to create the environment. | | `[integrity]` | The SHA-256 digest of `source.artifact`, plus optional signature evidence. | | `[load]` | The native library and symbol, Python module entrypoint, or Rust or custom-command executable entrypoint that Relay starts. Worker manifests can also set `forward_env`, `callback_timeouts`, and `max_message_bytes` here. | For native packages, `compat.native_api = "1"` is the authored manifest contract. It is not the [C host-table ABI number](/nemo/relay/v0.11.1/build-plugins/native/native-abi-reference). The current SDK requires ABI v7, and Relay 0.10 rejects v2-v6 compiled callback layouts. For workers, declare `compat.worker_protocol = "grpc-v1"`; the [handshake](/nemo/relay/v0.11.1/build-plugins/workers/grpc-v1-protocol) still negotiates the exact protocol, surfaces, authentication token, and lifecycle at startup. ## Start from a Complete Manifest These are the complete manifest shapes used by the checked examples. The path in `source.artifact` is the file whose bytes the digest covers. The native loader opens that same library; a worker loader either starts the compiled program or imports the Python entrypoint from the managed environment. For application embeddings, core initialization loads a private temporary copy of the native library bytes checked by the host trust policy. The copy remains available until the library unloads. Replacing the source manifest, library, or signature after validation does not change the library loaded by that activation. Native plugins used in embeddings must not depend on adjacent files being available relative to the loaded library's location. The CLI retains its activation snapshot, which includes adjacent native dependencies. Native library digest checks use bounded regular-file reads and reject symbolic links and special files. ```toml title={"Python Worker: relay-plugin.toml"} showLineNumbers={false} manifest_version = 1 [plugin] id = "examples.python_grpc_worker" kind = "worker" [compat] relay = ">=0.10.0,<1.0" worker_protocol = "grpc-v1" [defaults] enabled = false [capabilities] items = ["plugin_worker", "config_schema"] [config_schema] path = "config.schema.json" [source] manifest_root = "." artifact = "nemo_relay_python_grpc_worker_example/worker.py" [integrity] sha256 = "sha256:" [load] runtime = "python" entrypoint = "nemo_relay_python_grpc_worker_example.worker:main" forward_env = [] ``` ```toml title={"Native Rust: relay-plugin.toml"} showLineNumbers={false} manifest_version = 1 [plugin] id = "examples.rust_native_policy" kind = "rust_dynamic" [compat] relay = ">=0.10.0,<1.0" native_api = "1" [defaults] enabled = false [capabilities] items = ["plugin_native", "config_schema"] [config_schema] path = "config.schema.json" [source] artifact = "target/debug/" [integrity] sha256 = "sha256:" [load] library = "target/debug/" symbol = "nemo_relay_register_plugin" ``` ```toml title={"Rust Worker: relay-plugin.toml"} showLineNumbers={false} manifest_version = 1 [plugin] id = "examples.rust_grpc_worker" kind = "worker" [compat] relay = ">=0.10.0,<1.0" worker_protocol = "grpc-v1" [defaults] enabled = false [capabilities] items = ["plugin_worker", "config_schema"] [config_schema] path = "config.schema.json" [source] artifact = "target/debug/" [integrity] sha256 = "sha256:" [load] runtime = "rust" entrypoint = "target/debug/" forward_env = [] ``` The Python package has one extra responsibility: `source.manifest_root` identifies the directory Relay installs into its managed environment. The digest still covers only the declared artifact, so regenerate it whenever `worker.py` changes. A compiled worker has no managed Python environment and starts the executable named by `load.entrypoint`. Both worker examples register LLM execution intercepts, so their Relay compatibility range begins at `0.10.0`. A worker that does not register those surfaces does not need a 0.10 lower bound solely for the additive execution codec context field. The worker protocol remains `grpc-v1`. ## Package and Register the Artifact Use the following procedure to turn the checked manifest template into an artifact that Relay can validate and activate: 1. Build the shared library or worker from a clean checkout and place the entrypoint at the relative path declared by `relay-plugin.toml`. 2. Validate the [component schema](/nemo/relay/v0.11.1/build-plugins/fundamentals/configuration-and-validation) against valid, invalid, and disabled example configurations. Keep unknown-field behavior aligned between the schema and the plugin's validation callback. 3. Generate the artifact digest with `shasum -a 256 ` on macOS, `sha256sum ` on Linux, or `Get-FileHash -Algorithm SHA256 ` in PowerShell. Do this after the final `source.artifact` bytes are in place; changing that artifact invalidates the digest. For example, this Linux command materializes a usable Rust worker manifest without changing the checked template: ```bash cp relay-plugin.toml relay-plugin.local.toml sed -i 's##nemo-relay-rust-grpc-worker-plugin-example#g' \ relay-plugin.local.toml digest="$(sha256sum target/debug/nemo-relay-rust-grpc-worker-plugin-example | cut -d' ' -f1)" sed -i "s#sha256:#sha256:$digest#" relay-plugin.local.toml ``` The local manifest is deliberately untracked. Rebuilding the executable changes its bytes, so recalculate the digest before every validation attempt. 4. Validate the materialized manifest. For the compiled example above, run `nemo-relay plugins validate ./relay-plugin.local.toml`. The Python example keeps its checked digest and uses `./relay-plugin.toml` directly. A successful command confirms manifest structure, compatibility syntax, entrypoint resolution, schema loading, and integrity metadata without activating the component. 5. Register the same path with `nemo-relay plugins add --user `, or add an explicit `[[plugins.dynamic]]` reference to the intended `plugins.toml`. 6. Activate a valid component, inspect the runtime report, execute a representative managed call, then clear and unload or stop the plugin. Success means package validation passes before activation, tampering causes integrity validation to fail, the runtime report identifies the dynamic component, and teardown removes its registrations before the library unloads or worker exits. The operator-side trust policy and file layout are documented in [Manage](/nemo/relay/v0.11.1/configure-plugins/discoverable-plugins). ## Forward Host Environment Variables Add `load.forward_env` to `relay-plugin.toml` to pass selected host environment variables to a managed worker. This setting supports Python, Rust, and workers that Relay starts with a custom command. The following example selects one variable and all variables that start with `EXAMPLE_CONFIG_`: ```toml [load] runtime = "python" entrypoint = "example_plugin" forward_env = ["EXAMPLE_CONTEXT_ID", "EXAMPLE_CONFIG_*"] ``` This option requires Relay 0.11 or later. Set `compat.relay` to a range such as `>=0.11.0,<1.0` if your plugin needs forwarded variables. Names must start with an ASCII letter or underscore. The rest of the name can contain ASCII letters, digits, and underscores. An optional `*` at the end matches any suffix, including an empty suffix. You can't use `*` alone or leave an entry empty. Names are case-sensitive on Unix. Windows ignores letter case. Relay reads the selected variables from the host process environment when it starts the worker. It does not read Windows registry settings or shell profiles. It also does not read `launchd` or `systemd` configuration. Relay skips missing variables. It keeps these host variables when they exist: `PATH`, `SYSTEMROOT`, `WINDIR`, `TMPDIR`, `TEMP`, `TMP`, `SSL_CERT_FILE`, and `SSL_CERT_DIR`. If you omit `forward_env` or set it to `[]`, Relay forwards no other host variables. Reactivate the plugin to apply changes. The plugin author sets this option in the manifest. Component settings stay in `plugins.toml`. Relay stores only the names and patterns in the plugin record. It does not show forwarded values in plugin inspection output. Relay supplies its own token, endpoints, and message limits. Forwarded values cannot replace these settings. Worker plugins are trusted code. Running them in a separate process does not make them a security sandbox. Relay does not block any selected variable by name. Choose specific names or narrow prefixes. The plugin can read any credentials you forward. Settings such as `PYTHONPATH` can change how the worker starts. ## Worker Message Limits Set `[load.max_message_bytes]` in `relay-plugin.toml` with an optional `default_bytes` and `by_name` overrides for exact callback registration names. Keep these worker settings separate from `plugins.dynamic.config`. The following example sets a default limit and a larger limit for one callback: ```toml title={"relay-plugin.toml"} showLineNumbers={false} [load.max_message_bytes] default_bytes = 4194304 [load.max_message_bytes.by_name] input_guardrail = 16777216 ``` Limits must be whole numbers from 1 through 2,147,483,647 bytes. If you omit `default_bytes`, Relay uses 4 MiB. Each limit applies to each sent or received message, including stream chunks and calls to continuations. It does not limit the total stream size. Control RPCs use the default limit. Zero is invalid and does not disable the limit. Reactivate the plugin after changing its manifest or host environment. For example, `by_name.input_guardrail = 16777216` raises the registered `input_guardrail` callback and its continuations to 16 MiB. Host environment settings override plugin settings. Refer to the [execution message-size contract](/nemo/relay/v0.11.1/build-plugins/workers/grpc-v1-protocol#configure-execution-message-size) for validation, compatibility, and memory considerations. ## Callback Deadlines A deadline limits how long Relay waits for a callback to finish. Set deadlines for short worker callbacks in `relay-plugin.toml`. These settings work with Python, Rust, and workers that you start with a custom command. The following example sets a default deadline and an override for one callback: ```toml title={"relay-plugin.toml"} showLineNumbers={false} [load.callback_timeouts] default_ms = 60000 [load.callback_timeouts.by_name] check_request = 90000 ``` Use positive whole numbers in milliseconds. In this example, most short callbacks have a limit of 60,000 milliseconds. The `check_request` callback has a limit of 90,000 milliseconds. Relay chooses the deadline in this order: 1. The value in `by_name` for the callback's exact registration name. 2. The `default_ms` value, if set. 3. The built-in default of 30,000 milliseconds. Names are case-sensitive. They cannot be empty or have spaces at the start or end. If several callbacks use the same name, they share the same override, even across components or callback types. Use distinct names when callbacks need different deadlines. Names that don't match a registered callback have no effect. For a conditional middleware guardrail, use the guardrail's callback name. Plugin authors and operators use the same manifest settings. Edit the manifest and reactivate the plugin to apply a change. Relay keeps the settings it read when the plugin started until you reactivate it. The deadline includes any time the callback spends waiting for calls back to Relay. When the deadline expires, Relay asks the worker to cancel the callback and clears the host state for that call. The callback follows its usual error or fallback behavior. Tool and LLM execution intercepts, including streams, ignore these settings. Their callers control cancellation. Other worker calls, such as validation and shutdown, keep their 30,000-millisecond timeout. Native plugins run inside Relay and don't use worker deadlines.