Build PluginsLanguage Binding

Language Binding Plugins

View as Markdown

Use this guide to build an in-process, code-driven plugin in Rust, Python, or Node.js. The application registers the plugin kind with the loaded language binding, then initializes it with the same component configuration model used by built-in plugins.

This model does not load a shared library or start a worker process. For those models, use Native Dynamic Plugins (Rust), gRPC Worker Plugins (Rust), or gRPC Worker Plugins (Python).

What You Build

Define the plugin’s purpose, stable kind name, configuration boundary, runtime surfaces, and activation lifecycle. Then use the focused guides to validate and register the resulting plugin contract.

NeMo Relay plugin configuration keys use snake_case in every language and file format. Node.js helper function names are camelCase, but the objects passed to plugin.initialize(...) use the same canonical snake_case keys as Python, Rust, JSON, and TOML plugin configuration.

Before You Start

You need:

  • A reusable behavior that belongs outside one application call site.
  • A stable plugin kind name.
  • A JSON-compatible config shape.
  • A decision about which runtime surfaces the plugin installs.
  • A teardown plan for tests and applications that need to clear active configuration.

Plugin Shape and Requirements

A plugin needs a stable shape before operators can activate it from config:

RequirementWhy It Matters
Stable kindThe plugin registry uses this string to match config to implementation.
JSON-compatible configConfig must move across Python, Node.js, Rust, files, tests, and deployment systems.
Validation hookOperators need diagnostics before runtime behavior changes.
Registration hookRegister runtime behavior through PluginContext so Relay can qualify names and roll back failed setup.
Runtime ownershipThe plugin should clearly own subscribers, middleware, or a small bundle of related surfaces.

Keep runtime objects out of config. Create provider clients, callbacks, file handles, caches, and credentials in plugin code, or resolve them from safe references during registration.

What a Plugin Can Install

A plugin can install one or more of these runtime surfaces:

  • Subscribers: Event subscribers.
  • Tool middleware: Sanitize-request and sanitize-response guardrails, conditional-execution guardrails, request intercepts, and execution intercepts.
  • LLM middleware: Sanitize-request and sanitize-response guardrails, conditional-execution guardrails, request intercepts, execution intercepts, and stream execution intercepts.

Start with one surface. Add a bundle only when one configuration document clearly controls related behavior, such as a subscriber plus the request intercepts needed to add correlation metadata.

Registration Lifecycle

The diagram below shows how plugin configuration turns into registered runtime behavior.

The lifecycle works in stages. Register the plugin kind, validate component config, and initialize enabled components. PluginContext installs runtime behavior. If registration fails partway through, Relay rolls back the partial setup.

Keep the First Plugin Small

The easiest first plugin is one of these:

  • A subscriber-oriented plugin that exports events.
  • A request-intercept plugin that adds one provider header.
  • A sanitize guardrail plugin that redacts one field family.
  • A policy plugin that registers one conditional-execution guardrail.

Avoid a first plugin that combines unrelated subscribers, request transforms, and policy checks. Multi-surface bundles are useful later, but they need stronger validation and rollout controls. Refer to Adaptive Configuration when you need Adaptive behavior.

Minimal Config Contract

The top-level config document has version, components, and policy. Each component chooses a plugin kind and passes component-local JSON configuration to that plugin. The following document shows the complete shape:

1{
2  "version": 1,
3  "components": [
4    {
5      "kind": "header-plugin",
6      "enabled": true,
7      "config": {
8        "header_name": "x-tenant",
9        "value": "tenant-a"
10      }
11    }
12  ],
13  "policy": {
14    "unknown_component": "warn",
15    "unknown_field": "warn",
16    "unsupported_value": "error"
17  }
18}

Use this document as the boundary between operator intent and plugin implementation. Keep business logic in the plugin code, not in the config parser.

Design Checklist

Before you write the plugin implementation, answer these questions:

  • What is the stable plugin kind?
  • What runtime surface does it install first?
  • Which config fields are required?
  • Which fields are safe to expose as JSON?
  • What diagnostic should appear when each required field is missing?
  • What should happen when the component is disabled?
  • What should happen when registration fails halfway through?

Next Steps

Use these links to continue from this workflow into the next related task.