Build Plugins

Define a Plugin

View as Markdown

Use this guide when you want to package reusable NeMo Relay behavior as a plugin that can be activated from configuration.

What You Build

You will define the plugin’s purpose, stable kind name, configuration boundary, runtime surfaces, and activation lifecycle. The result is a small plugin contract that can be validated and registered through the more focused follow-on guides.

NeMo Relay plugin configuration keys use snake_case in every language and file format. Node.js helper function names are camelCase, but the objects passed to plugin.initialize(...) use the same canonical snake_case keys as Python, Rust, JSON, and TOML plugin configuration.

Before You Start

You need:

  • A reusable behavior that belongs outside one application call site.
  • A stable plugin kind name.
  • A JSON-compatible config shape.
  • A decision about which runtime surfaces the plugin installs.
  • A teardown plan for tests and applications that need to clear active configuration.

Plugin Shape and Requirements

A plugin needs a stable shape before operators can activate it from config:

RequirementWhy It Matters
Stable kindThe plugin registry uses this string to match config to implementation.
JSON-compatible configConfig must move across Python, Node.js, Rust, files, tests, and deployment systems.
Validation hookOperators need diagnostics before runtime behavior changes.
Registration hookRuntime behavior should be installed through PluginContext for name qualification and rollback.
Runtime ownershipThe plugin should clearly own subscribers, middleware, adaptive behavior, or a small bundle of related surfaces.

Keep runtime objects out of config. Provider clients, callbacks, file handles, caches, and credentials should be created inside plugin code or resolved from safe references during registration.

What a Plugin Can Install

A plugin can install one or more of these runtime surfaces:

  • Subscribers
  • Tool sanitize-request guardrails
  • Tool sanitize-response guardrails
  • Tool conditional-execution guardrails
  • Tool request intercepts
  • Tool execution intercepts
  • LLM sanitize-request guardrails
  • LLM sanitize-response guardrails
  • LLM conditional-execution guardrails
  • LLM request intercepts
  • LLM execution intercepts
  • LLM stream execution intercepts

Start with one surface. Add a bundle only when one configuration document clearly controls related behavior, such as a subscriber plus the request intercepts needed to add correlation metadata.

Registration Lifecycle

The diagram below shows how plugin configuration turns into registered runtime behavior.

The lifecycle is staged: register the plugin kind, validate component config, initialize enabled components, and let PluginContext install runtime behavior. If registration fails partway through, the plugin system can roll back partial setup.

Keep the First Plugin Small

The easiest first plugin is one of these:

  • A subscriber-oriented plugin that exports events.
  • A request-intercept plugin that adds one provider header.
  • A sanitize guardrail plugin that redacts one field family.
  • A policy plugin that registers one conditional-execution guardrail.

Avoid a first plugin that combines unrelated subscribers, request transforms, policy checks, and adaptive behavior. Multi-surface bundles are useful later, but they need stronger validation and rollout controls.

Minimal Config Contract

The top-level config document has version, components, and policy. Each component chooses a plugin kind and passes component-local JSON config to that plugin.

1{
2  "version": 1,
3  "components": [
4    {
5      "kind": "header-plugin",
6      "enabled": true,
7      "config": {
8        "header_name": "x-tenant",
9        "value": "tenant-a"
10      }
11    }
12  ],
13  "policy": {
14    "unknown_component": "warn",
15    "unknown_field": "warn",
16    "unsupported_value": "error"
17  }
18}

Use this document as the boundary between operator intent and plugin implementation. Keep business logic in the plugin code, not in the config parser.

Design Checklist

Before you write the plugin implementation, answer these questions:

  • What is the stable plugin kind?
  • What runtime surface does it install first?
  • Which config fields are required?
  • Which fields are safe to expose as JSON?
  • What diagnostic should appear when each required field is missing?
  • What should happen when the component is disabled?
  • What should happen when registration fails halfway through?

Next Steps

Use these links to continue from this workflow into the next related task.