For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
  • About NVIDIA NeMo Relay
    • Overview
    • Architecture
    • Ecosystem
    • Concepts
    • Release Notes
  • Getting Started
    • Agent Runtime Primer
    • Prerequisites
    • Installation
    • Configuration / Setup
    • Quick Start
  • NVIDIA NeMo Relay CLI
    • About
    • Basic Usage
    • Claude Code
    • Codex
    • Cursor
    • Hermes Agent
  • Supported Integrations
    • About
    • OpenClaw Plugin Guide
    • LangChain Integration Guide
    • LangGraph Integration Guide
    • Deep Agents Integration Guide
  • Instrument Applications
    • About
    • Adding Scopes and Marks
    • Instrument a Tool Call
    • Instrument an LLM Call
    • Add Middleware
    • Code Examples
  • Observability Plugin
    • About
    • Configuration
    • Agent Trajectory Interchange Format (ATIF)
    • Agent Trajectory Observability Format (ATOF)
    • OpenTelemetry
    • OpenInference
  • Adaptive Plugin
    • About
    • Configuration
    • Adaptive Cache Governor (ACG)
    • Adaptive Hints
  • NeMo Guardrails Plugin
    • About
    • Configuration
  • Integrate into Frameworks
    • About
    • Adding Scopes
    • Wrap Tool Calls
    • Wrap LLM Calls
    • Handle Non-Serializable Data
    • Using Codecs
    • Provider Codecs
    • Provider Response Codecs
    • Code Examples
  • Build Plugins
    • About
    • Define a Plugin
    • Validate Plugin Configuration
    • Plugin Configuration Files
    • Register Plugin Behavior
    • Design Plugin Configuration
    • NeMo Guardrails Example Plugin
    • Code Examples
  • Contribute
    • About
    • Development Setup
    • Workflow and Reviews
    • Testing and Documentation
  • Reference
    • APIs
    • Performance
  • Resources
    • Support and FAQs
    • Glossary
    • Troubleshooting Guide
    • Community
    • Legal
NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.

LogoLogo
On this page
  • What It Installs
  • plugins.toml Example
  • Per-Language Plugin Configuration
  • Validation And Teardown
Observability Plugin

Observability Configuration

||View as Markdown|
Previous

Observability

Next

Agent Trajectory Interchange Format (ATIF)

Use this page when an application should install standard observability exporters from one plugin configuration document instead of manually registering each subscriber.

The plugin kind is observability. It is registered by the core runtime, so applications do not need to register a plugin implementation before validation or initialization.

For plugin file discovery, precedence, merge behavior, editor controls, and gateway conflict rules, see Plugin Configuration Files.

Observability plugin configuration uses the generic NeMo Relay plugin document shape, so field names are snake_case in every binding. This differs from Node.js runtime classes such as OpenTelemetrySubscriber, which use Node-native camelCase option names outside the plugin system.

What It Installs

Every exporter section is optional and defaults to disabled. A section is active only when it includes enabled: true.

SectionRuntime behavior
atofRegisters a global Agent Trajectory Observability Format (ATOF) JSONL exporter for raw lifecycle events.
atifRegisters one Agent Trajectory Interchange Format (ATIF) dispatcher that writes one trajectory file for each top-level agent scope.
opentelemetryRegisters a global OpenTelemetry OTLP subscriber.
openinferenceRegisters a global OpenInference OTLP subscriber.

subscriber_name is not part of this config. The runtime infers component-local subscriber names and registers them under the observability plugin namespace:

  • Agent Trajectory Observability Format (ATOF): __nemo_relay_plugin__observability__atof
  • Agent Trajectory Interchange Format (ATIF) dispatcher: __nemo_relay_plugin__observability__atif
  • Per-agent ATIF scope subscriber: __nemo_relay_plugin__observability__atif-{agent_scope_uuid}
  • OpenTelemetry: __nemo_relay_plugin__observability__opentelemetry
  • OpenInference: __nemo_relay_plugin__observability__openinference

plugins.toml Example

1version = 1
2
3[[components]]
4kind = "observability"
5enabled = true
6
7[components.config]
8version = 1
9
10[components.config.atof]
11enabled = true
12output_directory = "logs"
13filename = "events.jsonl"
14mode = "overwrite"
15
16[components.config.atif]
17enabled = true
18output_directory = "logs"
19filename_template = "trajectory-{session_id}.json"
20
21[components.config.opentelemetry]
22enabled = true
23transport = "http_binary"
24endpoint = "http://localhost:4318/v1/traces"
25service_name = "nemo-relay"
26service_namespace = "agent"
27service_version = "0.4.0"
28instrumentation_scope = "nemo-relay-observability"
29timeout_millis = 3000
30
31[components.config.opentelemetry.headers]
32authorization = "Bearer <token>"
33
34[components.config.opentelemetry.resource_attributes]
35"deployment.environment" = "dev"
36"service.instance.id" = "local"
37
38[components.config.openinference]
39enabled = true
40transport = "http_binary"
41endpoint = "http://localhost:6006/v1/traces"
42service_name = "nemo-relay"
43service_namespace = "agent"
44service_version = "0.4.0"
45instrumentation_scope = "nemo-relay-openinference"
46timeout_millis = 3000
47
48[components.config.openinference.headers]
49authorization = "Bearer <token>"
50
51[components.config.openinference.resource_attributes]
52"deployment.environment" = "dev"
53"service.instance.id" = "local"
54
55[components.config.policy]
56unknown_component = "warn"
57unknown_field = "warn"
58unsupported_value = "error"

Include only the sections you want to configure. In layered plugins.toml files, omission inherits lower-precedence values; write enabled = false to disable an inherited section.

Per-Language Plugin Configuration

Python
Node.js
Rust
1import asyncio
2
3from nemo_relay import plugin, scope, ScopeType
4from nemo_relay.observability import (
5 AtifConfig,
6 AtofConfig,
7 ComponentSpec,
8 ObservabilityConfig,
9 OtlpConfig,
10)
11
12config = plugin.PluginConfig(
13 components=[
14 ComponentSpec(
15 ObservabilityConfig(
16 atof=AtofConfig(
17 enabled=True,
18 output_directory="logs",
19 filename="events.jsonl",
20 mode="overwrite",
21 ),
22 atif=AtifConfig(
23 enabled=True,
24 output_directory="logs",
25 filename_template="trajectory-{session_id}.json",
26 ),
27 opentelemetry=OtlpConfig(
28 enabled=True,
29 endpoint="http://localhost:4318/v1/traces",
30 service_name="nemo-relay",
31 service_namespace="agent",
32 service_version="0.4.0",
33 instrumentation_scope="nemo-relay-observability",
34 resource_attributes={"deployment.environment": "dev"},
35 ),
36 openinference=OtlpConfig(
37 enabled=True,
38 endpoint="http://localhost:6006/v1/traces",
39 service_name="nemo-relay",
40 service_namespace="agent",
41 service_version="0.4.0",
42 instrumentation_scope="nemo-relay-openinference",
43 resource_attributes={"deployment.environment": "dev"},
44 ),
45 )
46 )
47 ]
48)
49
50report = plugin.validate(config)
51if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
52 raise RuntimeError(report["diagnostics"])
53
54async def arun():
55 async with plugin.plugin(config):
56 with scope.scope("agent", ScopeType.Agent):
57 pass
58
59asyncio.run(arun())

Validation And Teardown

Validate plugin configuration before activating it. The plugin reports unsupported transports, unsupported ATOF modes, unsafe ATIF filename templates, unknown fields according to policy, and enabled exporters that are unavailable in the current build or target.

Call plugin.clear() or clear_plugin_configuration() during teardown. Clearing the plugin config deregisters inferred subscribers, flushes file exporters, and shuts down owned OTLP subscribers.

Use manual subscriber/exporter APIs instead of the plugin when you need custom subscriber names, explicit per-run exporter objects, or direct control over the collection window.