Agent Trajectory Observability Format (ATOF)

Use the atof section to write raw Agent Trajectory Observability Format (ATOF) 0.1 events to JSONL files, send them to raw-event collectors, or do both.

File sinks are useful for local debugging, offline inspection, and preserving the canonical event stream before another exporter translates it. Stream sinks send the same events to collectors in near real time. They can also transform dotted object keys before delivery.

plugins.toml Example

Add the following ATOF exporter configuration to plugins.toml:

1
version = 1
2
3
[[components]]
4
kind = "observability"
5
enabled = true
6
7
[components.config]
8
version = 2
9
10
[components.config.atof]
11
enabled = true
12
13
[[components.config.atof.sinks]]
14
type = "file"
15
output_directory = "logs"
16
filename = "events.jsonl"
17
mode = "overwrite"
18
19
[[components.config.atof.sinks]]
20
type = "stream"
21
name = "archive"
22
url = "http://localhost:8080/events"
23
transport = "http_post"
24
timeout_millis = 3000
25
field_name_policy = "replace_dots"
26
27
[components.config.atof.sinks.header_env]
28
authorization = "NEMO_RELAY_ATOF_AUTH_HEADER"

Before you validate or activate this example, set NEMO_RELAY_ATOF_AUTH_HEADER to the complete authorization header value, such as Bearer <token>. NeMo Relay rejects a missing or blank value.

This configuration registers the plugin-managed ATOF exporter and writes one JSON object per lifecycle event to logs/events.jsonl. It also applies the replace_dots field-name policy and sends each raw event to the named archive stream sink.

Fields

The following table describes the top-level ATOF settings:

File Sinks

Each file sink writes the raw event stream to one JSONL file. The following table describes each file sink:

Stream Sinks

Each stream sink receives the same canonical ATOF event that file sinks write. Before sending it, NeMo Relay applies the configured field-name policy and then uses the transport-specific encoding. WebSocket sends a JSON text message, while the HTTP transports send a JSONL record. Each stream runs independently, so a failed connection does not block file output or other streams.

The following table describes each stream sink:

Use header_env for secrets so the configuration stores the environment variable name instead of the secret value.

preserve sends canonical ATOF field names unchanged. replace_dots replaces dots in JSON object keys with underscores recursively. If that creates a duplicate key, NeMo Relay keeps both values and appends _2, _3, and later suffixes in a deterministic order. Validation rejects any other sinks[i].field_name_policy value for a stream sink.

The following transports are supported:

• http_post sends each event as one JSONL record in an HTTP POST request with Content-Type: application/x-ndjson. Any 2xx response is treated as success.
• websocket opens one connection and sends each event as one JSON text message.
• ndjson opens one long-lived HTTP upload and writes each event as one newline-delimited JSON record.

force_flush() flushes file output and drains queued stream events while keeping stream connections open. shutdown() is terminal: it flushes pending work, closes the connections, and ignores events that arrive later.

Migrate From Version 1

Observability component configuration now requires version = 2. NeMo Relay does not translate the removed ATOF fields from version 1.

Make the following changes when you migrate a plugin configuration:

• Move output_directory, filename, and mode from [components.config.atof] into a sink with type = "file".
• Replace each [[components.config.atof.endpoints]] entry with a [[components.config.atof.sinks]] entry that has type = "stream".
• Keep static request headers in headers, or use header_env to read complete header values from environment variables at activation time.

The plugin sends every event to every configured sink. The manual AtofExporter API is different: each exporter owns one file or stream sink.

Expected Output

Each file sink writes an emitted scope, tool, LLM, middleware, or mark event as one ATOF JSON object per line. Stream sinks encode the event as described in Stream Sinks. For event field semantics, refer to Events.

Coding-agent sessions emit a session.start mark with the logical metadata.session_id, a Relay-owned UUIDv7 metadata.session_instance_id, and optional session metadata such as metadata.user_id. Later trace exporters use these identities for correlation, but ATOF preserves them as raw event metadata.

ATOF is the raw-event export path. It preserves the canonical event shape; it does not add semantic extraction, replay policy, scope-owner resolution, or exporter projection rules. Each owning agent scope starts fresh, and a compaction mark refreshes it. The first subsequent LLM start annotation retains complete history. Later starts retain system instructions, the latest user message, and every following assistant or tool message. With a request codec annotation, the emitted event input uses the same projection. If that event-only encoding fails, Relay retains the complete event input and annotation. The request used for provider execution remains unchanged. Refer to Events for the complete freshness contract.

Register the plugin before instrumented work starts. Clear it during shutdown so every sink flushes pending events and stream connections close cleanly.

Plugin Configuration

Use plugin configuration when you want NeMo Relay to manage the ATOF exporter lifecycle. The following examples configure and activate the exporter through each supported language binding.

validate() checks only the supplied in-memory object. initialize() also layers discovered plugins.toml configuration. For effective file-backed validation, refer to Plugin Configuration Files and run the gateway with the same configuration path that production uses.

1
import asyncio
2
3
from nemo_relay import plugin
4
from nemo_relay.observability import (
5
    AtofConfig,
6
    AtofFileSinkConfig,
7
    AtofStreamSinkConfig,
8
    ComponentSpec,
9
    ObservabilityConfig,
10
)
11
12
config = plugin.PluginConfig(
13
    components=[
14
        ComponentSpec(
15
            ObservabilityConfig(
16
                atof=AtofConfig(
17
                    enabled=True,
18
                    sinks=[
19
                        AtofFileSinkConfig(
20
                            output_directory="logs",
21
                            filename="events.jsonl",
22
                            mode="overwrite",
23
                        ),
24
                        AtofStreamSinkConfig(
25
                            url="http://localhost:8080/events",
26
                            transport="http_post",
27
                        ),
28
                    ],
29
                )
30
            )
31
        )
32
    ]
33
)
34
35
report = plugin.validate(config)
36
if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
37
    raise RuntimeError(report["diagnostics"])
38
39
async def main():
40
    await plugin.initialize(config)
41
    try:
42
        # Run instrumented application work here.
43
        pass
44
    finally:
45
        plugin.clear()
46
47
asyncio.run(main())

Manual API

Use the manual AtofExporter API when a test or script needs a custom subscriber name or explicit registration window. A manual exporter owns one sink. To send the same events to several destinations, create and register one exporter for each sink or use the observability plugin.

1
from nemo_relay import AtofExporter, AtofExporterConfig
2
3
config = AtofExporterConfig()
4
config.sink_type = "stream"
5
config.url = "http://localhost:8080/events"
6
config.transport = "http_post"
7
8
exporter = AtofExporter(config)
9
exporter.register("atof-exporter")
10
11
# Run instrumented application work here.
12
13
exporter.force_flush()
14
exporter.deregister("atof-exporter")
15
exporter.shutdown()

Common Configuration and Runtime Issues

Common configuration and runtime issues include:

• ATOF is enabled without at least one plugin-configured sink.
• A file sink’s mode is not append or overwrite.
• A stream sink has an empty url, an unsupported transport, or a timeout_millis value of 0.
• The output directory is not writable at runtime.
• nemo-relay doctor cannot deliver its synthetic ATOF mark probe to a configured stream sink.
• A file sink is configured in a target that cannot access the native filesystem.