OpenInference

Use the openinference section when you want NeMo Relay lifecycle events exported as OTLP trace spans with OpenInference-oriented semantics.

OpenInference export maps model-centric payloads directly into trace attributes. Scope, tool, and LLM start inputs become input.value; end outputs become output.value; LLM usage metadata maps to token-count attributes when the provider response includes usage information. For LLM spans, NeMo Relay emits flattened request and response message attributes from typed codec annotations and supported replay request/response payloads. Typed codec annotations also provide tool schema, finish-reason, and invocation-parameter attributes when those fields are available.

plugins.toml Example

1
version = 1
2
3
[[components]]
4
kind = "observability"
5
enabled = true
6
7
[components.config]
8
version = 1
9
10
[components.config.openinference]
11
enabled = true
12
transport = "http_binary"
13
endpoint = "http://localhost:6006/v1/traces"
14
service_name = "agent-service"
15
service_namespace = "nemo"
16
service_version = "1.0.0"
17
instrumentation_scope = "nemo-relay-openinference"
18
timeout_millis = 3000
19
20
[components.config.openinference.headers]
21
authorization = "Bearer <token>"
22
23
[components.config.openinference.resource_attributes]
24
"deployment.environment" = "dev"

This configuration registers a plugin-owned OpenInference subscriber and sends OpenInference-style OTLP spans to Phoenix or another compatible backend.

Fields

OpenInference uses the same OTLP section shape as OpenTelemetry:

Expected Output

The backend should show OpenInference-oriented spans for scopes, tools, and LLM calls grouped by root scope. LLM usage metadata appears as token counters when provider responses include usage information. LLM request and response messages, system prompts, and model-emitted tool calls are emitted as flattened OpenInference attributes when available from codec annotations or supported replay request/response payloads. Tool schemas, finish reasons, and invocation parameters are emitted when typed codec annotations supply them. Exported LLM attributes exclude request headers and other non-observable transport metadata.

Each lifecycle span includes nemo_relay.uuid and nemo_relay.parent_uuid attributes. These values match ATIF step.extra.ancestry.function_id and step.extra.ancestry.parent_id for the same events. For plugin-managed ATIF, the root agent span’s nemo_relay.uuid also matches the ATIF session_id. Backend-native trace_id and span_id values are not written into ATIF.

Redact sensitive event payloads with sanitize guardrails before production export.

Plugin Configuration

Use plugin configuration when the application should let NeMo Relay own the OpenInference subscriber lifecycle.

1
from nemo_relay import plugin
2
from nemo_relay.observability import ComponentSpec, ObservabilityConfig, OtlpConfig
3
4
config = plugin.PluginConfig(
5
    components=[
6
        ComponentSpec(
7
            ObservabilityConfig(
8
                openinference=OtlpConfig(
9
                    enabled=True,
10
                    transport="http_binary",
11
                    endpoint="http://localhost:6006/v1/traces",
12
                    service_name="agent-service",
13
                    service_namespace="nemo",
14
                    service_version="1.0.0",
15
                    instrumentation_scope="nemo-relay-openinference",
16
                    resource_attributes={"deployment.environment": "dev"},
17
                    headers={"authorization": "Bearer <token>"},
18
                )
19
            )
20
        )
21
    ]
22
)
23
24
report = plugin.validate(config)
25
if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
26
    raise RuntimeError(report["diagnostics"])
27
28
async with plugin.plugin(config):
29
    # Run instrumented application work here.
30
    pass

Manual API

Use the manual subscriber API when you need an explicit subscriber name or direct force_flush control.

1
from nemo_relay import OpenInferenceConfig, OpenInferenceSubscriber
2
3
config = OpenInferenceConfig()
4
config.transport = "http_binary"
5
config.endpoint = "http://localhost:6006/v1/traces"
6
config.service_name = "agent-service"
7
config.set_resource_attribute("deployment.environment", "dev")
8
9
subscriber = OpenInferenceSubscriber(config)
10
subscriber.register("openinference-exporter")
11
12
# Run instrumented application work here.
13
14
subscriber.force_flush()
15
subscriber.deregister("openinference-exporter")
16
subscriber.shutdown()

Common Validation Failures

• transport is not http_binary or grpc.
• Headers or resource attributes are not string-to-string maps.
• The OpenInference feature is unavailable in the current build or target.
• Tool and LLM calls do not use managed helpers, so spans contain only scope lifecycle data.