Scopes

This page explains how scope stacks establish ownership, parentage, cleanup, and isolation.

Why Scopes Exist

Scopes track where work belongs in NeMo Relay. Every tool call, LLM call, and mark event attaches to a scope hierarchy.

That hierarchy lets the runtime:

• Model nested agent work
• Preserve parent-child relationships
• Expose scope-local middleware and subscribers
• Clean up scope-owned runtime state automatically
• Isolate concurrent work

What a Scope Represents

A scope represents a logical unit of work such as:

• An agent run
• A request
• A workflow step
• A background task
• A nested function or tool workflow

Scopes are not just labels. They determine event parentage and which local middleware and subscribers are visible.

Scope Hierarchy and Ownership

Scopes form a tree. A child scope inherits the active context from its parent and contributes new nested work beneath it.

That hierarchy determines:

• Event parentage
• Lifetime boundaries
• Scope-local middleware visibility
• Scope-local subscriber visibility

Scope Types

NeMo Relay includes standard scope types for common runtime semantics, including:

• Agent
• Function
• Tool
• Llm
• Retriever
• Embedder
• Reranker
• Guardrail
• Evaluator
• Custom
• Unknown

The specific type helps subscribers and downstream tracing systems understand what the scope represents semantically.

Scope Behavior

These scope behaviors define how root, child, and scope-local runtime state interact.

Root Scope

A root scope is always present. Other scopes are pushed beneath that root as work becomes more specific.

Parent-Child Relationships

Nested scopes create the ownership tree used by emitted events. Tool and LLM calls then attach beneath the active scope.

Scope Lifetimes

Scopes have explicit lifetime boundaries. A scope starts when it becomes active and ends when it is popped or closed.

Replay Timestamps

You can pass timestamps to scope push and pop operations to replay events. Python datetime values must include a time zone. Agent Trajectory Observability Format (ATOF) events keep the supplied timestamp.

OpenTelemetry traces and logs encode timestamps as unsigned 64-bit nanoseconds since the Unix epoch. Their supported range is 1970-01-01T00:00:00Z through 2554-07-21T23:34:33.709551615Z, inclusive. The trace and log subscribers drop events outside this range and report an otel.timestamp_out_of_range runtime diagnostic. The scope operation still completes, and ATOF keeps the original timestamp. On Windows, trace and log timestamps have 100-nanosecond precision.

A dropped start event creates no trace span. A dropped end event does not close an existing span. A later valid end event can close it. A dropped mark creates no span event, standalone span, or log record.

Scope-Local Cleanup

Scope-local middleware and subscribers are tied to the owning scope lifecycle. When the scope closes, those registrations disappear automatically.

Worked Scope Lifetime

Consider an agent that calls one tool and then one LLM:

1. The application or framework integration pushes an Agent scope beneath the root and emits its start event.
2. The application registers middleware and a subscriber on the agent scope. Both are visible while the agent or any of its nested scopes are active.
3. A managed tool wrapper emits the tool start event, runs the tool, and emits the tool end event. The agent remains the active scope.
4. The application emits a mark under the active agent scope. The mark does not change the stack.
5. A managed LLM wrapper emits the LLM start event, runs the model call, and emits the LLM end event. The agent remains the active scope.
6. The application or framework integration emits the agent end event and pops the agent scope. Its local middleware and subscriber are removed.

The active scope stack changes over time. The emitted events remain in a parent-linked tree after their scopes close. A mark event attaches beneath the active scope but does not push another scope onto the stack.

Agent-local middleware and subscribers are visible during the first four stack states. They are no longer visible after the agent scope closes.

Semantic Payloads

Scopes may expose semantic input and output payloads on their emitted start and end events.

Scope Input

Use scope input when the scope itself represents a request-style or task-style unit of work whose starting payload matters semantically.

Scope Output

Use scope output when the scope itself produces a meaningful semantic result.

Those payloads live on the emitted events rather than on the scope handle itself.

Context Isolation

Context isolation keeps concurrent requests, tenants, and agents from sharing scope- local state accidentally.

Choose the context behavior based on whether the work belongs to the same logical trace and whether it runs concurrently:

Why Isolation Matters

Concurrent requests must not share the same active scope stack accidentally. Otherwise:

• Unrelated work can appear under the wrong parent
• Scope-local middleware can leak across requests
• Scope-local subscribers can observe the wrong execution tree

Reuse an Existing Logical Trace

Reuse or propagate the active scope stack when detached work should continue the same logical request or agent trace.

Use this when:

• Worker events should appear under the same parent request
• Scope-local middleware from the parent should still apply
• Subscribers should observe one continuous execution tree

Start a Fresh Isolated Context

Create and bind a fresh stack when detached work should be independent.

Use this when:

• The worker is a separate job rather than part of the parent trace
• The boundary cannot safely carry a native stack handle
• You want a clean root scope with isolated scope-local registrations

Fork Concurrent Work

Fork a scope stack before starting concurrent work when the child should remain part of the parent’s event tree without sharing its mutable stack. The fork preserves the immediate parent but does not transfer scope-local middleware or subscribers. The fork also preserves the captured OpenTelemetry parent, so its work continues the parent’s exported trace.

import asyncio
import nemo_relay
async def worker() -> None:
    with nemo_relay.scope.scope("worker", nemo_relay.ScopeType.Function):
        await asyncio.sleep(0)
async def main() -> None:
    with nemo_relay.scope.scope("parent", nemo_relay.ScopeType.Agent):
        await asyncio.create_task(
            worker(),
            context=nemo_relay.fork_asyncio_context(),
        )

Cross-Process Propagation

When work crosses a process or remote-workflow boundary, applications can carry the versioned Relay propagation context instead of a native stack handle. The context contains an immediate parent_uuid and an optional root_uuid. Rooted contexts carry a root UUID. Explicit rootless contexts carry None. By default, Relay uses the propagated root or the first local Agent scope as that root.

The receiver creates a fresh isolated stack from that context and installs it only for request handling. Its first local event becomes a child of parent_uuid. Relay does not transfer scope-local middleware or subscribers. The transport is application-owned: authenticate and authorize inbound context before importing it. Relay does not send headers, make IPC connections, or trust remote identifiers automatically.

The default capture_propagation_context() API preserves Relay event parentage and includes the exact active OpenTelemetry context when one is available. When no valid W3C parent is available, Relay can reconstruct a Relay-derived trace from the root UUID. An explicit rootless context stays rootless until a local Agent scope establishes a new root. Rootless capture omits W3C context derived from the sender’s local scope, so importing a locally captured rootless context starts a separate OpenTelemetry trace. capture_propagation_context_with_root() lets an application choose a stable run UUID explicitly.

Relay’s JSON context can also carry optional W3C traceparent and tracestate fields. With a valid traceparent, the first local OpenTelemetry span is a child of the upstream span and keeps its trace ID and sampling flag. Relay keeps its UUID lineage unchanged.

Relay drops invalid W3C fields and logs a warning without rejecting the Relay context. If only tracestate is invalid, Relay keeps the valid traceparent. Valid vendor state keeps its member order. Each value must contain 1 to 256 printable ASCII characters. Keys must be unique, and the list can contain at most 32 members. Relay rejects values with control characters, such as carriage returns and newlines.

For outbound-only W3C propagation, use capture_traceparent() (captureTraceparent() in Node.js) to obtain the current Relay traceparent directly, or convert a PropagationContext with PropagationContext::to_traceparent (and the equivalent binding methods). Capture requires an active Relay scope or a valid imported trace context. It does not require an exporter or collector. A managed LLM call closes its LLM scope before returning. Its completed span does not remain the current context. Without an enclosing active scope or imported context, capture returns an error.

In these examples, the managed LLM call has finished when capture runs, but the enclosing Agent scope is still active:

from nemo_relay import ScopeType, capture_traceparent, llm, scope
async def run_agent(request, provider):
    with scope.scope("agent-run", ScopeType.Agent):
        result = await llm.execute("provider", request, provider)
        # The LLM scope has closed. The Agent scope is still active.
        agent_traceparent = capture_traceparent()
        return result, agent_traceparent

Each example captures the enclosing Agent context. To retain the LLM context instead, capture it inside the provider callback while the managed LLM scope is active and save that value for later propagation.

When a W3C context was imported upstream, rootless capture preserves that context and the receiver continues the upstream trace. To require a new trace even then, remove traceparent and tracestate before importing the context. Managed LLM execution automatically replaces both runtime-owned traceparent and tracestate headers on the provider request. A standalone request-intercept call adds the same pair only when a real Relay trace context exists.

Use the binding’s JSON helpers at the transport boundary: Rust PropagationContext::to_json and PropagationContext::from_json, Python context.to_json() and PropagationContext.from_json(...), Go context.ToJSON() and PropagationContextFromJSON(...), or Node.js propagationContextToJson(...) and propagationContextFromJson(...). The helpers validate the version and UUIDs before a context is imported.

Practical Guidance

Use these practices when applying the concept in application or integration code.

• Push a top-level scope at the entry point of a request, workflow, or agent run.
• Let nested helpers attach work beneath that scope whenever possible.
• Use scope-local registrations when the behavior should disappear with the owning scope.
• Emit mark events for retries, checkpoints, interrupts, or state transitions that are important for debugging but are not full spans.
• Prefer explicit isolation decisions when work crosses thread, task, or worker boundaries.