Instrument Applications

Adding Scopes and Marks

View as Markdown

Use this guide when you want NeMo Relay to identify one agent run, request, workflow, or operation before you instrument individual tool and LLM calls.

What You Build

You will create an active scope, emit named mark events inside that scope, and validate that subscribers can observe the lifecycle. The result is a small trace boundary that later tool calls, LLM calls, middleware, and exporters can attach to.

Scopes give emitted work ownership. Marks record point-in-time checkpoints inside that ownership boundary.

Before You Start

Complete one binding Quick Start guide first:

You should know which application boundary should own the trace. Common scope boundaries include:

  • One HTTP request
  • One agent run
  • One workflow step
  • One background job
  • One tenant-isolated experiment

Integration Pattern

Follow this sequence to keep framework work attached to the expected runtime context.

  1. Choose the scope boundary that owns the work.
  2. Register a temporary subscriber while validating instrumentation.
  3. Push or enter the scope before tool and LLM work begins.
  4. Emit marks for important checkpoints that are not full nested calls.
  5. Close the scope when the request, agent run, or workflow ends.
  6. Confirm that the subscriber sees scope start, mark, and scope end events.

Minimal Example

The examples below create one agent-run scope and emit two marks.

1import nemo_relay
2
3def log_event(event) -> None:
4    print(f"{event.kind} {event.name}")
5
6nemo_relay.subscribers.register("scope-check", log_event)
7
8try:
9    with nemo_relay.scope.scope(
10        "agent-run",
11        nemo_relay.ScopeType.Agent,
12        input={"request_id": "req-123"},
13    ) as handle:
14        nemo_relay.scope.event("planning-started", handle=handle, data={"step": 1})
15        nemo_relay.scope.event("planning-finished", handle=handle, data={"step": 2})
16finally:
17    nemo_relay.subscribers.flush()
18    nemo_relay.subscribers.deregister("scope-check")

When to Add Marks

Use marks for checkpoints that should appear in the event stream but do not wrap a callback:

  • Request accepted
  • Plan created
  • Memory loaded
  • Routing decision made
  • Final answer assembled
  • Retry scheduled

Use a nested scope instead of a mark when the work has a meaningful start and end boundary, child work, or a duration that matters.

Validate the Integration

Check that the subscriber prints:

  • One scope start event for agent-run
  • One mark event for planning-started
  • One mark event for planning-finished
  • One scope end event for agent-run

Native subscribers are delivered asynchronously. Flush subscribers before validating printed or captured output. In Node.js, also wait one event-loop tick after flushSubscribers() so queued JavaScript callbacks can run.

If marks appear outside the intended trace, pass the active scope handle explicitly or make sure the mark is emitted while the scope is active.

Production Checklist

Use this checklist before running the pattern in production traffic.

  • Keep scope names stable enough for filtering and dashboards.
  • Use marks for business checkpoints, not verbose debug logging.
  • Include only JSON-compatible data, metadata, input, and output payloads.
  • Do not attach sensitive payloads unless sanitize guardrails or exporter filters will remove them.
  • Propagate the active scope stack when work crosses thread or worker boundaries.

Next Steps

Use these links to continue from this workflow into the next related task.