> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/lens/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/lens/_mcp/server.

# Context Propagation

When a traced request crosses a process boundary (such as an HTTP call, gRPC message, message queue, or Ray remote call), its **trace context** (trace ID, parent span ID, and baggage) must travel with it. Otherwise, downstream spans end up in a different trace, and the cross-service waterfall is lost.

NeMo Lens exposes two primitives for this, matching the OTel W3C TraceContext and Baggage propagators.

## `inject_context` for Outbound Calls

```python
from nemo.lens import inject_context

headers = {}
inject_context(headers)
# headers == {'traceparent': '00-<trace_id>-<span_id>-01'}  # plus 'tracestate'/'baggage' only if present

await http_client.post(url, headers=headers, json=body)
```

`inject_context(carrier)` writes the current OTel context into the `carrier` dict. The carrier is whatever your transport uses, such as HTTP headers, gRPC metadata, or message attributes.

## `extract_context` for Inbound Calls

```python
from nemo.lens import extract_context
from opentelemetry import trace, context

# On the receiving side
ctx = extract_context(request.headers)
token = context.attach(ctx)
try:
    with trace.get_tracer(__name__).start_as_current_span("handle.request"):
        ...
finally:
    context.detach(token)
```

`extract_context(carrier)` parses W3C headers and returns an OTel `Context`. Attach it before starting child spans, detach when done.

If the carrier has no valid trace context, the returned `Context` is empty, so new spans start a fresh trace, which is the correct behavior.

## Auto-Instrumentation for Common Transports

Writing `inject_context` or `extract_context` by hand on every call is error-prone. For common frameworks, NeMo Lens ships auto-instrumentation helpers:

* **FastAPI**: `nemo.lens.contrib.fastapi.instrument_fastapi(app)` extracts context from every incoming request and makes it the parent of the current span.
* **aiohttp client**: `nemo.lens.contrib.aiohttp.instrument_aiohttp_client()` injects context into every outgoing request.
* **Ray**: `nemo.lens.contrib.ray.inject_ray_context()`, `extract_ray_context()`, and `traced_remote_call(method)` provide helpers for Ray's kwargs-based propagation.
* **NCCL**: `nemo.lens.contrib.nccl.serialize_context()` and `extract_nccl_context(data)` provide helpers for piggy-backing context on NCCL byte transfers.

See [Contrib Helpers](/nemo/lens/user-guide/contrib-helpers) for details.

**When auto-instrumentation is available, use it instead of manual injection.** This approach covers every call site, handles error paths, and prevents code rot as the codebase grows.

## Cross-Rank Propagation in Distributed Training

HTTP-style propagation does not apply to `torch.distributed`, as those are not carrier-based transports. For distributed training, NeMo Lens provides:

* `broadcast_trace_context(rank, src_rank=0)`, which uses `torch.distributed.broadcast` to share trace context across ranks.
* `create_linked_span(tracer, name, remote_carrier=carrier)`, which creates a span with an OTel Link (not a parent-child relationship) to a remote span, which is useful for pipeline-parallel correlation.

See [Distributed Tracing](/nemo/lens/user-guide/distributed-tracing) for the full pattern.

## Baggage

[Baggage](https://www.w3.org/TR/baggage/) is a W3C standard for propagating small key/value context alongside trace context (such as "which customer is this request for?" or "is this a canary deployment?").

NeMo Lens's propagator is a `CompositePropagator` of `TraceContextTextMapPropagator` and `W3CBaggagePropagator`, so both flow through `inject_context` and `extract_context` automatically. Set baggage with:

```python
from opentelemetry import baggage, context

ctx = baggage.set_baggage("customer.id", "42")
token = context.attach(ctx)
try:
    # All downstream inject_context calls will include customer.id in the baggage header
    ...
finally:
    context.detach(token)
```

Baggage values are propagated across service boundaries but are not automatically added to spans. Use them for filtering/routing decisions, not for span attributes.