Quickstart

View as Markdown

Instrument a minimal script in four steps.

Set Environment Variables

$export NEMO_LENS_ENABLED=1
$export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 # where your collector listens
$export NEMO_LENS_SPAN_GROUPS=per_step # adds per-step boundaries (step/forward_backward/optimizer)

Initialize Telemetry

1from nemo.lens import NemoLensConfig, setup_telemetry
2
3config = NemoLensConfig.from_env()
4handle = setup_telemetry(config, rank=0, world_size=1)
5
6# handle.tracer — OTel Tracer (real on exporting rank, no-op elsewhere)
7# handle.meter — OTel Meter
8# handle.is_exporting — whether this rank exports

Call this once per process, typically at startup.

Add Instrumentation

Three primitives cover most cases:

1from nemo.lens import managed_span, trace_fn, span_cm
2
3# Group-gated context manager — cheap when disabled (gated by a frozenset lookup)
4with managed_span('step', 'train.step', iteration=42) as span:
5 do_training_step()
6 if span is not None:
7 span.set_attribute('loss', compute_loss())
8
9# Group-gated decorator — no re-indentation
10@trace_fn('forward_backward', 'train.forward_backward')
11def forward_pass(batch):
12 ...
13
14# Simple ungated context manager — always creates a span
15with span_cm('demo.evaluate', tracer=handle.tracer):
16 ...

Shut Down Cleanly

1try:
2 ... # your training loop
3finally:
4 handle.shutdown()

handle.shutdown() flushes pending spans and metrics, then shuts down the providers. Do not call force_flush() on the global providers manually; the handle encapsulates this correctly.

Complete Example

1import time
2from nemo.lens import NemoLensConfig, setup_telemetry, managed_span
3
4def main():
5 config = NemoLensConfig.from_env()
6 handle = setup_telemetry(config, rank=0, world_size=1)
7
8 try:
9 with managed_span('job', 'demo.job'):
10 for i in range(5):
11 with managed_span('step', 'demo.step', iteration=i):
12 time.sleep(0.1)
13 finally:
14 handle.shutdown()
15
16if __name__ == "__main__":
17 main()

Run with NEMO_LENS_ENABLED=1 to export; without it, the script is a no-op at the OTel level.

Next Steps