> This page is for NeMo Relay, version v0.10.0.
> For other versions, use one of these documentation indexes:
> - Latest (v0.10.0) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.10.0: https://docs.nvidia.com/nemo/relay/v0.10.0/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt
> - v0.6.0: https://docs.nvidia.com/nemo/relay/v0.6.0/llms.txt
> - v0.5.0: https://docs.nvidia.com/nemo/relay/v0.5.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# NeMo Relay LangGraph Integration

> Add NeMo Relay observability to LangGraph applications.

Use the `nemo_relay.integrations.langgraph` package to add NeMo Relay
observability to [LangGraph](https://www.langchain.com/langgraph) workflows through public LangGraph APIs.

## Setup

Install the LangGraph integration extra in your application environment.

#### uv

```bash
uv add "nemo-relay[langgraph]"
```

#### pip

```bash
pip install "nemo-relay[langgraph]"
```

Installing the `langgraph` extra also installs the LangChain integration
dependencies.

## Usage Example

```python
from typing_extensions import TypedDict

import nemo_relay
from langgraph.graph import END, START, StateGraph
from nemo_relay.integrations.langgraph import NemoRelayCallbackHandler

class State(TypedDict):
    value: int

def increment(state: State) -> State:
    return {"value": state["value"] + 1}

builder = StateGraph(State)
builder.add_node("increment", increment)
builder.add_edge(START, "increment")
builder.add_edge("increment", END)

graph = builder.compile()

with nemo_relay.scope.scope("langgraph-request", nemo_relay.ScopeType.Agent):
    result = graph.invoke(
        {"value": 1},
        config={"callbacks": [NemoRelayCallbackHandler()]},
    )

print(result)
```

## Managed ToolNode Calls

For a standalone LangGraph `ToolNode`, construct the node with
`create_tool_node`. It routes each model-requested tool call through Relay's
managed execution pipeline while preserving LangGraph state, store, and runtime
argument injection.

```python
from langchain_core.messages import AIMessage
from langchain_core.tools import tool
from langgraph.graph import END, START, MessagesState, StateGraph
from nemo_relay.integrations.langgraph import (
    NemoRelayCallbackHandler,
    create_tool_node,
)

@tool
def get_weather(location: str) -> str:
    """Return the weather for a location."""
    return f"Sunny in {location}"

builder = StateGraph(MessagesState)
builder.add_node("tools", create_tool_node([get_weather]))
builder.add_edge(START, "tools")
builder.add_edge("tools", END)
graph = builder.compile()

with nemo_relay.scope.scope("langgraph-request", nemo_relay.ScopeType.Agent):
    result = graph.invoke(
        {
            "messages": [
                AIMessage(
                    content="",
                    tool_calls=[{"name": "get_weather", "args": {"location": "Boston"}, "id": "call-1"}],
                )
            ]
        },
        config={"callbacks": [NemoRelayCallbackHandler()]},
    )
```

`NemoRelayCallbackHandler` records the graph lifecycle and provides its scopes.
`create_tool_node` provides managed tool execution. Use both for complete
standalone LangGraph instrumentation. For custom ToolNode wrapper composition,
construct `ToolNode` directly and pass `wrap_tool_call` and `awrap_tool_call`
from this integration.

## Served LangChain Agents

When you export a LangChain agent for `langgraph dev` or `langgraph up`, add
the Relay callback handler to the compiled graph before exporting it. Request
callbacks are not part of the serialized LangGraph API request.

```python
from langchain.agents import create_agent
from nemo_relay.integrations.langgraph import NemoRelayMiddleware, configure_graph

agent = create_agent(
    model="nvidia:nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
    tools=[],
    middleware=[NemoRelayMiddleware()],
)
# Ensure the callables are configured with NeMo Relay scope events
agent = configure_graph(agent)
```

`configure_graph` stores a `NemoRelayCallbackHandler` in the graph's base
configuration and returns the same compiled graph, so the LangGraph server can
load it. The callback handler creates the graph and node scopes. The middleware
routes model and tool calls through Relay under those scopes.

The handler closes each graph scope on the stack where it started. This works
even when the server runs the completion callback in a different context.
Successful runs emit an end event with status `OK`. Failed runs emit an end
event with status `ERROR`.

For tracked runs, `Graph Interrupt` and `Graph Resume` marks belong to the graph
scope identified by their run ID, including when the server delivers them in a
different context. When the run ID is absent or untracked, the marks use the
current scope.

For LangChain agents inside a LangGraph workflow, use `NemoRelayMiddleware` from
this package the same way as the LangChain integration and pass the LangGraph
`config` into the nested agent call:

```python
from langchain.agents import create_agent
from langchain_core.runnables import RunnableConfig
from nemo_relay.integrations.langgraph import NemoRelayMiddleware

agent = create_agent(
    model="nvidia:nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
    tools=[],
    middleware=[NemoRelayMiddleware()],
)

def agent_node(state: dict, config: RunnableConfig) -> dict:
    return agent.invoke({"messages": state["messages"]}, config=config)
```

Install the NVIDIA LangChain provider if you want to run the nested agent
example as written:

#### uv

```bash
uv add "nemo-relay[langgraph,langchain-nvidia]"
```

#### pip

```bash
pip install "nemo-relay[langgraph,langchain-nvidia]"
```

## Verify the Integration

The integration works correctly when:

* `graph.invoke(...)` returns the incremented state from the example.
* The `langgraph-request` scope contains the LangGraph run.
* Nested LangChain agents inherit the same callback config when you pass the LangGraph `config` through.

## Observability

Refer to [Observability](/configure-plugins/observability/about) for details on exporting NeMo Relay observability data to third-party systems.