NeMo Relay LangChain Integration

Use the nemo_relay.integrations.langchain package to add NeMo Relay observability to LangChain agents.

Setup

Install the LangChain integration extra in your application environment.

uv add "nemo-relay[langchain]"

The example below uses the NVIDIA LangChain provider. Install that provider extra too if you want to run the example as written:

uv add "nemo-relay[langchain,langchain-nvidia]"

Usage Example

import asyncio
import nemo_relay
from langchain.agents import create_agent
from langchain_core.tools import tool
from nemo_relay.integrations.langchain import NemoRelayCallbackHandler, NemoRelayMiddleware
@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72 degrees."
agent = create_agent(
    model="nvidia:nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
    tools=[get_weather],
    middleware=[NemoRelayMiddleware()],
    system_prompt="Use tools when they are relevant. Keep the final answer brief.",
)
input_payload = {
    "messages": [
        {
            "role": "user",
            "content": "What is the weather in San Francisco?",
        }
    ]
}
with nemo_relay.scope.scope("langchain-request", nemo_relay.ScopeType.Agent):
    result = asyncio.run(
        agent.ainvoke(input_payload, config={"callbacks": [NemoRelayCallbackHandler()]})
    )
final_message = result["messages"][-1]
print(f"Final response: {final_message.content}")

When the agent uses ChatNVIDIA, NeMo Relay forwards request headers, including Dynamo session lineage, as HTTP headers. They are not added to the NIM request body. For chat models built on the OpenAI or Anthropic Python SDKs, such as langchain-openai and langchain-anthropic, the headers travel in the per-request extra_headers model setting, which those SDKs send as HTTP headers. NeMo Relay also uses extra_headers when your model_settings already configure it. Other providers, such as ChatOCIGenAI from langchain-oci, receive no Relay headers because their SDKs reject unknown request fields. The model call proceeds without them.

Human Approval in Tools

NemoRelayMiddleware supports tools that pause for human approval with both invoke and ainvoke. When a tool calls langgraph.types.interrupt(), Relay closes its tool scope. The agent returns __interrupt__ with the approval request.

Configure the agent with a checkpointer and set configurable.thread_id to identify the conversation. To resume the tool, invoke the same agent with Command(resume=...) and the same thread ID.

To record Graph Interrupt and Graph Resume marks, install the LangGraph callbacks with nemo_relay.integrations.langgraph.configure_graph(agent) before you run or serve the agent. Refer to the LangGraph integration guide for callback configuration.

Verify the Integration

The integration works correctly when:

• The agent run completes and prints a final response.
• The langchain-request scope contains the managed model call.
• Tool activity appears under the same request when the agent decides to use get_weather.

Observability

Refer to Observability for details on exporting NeMo Relay observability data to third-party systems.