NeMo Relay LangChain Integration

Use the nemo_relay.integrations.langchain package to add NeMo Relay observability to LangChain agents.

Setup

Install the LangChain integration extra in your application environment.

uv add "nemo-relay[langchain]"

The example below uses the NVIDIA LangChain provider. Install that provider extra too if you want to run the example as written:

uv add "nemo-relay[langchain,langchain-nvidia]"

Usage Example

import asyncio
import nemo_relay
from langchain.agents import create_agent
from langchain_core.tools import tool
from nemo_relay.integrations.langchain import NemoRelayCallbackHandler, NemoRelayMiddleware
@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72 degrees."
agent = create_agent(
    model="nvidia:nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
    tools=[get_weather],
    middleware=[NemoRelayMiddleware()],
    system_prompt="Use tools when they are relevant. Keep the final answer brief.",
)
input_payload = {
    "messages": [
        {
            "role": "user",
            "content": "What is the weather in San Francisco?",
        }
    ]
}
with nemo_relay.scope.scope("langchain-request", nemo_relay.ScopeType.Agent):
    result = asyncio.run(
        agent.ainvoke(input_payload, config={"callbacks": [NemoRelayCallbackHandler()]})
    )
final_message = result["messages"][-1]
print(f"Final response: {final_message.content}")

When the agent uses ChatNVIDIA, NeMo Relay forwards request headers, including Dynamo session lineage, as HTTP headers. They are not added to the NIM request body. For chat models built on the OpenAI or Anthropic Python SDKs, such as langchain-openai and langchain-anthropic, the headers travel in the per-request extra_headers model setting, which those SDKs send as HTTP headers. NeMo Relay also uses extra_headers when your model_settings already configure it. Other providers, such as ChatOCIGenAI from langchain-oci, receive no Relay headers because their SDKs reject unknown request fields; the model call proceeds without them.

Verify the Integration

The integration works correctly when:

• The agent run completes and prints a final response.
• The langchain-request scope contains the managed model call.
• Tool activity appears under the same request when the agent decides to use get_weather.

Observability

Refer to Observability for details on exporting NeMo Relay observability data to third-party systems.