NVIDIA NeMo Agent Toolkit Overview#

NVIDIA NeMo Agent toolkit is a flexible, lightweight, and unifying library that allows you to easily connect existing enterprise agents to data sources and tools across any framework.

Install#

uv

uv pip install nvidia-nat

pip

pip install nvidia-nat

For detailed installation instructions, including optional dependencies, please refer to the Install Guide.

Key Features#

Framework Agnostic: NeMo Agent toolkit works side-by-side and around existing agentic frameworks, such as LangChain, LlamaIndex, CrewAI, Microsoft Semantic Kernel, Google ADK, as well as customer enterprise frameworks and simple Python agents. This allows you to use your current technology stack without replatforming. NeMo Agent toolkit complements any existing agentic framework or memory tool you’re using and isn’t tied to any specific agentic framework, long-term memory, or data source.
Reusability: Every agent, tool, and agentic workflow in this library exists as a function call that works together in complex software applications. The composability between these agents, tools, and workflows allows you to build once and reuse in different scenarios.
Rapid Development: Start with a pre-built agent, tool, or workflow, and customize it to your needs. This allows you and your development teams to move quickly if you’re already developing with agents.
Profiling: Use the profiler to profile entire workflows down to the tool and agent level, track input/output tokens and timings, and identify bottlenecks.
Observability: Monitor and debug your workflows with dedicated integrations for popular observability platforms such as Phoenix, Weave, and Langfuse, plus compatibility with OpenTelemetry-based systems. Track performance, trace execution flows, and gain insights into your agent behaviors.
Evaluation System: Validate and maintain accuracy of agentic workflows with built-in evaluation tools.
User Interface: Use the NeMo Agent toolkit UI chat interface to interact with your agents, visualize output, and debug workflows.
Full MCP Support: Compatible with Model Context Protocol (MCP). You can use NeMo Agent toolkit as an MCP client to connect to and use tools served by remote MCP servers. You can also use NeMo Agent toolkit as an MCP server to publish tools via MCP.
A2A Protocol Support: Compatible with Agent-to-Agent (A2A) Protocol. You can use NeMo Agent toolkit as an A2A client to connect to and delegate tasks to remote A2A agents. You can also use NeMo Agent toolkit as an A2A server to publish workflows as discoverable A2A agents.

Hello World Example#

Before getting started, it’s possible to run this simple workflow and many other examples in Google Colab with no setup. Click here to open the introduction notebook: .

Install NeMo Agent toolkit along with the LangChain integration plugin:

uv

uv pip install "nvidia-nat[langchain]"

pip

pip install "nvidia-nat[langchain]"

Ensure you have set the NVIDIA_API_KEY environment variable to allow the example to use NVIDIA NIMs. An API key can be obtained by visiting build.nvidia.com and creating an account.
```
export NVIDIA_API_KEY=<your_api_key>
```

Create the NeMo Agent Toolkit workflow configuration file. This file will define the agents, tools, and workflows that will be used in the example. Save the following as workflow.yml:

functions:
  # Add a tool to search wikipedia
  wikipedia_search:
    _type: wiki_search
    max_results: 2

llms:
  # Tell NeMo Agent Toolkit which LLM to use for the agent
  nim_llm:
    _type: nim
    model_name: meta/llama-3.1-70b-instruct
    temperature: 0.0

workflow:
  # Use an agent that 'reasons' and 'acts'
  _type: react_agent
  # Give it access to our wikipedia search tool
  tool_names: [wikipedia_search]
  # Tell it which LLM to use
  llm_name: nim_llm
  # Make it verbose
  verbose: true
  # Retry up to 3 times
  parse_agent_response_max_retries: 3

Run the Hello World example using the nat CLI and the workflow.yml file.

nat run --config_file workflow.yml --input "List five subspecies of Aardvarks"

This will run the workflow and output the results to the console.

Workflow Result:
['Here are five subspecies of Aardvarks:\n\n1. Orycteropus afer afer (Southern aardvark)\n2. O. a. adametzi  Grote, 1921 (Western aardvark)\n3. O. a. aethiopicus  Sundevall, 1843\n4. O. a. angolensis  Zukowsky & Haltenorth, 1957\n5. O. a. erikssoni  Lönnberg, 1906']

FAQs#

For frequently asked questions, refer to FAQs.

Feedback#

We would love to hear from you! Please file an issue on GitHub if you have any feedback or feature requests.