Install NVIDIA NeMo Agent Toolkit#

This guide will help you set up your NVIDIA NeMo Agent toolkit development environment, run existing workflows, and create your own custom workflows using the nat command-line interface.

Supported LLM APIs#

The following LLM API providers are supported:

  • NIM (such as Llama-3.1-70b-instruct and Llama-3.3-70b-instruct)

  • OpenAI

  • AWS Bedrock

  • Azure OpenAI

Framework Integrations#

To keep the library lightweight, many of the first-party plugins supported by NeMo Agent toolkit are located in separate distribution packages. For example, the nvidia-nat-langchain distribution contains all the LangChain-specific and LangGraph-specific plugins, and the nvidia-nat-mem0ai distribution contains the Mem0-specific plugins.

To install these first-party plugin libraries, you can use the full distribution name (for example, nvidia-nat-langchain) or use the nvidia-nat[langchain] extra distribution. The following extras are supported:

  • nvidia-nat[adk] or nvidia-nat-adk - Google ADK

  • nvidia-nat[agno] or nvidia-nat-agno - Agno

  • nvidia-nat[all] or nvidia-nat-all - Pseudo-package for installing all optional dependencies

  • nvidia-nat[crewai] or nvidia-nat-crewai - CrewAI

  • nvidia-nat[data-flywheel] or nvidia-nat-data-flywheel - NeMo DataFlywheel

  • nvidia-nat[ingestion] or nvidia-nat-ingestion - Additional dependencies needed for data ingestion

  • nvidia-nat[langchain] or nvidia-nat-langchain - LangChain, LangGraph

  • nvidia-nat[llama-index] or nvidia-nat-llama-index - LlamaIndex

  • nvidia-nat[mcp] or nvidia-nat-mcp - Model Context Protocol (MCP)

  • nvidia-nat[mem0ai] or nvidia-nat-mem0ai - Mem0

  • nvidia-nat[mysql] or nvidia-nat-mysql - MySQL

  • nvidia-nat[opentelemetry] or nvidia-nat-opentelemetry - OpenTelemetry

  • nvidia-nat[phoenix] or nvidia-nat-phoenix - Arize Phoenix

  • nvidia-nat[profiling] or nvidia-nat-profiling - Additional dependencies needed for profiling

  • nvidia-nat[ragaai] or nvidia-nat-ragaai - RagaAI Catalyst

  • nvidia-nat[redis] or nvidia-nat-redis - Redis

  • nvidia-nat[s3] or nvidia-nat-s3 - Amazon S3

  • nvidia-nat[semantic-kernel] or nvidia-nat-semantic-kernel - Microsoft Semantic Kernel

  • nvidia-nat[test] or nvidia-nat-test - NeMo Agent toolkit test

  • nvidia-nat[weave] or nvidia-nat-weave - Weights & Biases Weave

  • nvidia-nat[zep-cloud] or nvidia-nat-zep-cloud - Zep

Supported Platforms#

Operating System

Architecture

Python Version

Supported

Linux

x86_64

3.11, 3.12, 3.13

✅ Tested, Validated in CI

Linux

aarch64

3.11, 3.12, 3.13

✅ Tested, Validated in CI

macOS

x86_64

3.11, 3.12, 3.13

❓ Untested, Should Work

macOS

aarch64

3.11, 3.12, 3.13

✅ Tested

Windows

x86_64

3.11, 3.12, 3.13

❓ Untested, Should Work

Windows

aarch64

3.11, 3.12, 3.13

❌ Unsupported

Software Prerequisites#

NVIDIA NeMo Agent toolkit is a Python library that doesn’t require a GPU to run by default. Before you begin using NeMo Agent toolkit, ensure that you meet the following software prerequisites:

Additional Prerequisites for Development#

Install from Package#

The package installation is recommended for production use.

Note

To run any examples, you need to install the NeMo Agent toolkit from source.

To install the latest stable version of NeMo Agent toolkit, run the following command:

pip install nvidia-nat

NeMo Agent toolkit has many optional dependencies which can be installed with the core package. Optional dependencies are grouped by framework and can be installed with the core package. For example, to install the LangChain/LangGraph plugin, run the following:

pip install "nvidia-nat[langchain]"

Or for all optional dependencies:

pip install "nvidia-nat[all]"

The full list of optional dependencies can be found here.

Install From Source#

Installing from source is required to run any examples provided in the repository or to contribute to the project.

  1. Clone the NeMo Agent toolkit repository to your local machine.

    git clone -b main https://github.com/NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit
cd nemo-agent-toolkit

  2. Initialize, fetch, and update submodules in the Git repository.

    git submodule update --init --recursive

  3. Fetch the data sets by downloading the LFS files.

    git lfs install
git lfs fetch
git lfs pull

  4. Create a Python environment.

    uv venv --python 3.13 --seed .venv
source .venv/bin/activate

    Note

    Python 3.11 and 3.12 are also supported simply replace 3.13 with 3.11 or 3.12 in the uv command above.

  5. Install the NeMo Agent toolkit library. To install the NeMo Agent toolkit library along with all of the optional dependencies. Including developer tools (--all-groups) and all of the dependencies needed for profiling and plugins (--all-extras) in the source repository, run the following:

    uv sync --all-groups --all-extras

    Alternatively to install just the core NeMo Agent toolkit without any optional plugins, run the following:

    uv sync

    At this point individual plugins, which are located under the packages directory, can be installed with the following command uv pip install -e '.[<plugin_name>]'. For example, to install the LangChain/LangGraph plugin, run the following:

    uv pip install -e '.[langchain]'

    Note

    Many of the example workflows require plugins, and following the documented steps in one of these examples will in turn install the necessary plugins. For example following the steps in the examples/getting_started/simple_web_query/README.md guide will install the nvidia-nat-langchain plugin if you haven’t already done so.

    In addition to plugins, there are optional dependencies needed for profiling. Installing the profiling sub-package is required for evaluation and profiling workflows using nat eval. To install these dependencies, run the following:

    uv pip install -e '.[profiling]'

  6. Verify that you’ve installed the NeMo Agent toolkit library.

    nat --help
nat --version

    If the installation succeeded, the nat command will log the help message and its current version.

Obtaining API Keys#

Depending on which workflows you are running, you may need to obtain API keys from the respective services. Most NeMo Agent toolkit workflows require an NVIDIA API key defined with the NVIDIA_API_KEY environment variable. An API key can be obtained by visiting build.nvidia.com and creating an account.

Optional OpenAI API Key#

Some workflows may also require an OpenAI API key. Visit OpenAI and create an account. Navigate to your account settings to obtain your OpenAI API key. Copy the key and set it as an environment variable using the following command:

export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"

Running Example Workflows#

Before running any of the NeMo Agent toolkit examples, set your NVIDIA API key as an environment variable to access NVIDIA AI services.

export NVIDIA_API_KEY=<YOUR_API_KEY>

Note

Replace <YOUR_API_KEY> with your actual NVIDIA API key.

Running the Simple Workflow#

  1. Install the nat_simple_web_query Workflow

    uv pip install -e examples/getting_started/simple_web_query

  2. Run the nat_simple_web_query Workflow

    nat run --config_file=examples/getting_started/simple_web_query/configs/config.yml --input "What is LangSmith"

  3. Run and evaluate the nat_simple_web_query Workflow

    The eval_config.yml YAML is a super-set of the config.yml containing additional fields for evaluation. To evaluate the nat_simple_web_query workflow, run the following command:

    nat eval --config_file=examples/evaluation_and_profiling/simple_web_query_eval/configs/eval_config.yml

NeMo Agent Toolkit Packages#

Once a NeMo Agent toolkit workflow is ready for deployment to production, the deployed workflow will need to declare a dependency on the nvidia-nat package, along with the needed plugins. When declaring a dependency on NeMo Agent toolkit it is recommended to use the first two digits of the version number. For example if the version is 1.0.0 then the dependency would be 1.0.

For more information on the available plugins, refer to Framework Integrations.

Example dependency for NeMo Agent toolkit using the LangChain/LangGraph plugin for projects using a pyproject.toml file:

dependencies = [
"nvidia-nat[langchain]~=1.0",
# Add any additional dependencies your workflow needs
]

Alternately for projects using a requirements.txt file:

nvidia-nat[langchain]==1.0.*

Next Steps#

  • Explore the examples in the examples directory to learn how to build custom workflows and tools with NeMo Agent toolkit.

  • Review the NeMo Agent toolkit tutorials for detailed guidance on using the toolkit.