Launch the NVIDIA NeMo Agent Toolkit API Server and User Interface#

NVIDIA NeMo Agent toolkit provides a user interface for interacting with your running workflow. This guide walks you through starting the API server and launching the web-based user interface to interact with your workflows.

User Interface Features#

Modern and responsive user interface
Real-time streaming responses
Human-in-the-loop workflow support
Chat history and conversation management
Light and Dark theme support
WebSocket and HTTP API integration
Intermediate steps visualization
Docker deployment support

Walk-through#

This walk-through guides you through the steps to set up and configure the NeMo Agent toolkit user interface.

Prerequisites#

Before starting, ensure you have:

NeMo Agent toolkit installed and configured
Set up the simple calculator workflow by following the instructions in examples/getting_started/simple_calculator/README.md
Node.js v18+ installed (required for the web interface)

The NeMo Agent toolkit UI is located in a Git submodule at external/nat-ui. Ensure you have checked out all of the Git submodules by running the following:

git submodule update --init --recursive

Start the NeMo Agent Toolkit Server#

You can start the NeMo Agent toolkit server using the nat serve command with the appropriate configuration file.

nat serve --config_file=examples/getting_started/simple_calculator/configs/config.yml

Running this command will produce the expected output as shown below (truncated for brevity):

INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)

Verify the NeMo Agent Toolkit Server is Running#

After the server is running, you can make HTTP requests to interact with the workflow. This step confirms that the server is properly configured and can process requests.

curl --request POST \
  --url http://localhost:8000/generate \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "Is 4 + 4 greater than the current hour of the day?"
}'

Running this command will produce the following expected output:

The response depends on the current time of day that the command is run.

{
  "value": "No, 8 is less than the current hour of the day (4)."
}

Launch the NeMo Agent Toolkit User Interface#

After the NeMo Agent toolkit server starts, launch the web user interface. Launching the UI requires that Node.js v18+ is installed. Instructions for downloading and installing Node.js can be found in the official Node.js documentation.

For comprehensive information about the NeMo Agent Toolkit UI, including setup instructions, configuration options, and UI components documentation, see:

NeMo Agent Toolkit UI README - Complete UI documentation and setup guide
UI Components Documentation - Detailed information about components, features, and interface elements

Local Development#

cd external/nat-ui
npm ci
npm run dev

After the web development server starts, open a web browser and navigate to http://localhost:3000/.

NeMo Agent toolkit Web User Interface

Important

Workflows requiring human input or interaction (such as human-in-the-loop workflows, OAuth authentication, or interactive prompts) must use WebSocket connections. HTTP requests are the default method of communication, but human-in-the-loop functionality is not supported through HTTP.

To enable WebSocket mode:

Open the panel on the top right of the webpage
Toggle the WebSocket button to ON
You will see a notification that says “websocket connected” when successfully connected

Connect the User Interface to the NeMo Agent Toolkit Server Using HTTP API#

Configure the settings by selecting the Settings icon located on the bottom left corner of the home page.

NeMo Agent toolkit Web UI Settings

Settings Options#

Appearance:

Theme: Switch between Light and Dark mode

API Configuration:

HTTP Endpoint: Select API endpoint type:
- Chat Completions — Streaming - Real-time OpenAI Chat Completions compatible API endpoint with streaming responses (recommended for intermediate results)
- Chat Completions — Non-Streaming - Standard OpenAI Chat Completions compatible API endpoint
- Generate — Streaming - Text generation with streaming
- Generate — Non-Streaming - Standard text generation
Optional Generation Parameters: OpenAI Chat Completions compatible JSON parameters that can be added to the request body (available for chat endpoints)

WebSocket Configuration:

The WebSocket path defaults to websocket.

WebSocket Schema: Select schema for real-time connections:
- Chat Completions — Streaming - Streaming chat over WebSocket (recommended for intermediate results)
- Chat Completions — Non-Streaming - Non-streaming chat over WebSocket
- Generate — Streaming - Streaming generation over WebSocket
- Generate — Non-Streaming - Non-streaming generation over WebSocket

For intermediate results streaming, use Chat Completions — Streaming or Generate — Streaming.

Simple Calculator Example Conversation#

Interact with the chat interface by prompting the Agent with the message: Is 4 + 4 greater than the current hour of the day?

NeMo Agent Toolkit Web UI Workflow Result