Interface Introduction#

NVIDIA ACE Agent supports four interfaces to interact with bots: CLI, HTTP Server, Event, and gRPC interfaces. You might choose any one of the interfaces based on bot pipeline architecture and use case or based on the stage of bot development, and should be used accordingly for the best experience.

CLI Interface#

The Command Line Interface (CLI) is a simple interface that allows you to interact with the bot by typing commands in a terminal window and receiving the response on the same terminal window. This interface is well suited for the initial development phase of the bot, where you want to quickly try out queries without any overhead or additional steps. The CLI interface is exposed by the ACE Agent Chat Engine microservice.

You can deploy a CLI interface using the Python Environment or the Docker Environment. On successful deployment, this opens a text prompt in the terminal where you can enter a query. The CLI interface looks like this:

[YOU] Are you a person?
[BOT] No, I am just a chatbot.

[YOU] What is your name?
[BOT] I do not have a name yet.

In the CLI interface, if a query starts with \ such as \UtteranceUserActionFinished(final_transcript="Hello"), we process the query as an event. List of events can be found at Standard Events and Actions. The event should be handled by the corresponding bot, otherwise there will be no response. For example:

[YOU] \UtteranceUserActionFinished(final_transcript="Hello")
[BOT] Hello! I am an AI assistant designed to provide helpful information and assist with tasks. Is there something specific you would like to know or discuss?

This feature enables developers to test multimodal bots without the need of full multimodal pipeline deployment such as the NVIDIA Tokkio pipeline.

Note

The CLI Interface can only be used for bots that have colang_disable_async_execution: True config in the bot_config.yaml file because the CLI interface needs to respond in synchronous mode for each user query.

Event Interface#

The event interface provides an asynchronous, event-based interface to interact with bots written in Colang 2.0. It manages interactions using Redis streams by forwarding UMIM events to the Colang 2.0 runtime (that is part of the Chat Engine) and publishes back events from the runtime to the event streams. The interface supports multiple user sessions and provides low latency event handling. The event interface is exposed by the ACE Agent Chat Engine microservice. We can deploy the event interface using any of the deployment environments.

ACE Agent Event Interface

After launching the event interface, it will wait for PipelineAcquired events on stream ace_agent_system_events. These events indicate that a new stream or pipeline has become available. The ACE Agent will spawn an event worker that will be dedicated to this stream to forward any events to the configured bot.

HTTP Server Interface#

The HTTP server interface exposes several rest APIs which can be used to interact with the bot, view the bot’s status, and update the bot’s state. You can interact with the bot from a custom UI or application using HTTP calls for just text-based pipelines. The HTTP interface is exposed by the Chat Engine microservice.

We can deploy an HTTP server using any of the deployment environments. This launches the HTTP server at http://<server_ip>:9000. For more information about the REST APIs exposed by the server, refer to the HTTP Interface API documentation.

gRPC Interface#

This interface starts a gRPC server which exposes several APIs that can be used to interact with the bot using speech or text and update the bot’s state. You can interact with the bot from a custom UI or application using gRPC calls for just text-based pipelines, speech AI, or avatar-based pipelines.

The gRPC server is exposed by the Chat Controller microservice. The gRPC server is supported only in the Docker Environment and Kubernetes Environment. For more information about the gRPC APIs exposed by the server, refer to the gRPC Interface API documentation.

Under the hood, the gRPC interface can utilize HTTP APIs or use UMIM bus for event interface based on the architecture of bot deployment.

On this page