Plugin Server

The Plugin server is a FastAPI-based server that enables ACE Agent to interact with third-party applications or APIs over a REST interface. It exposes a Swagger endpoint, which allows developers to easily write and validate Plugin servers in a sandbox environment. Refer to the Plugin Configurations section to add plugins in your bot. The Plugin server allows you to integrate your own agent built using LangChain or LlamaIndex or any other tool with a simple interface and allows you to add Speech AI and Avatar AI using ACE microservices.

Using Plugin with Colang

By default, ACE Agent assumes the plugin is hosted on http://localhost:9002. When the Plugin server is hosted on a different IP, its URL can be provided using plugin_server_url in the bot_config.yaml file.

configs:
    plugin_server_url: http://<workstation-ip>:<port>

The Plugin server is integrated as a custom action plugin in Colang. An endpoint hosted as part of the plugin can be called in a flow with a given endpoint.

define flow bot responds with the time
    user asks time
    $result = execute plugin(endpoint="/date/get_time")
    bot provide $result

Passing URL Parameters

We can provide URL parameters using the custom plugin action using comma separated parameters. Consider an example where an endpoint provides the temperature for a given location in unit fahrenheit. You can pass information to the plugin by comma separated parameters weather plugin’s get_temperature endpoint.

@router.get("/get_temperature")
def get_temperature(location: str, unit: str) -> Optional[int]:
    """Return temperature for given location"""

    # Temperature extraction logic
    return temperature_at_location

Use the get_temperature endpoint in the Colang flow:

define flow bot responds temperature
    user asks temperature
    $result = execute plugin(endpoint="/weather/get_temperature", location='santa clara', unit='fahrenheit')
    bot provide $result

Passing Context Information and Conversation History

Colang stores context information which contains all the context variables used in the flow. This context information is always sent in POST requests. You can accept context from the plugin and access this information. History of conversations stored in the ACE Agent can be accepted and used at the Plugin server.

@router.post("/get_temperature")
def get_prompt(location: str, context: Optional[Dict[str, Any]] = {}, conv_history: Optional[List[Dict[str, str]]] = []) -> str:

    unit = context.get("unit", "fahrenheit")
    # Temperature extraction logic
    return temperature_at_location

Use the get_temperature endpoint in the Colang flow, with context:

define flow bot responds temperature
    user asks temperature
    $result = execute plugin(endpoint="/weather/get_temperature", location='santa clara', request_type='post')
    bot provide $result

Maintaining Plugin Memory

The Plugin server exposes a custom decorator that can be attached to endpoints of a plugin to maintain a custom memory for each userId corresponding to that plugin. The endpoint can update the memory by modifying the memory object by reference.

from fastapi import APIRouter
from typing import Optional, Dict, Any

from plugin_server.utils import memory_handler

router = APIRouter()

@router.get("/count")
@memory_handler
def count(memory: Optional[Dict[str, Any]] = {}, context: Optional[Dict[str, Any]] = {}) -> int:
    """
    Count the number of times this endpoint has been called by the given user/session.
    Modifies the count in the memory (by reference) and returns the count.
    """
    memory["count"] = memory.get("count", 0) + 1
    return memory["count"]

Accessing Custom Metadata

In certain cases, the request to the Chat Engine server may include certain custom information in the Metadata field of the body. The Chat Engine passes this information in the request to any POST requests made to the Plugin server. The metadata can be accessed in the metadata field of the request body.

@router.post("/endpoint")
def endpoint(question: str, context: Dict[str, Any] = {}, metadata: Dict[str, Any] = {}) -> str:

    logger.info(f"Metadata: {metadata}")
    ...

Parameter List

Named Parameters Supported by Plugin
Parameter	Description
`endpoint`	Plugin path which needs to be sent.
`request_type`	`GET`/`POST` request time.
`request_timeout`	Max time to wait for a response.

Logging and Monitoring

The Plugin server includes a logging feature that enables you to efficiently manage and access log data. To initiate logging support, you can access the logger named plugin by using the following Python code:

import logging
logger = logging.getLogger("fulfillment")

Log file management:

The Plugin server logs are written to log files managed by the server.

By default, log files are stored in the current working directory ($PWD) within a subdirectory named log.

Each log file is named in the following format: plugin_<host_machine_name>_<current_time>.log

A symbolic link named plugin.log is created to provide convenient access to the most recent log file.

Log files are rotated after reaching a size of 10 MB, ensuring that log data remains manageable and does not consume excessive storage.

Adding Plugin Requirements

Plugin requirements can be added to a file called requirements.txt. This file should be placed in the directory where the plugin entrypoint is kept. The requirements.txt file should contain all of the requirements that are needed by the plugin.

When the Plugin server starts, it will install all of the requirements from the requirements.txt file. Once the requirements have been installed, the Plugin server will start.

Interacting with the Plugin Server

To launch the Plugin server, use the aceagent tool.

aceagent plugin-server deploy --config <plugin_config.yaml>

By default, the Plugin server is launched on port 9002. You can interact with the Swagger API at http://<workstation-ip>:<port>/docs.

For the detailed API schema with examples, refer to the Plugin Server.