Plugin Server

The Plugin server is a FastAPI-based server that enables ACE Agent to interact with third-party applications or APIs over a REST interface. It exposes a Swagger endpoint, which allows developers to easily write and validate Plugin servers in a sandbox environment. Refer to the Plugin Configurations section to add plugins in your bot. The Plugin server allows you to integrate your own agent built using LangChain or LlamaIndex or any other tool with a simple interface and allows you to add Speech AI and Avatar AI using ACE microservices.

Using Plugin with Colang

By default, ACE Agent assumes the plugin is hosted on http://localhost:9002. When the Plugin server is hosted on a different IP, its URL can be provided using plugin_server_url in the bot_config.yaml file.

configs:
    plugin_server_url: http://<workstation-ip>:<port>

The Plugin server is integrated as a custom actions. An endpoint hosted as part of the plugin can be called in a flow with a given endpoint.

To form a response from the Plugin server, you can use the following prebuilt custom actions.

• If the Plugin server is called by the bot and returns a non-streaming response, you should use InvokeFulfillmentAction action for getting response.

  flow tell time
    user asked current time
    $time = await InvokeFulfillmentAction(request_type="get", endpoint="/time/get_current_time")
    bot say "Current time is {$time}"
  

• If the response from the Plugin server is a text stream, you can use the InvokeStreamingFulfillmentAction action which starts gathering streaming text chunks. You can also call StreamingResponseFulfillmentAction to receive all chunks as a single response. If you want to break the response into sentences or even partial sentences, you can use the regex pattern for the same.

  # Invoke endpoint from plugin
    $started =  await InvokeStreamingFulfillmentAction(question=$transcript,endpoint="your/endpoint")
    if $started
      # Get first sentence from plugin response
      $response = await StreamingResponseFulfillmentAction(endpoint="your/endpoint",pattern=r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<![0-9]\.)(?<![A-Z]\.)(?<=\.|\?|\!)\s")
      while $response
        bot say $response
        # Check for next sentence
        $response = await StreamingResponseFulfillmentAction(endpoint="your/endpoint",pattern=r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<![0-9]\.)(?<![A-Z]\.)(?<=\.|\?|\!)\s")
  

• If the response from the Plugin server is a stream of JSON responses in the Chat Engine response schema, the JSON chunks will be parsed with their Response. You can use the InvokeStreamingChatAction action which starts gathering streaming JSON chunks. You can also call the StreamingResponseChatAction action repeatedly to receive the next sentence.

  # Invoke /chat endpoint from plugin
    $started =  await InvokeStreamingChatAction(question=$transcript,endpoint="rag/chat",chat_history=True)
    if $started
      # Get first sentence from RAG response
      $response = await StreamingResponseChatAction(endpoint="rag/chat")
      while $response
        bot say $response
        # Check for next sentence
        $response = await StreamingResponseChatAction(endpoint="rag/chat")
  

The pattern used for breaking sentences is r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<![0-9]\.)(?<![A-Z]\.)(?<=\.|\?|\!)\s". You can set the pattern parameter in StreamingResponseChatAction for your own pattern for custom logic. Setting chat_history as True will send the current user chat history as part of the Metadata field in the /chat request.

Passing URL Parameters

We can provide URL parameters using the custom plugin action using comma separated parameters. Consider an example where an endpoint provides the temperature for a given location in unit fahrenheit. You can pass information to the plugin by comma separated parameters weather plugin’s get_temperature endpoint.

@router.get("/get_temperature")
def get_temperature(location: str, unit: str) -> Optional[int]:
    """Return temperature for given location"""

    # Temperature extraction logic
    return temperature_at_location

Use the get_temperature endpoint in the Colang flow:

flow bot responds temperature
    user asks temperature
    $result = execute InvokeFulfillmentAction(endpoint="/weather/get_temperature", location='santa clara', unit='fahrenheit')
    bot say "Current temperature is {$result}"

Passing Context Information and Conversation History

Colang stores context information which contains all the context variables used in the flow. This context information is always sent in POST requests. You can accept context from the plugin and access this information. History of conversations stored in the ACE Agent can be accepted and used at the Plugin server.

@router.post("/get_temperature")
def get_prompt(location: str, context: Optional[Dict[str, Any]] = {}, conv_history: Optional[List[Dict[str, str]]] = []) -> str:

    unit = context.get("unit", "fahrenheit")
    # Temperature extraction logic
    return temperature_at_location

Use the get_temperature endpoint in the Colang flow, with context:

flow bot responds temperature
    user asks temperature
    $result = execute InvokeFulfillmentAction(endpoint="/weather/get_temperature", location='santa clara', unit='fahrenheit')
    bot say "Current temperature is {$result}"

Maintaining Plugin Memory

The Plugin server exposes a custom decorator that can be attached to endpoints of a plugin to maintain a custom memory for each userId corresponding to that plugin. The endpoint can update the memory by modifying the memory object by reference.

from fastapi import APIRouter
from typing import Optional, Dict, Any

from plugin_server.utils import memory_handler

router = APIRouter()

@router.get("/count")
@memory_handler
def count(memory: Optional[Dict[str, Any]] = {}, context: Optional[Dict[str, Any]] = {}) -> int:
    """
    Count the number of times this endpoint has been called by the given user/session.
    Modifies the count in the memory (by reference) and returns the count.
    """
    memory["count"] = memory.get("count", 0) + 1
    return memory["count"]

Accessing Custom Metadata

In certain cases, the request to the Chat Engine server may include certain custom information in the Metadata field of the body. The Chat Engine passes this information in the request to any POST requests made to the Plugin server. The metadata can be accessed in the metadata field of the request body.

@router.post("/endpoint")
def endpoint(question: str, context: Dict[str, Any] = {}, metadata: Dict[str, Any] = {}) -> str:

    logger.info(f"Metadata: {metadata}")
    ...

Parameter List

Named Parameters Supported by Plugin

Parameter	Description
endpoint	Plugin path which needs to be sent.
request_type	GET/POST request time.
request_timeout	Max time to wait for a response.

Logging and Monitoring

The Plugin server includes a logging feature that enables you to efficiently manage and access log data. To initiate logging support, you can access the logger named plugin by using the following Python code:

import logging
logger = logging.getLogger("fulfillment")

Log file management:

• The Plugin server logs are written to log files managed by the server.

• By default, log files are stored in the current working directory ($PWD) within a subdirectory named log.

• Each log file is named in the following format: plugin_<host_machine_name>_<current_time>.log

• A symbolic link named plugin.log is created to provide convenient access to the most recent log file.

• Log files are rotated after reaching a size of 10 MB, ensuring that log data remains manageable and does not consume excessive storage.

Adding Plugin Requirements

Plugin requirements can be added to a file called requirements.txt. This file should be placed in the directory where the plugin entrypoint is kept. The requirements.txt file should contain all of the requirements that are needed by the plugin.

When the Plugin server starts, it will install all of the requirements from the requirements.txt file. Once the requirements have been installed, the Plugin server will start.

Interacting with the Plugin Server

To launch the Plugin server, use the aceagent tool.

aceagent plugin-server deploy --config <plugin_config.yaml>

By default, the Plugin server is launched on port 9002. You can interact with the Swagger API at http://<workstation-ip>:<port>/docs.

For the detailed API schema with examples, refer to the Plugin Server.