Plugin Server

The Plugin server is a FastAPI-based server that enables the ACE Agent to interact with third-party applications or APIs over a REST interface. It exposes a Swagger endpoint, which allows developers to easily write and validate Plugin servers in a sandbox environment.

Refer to Plugin Server Microservice for more information.

Tokkio reference applications of LLM-RAG and Retail RAG use custom plugin servers that serve the purpose of fulfilling the user query. Refer to the details of both the plugin servers here:

Tokkio Retail Tokkio LLM-RAG

Secrets requirements for Tokkio Configuration

The Tokkio Plugin server for LLM RAG bot is configured to use the NIM models or OpenAI models for LLM calls. The users will need to configure passing the correct secrets for this microservice. The current configuration requires the following secrets to be defined and passed.

Note

If the users are not planning to use a NIM model, they can define a placeholder (non-empty) string for the NVIDIA_API_KEY, but they will need to define the required secret.

secrets:
k8sSecret/ngc-api-key-secret/NGC_CLI_API_KEY:
    k8sSecret:
    secretName: ngc-api-key-secret
    key: NGC_CLI_API_KEY
k8sSecret/nvidia-api-key-secret/NVIDIA_API_KEY:
    k8sSecret:
    secretName: nvidia-api-key-secret
    key: NVIDIA_API_KEY
k8sSecret/openai-key-secret/OPENAI_API_KEY:
    k8sSecret:
    secretName: openai-key-secret
    key: OPENAI_API_KEY
    .
    .
 - name: plugin-server
    type: ucf.svc.ace-agent.plugin-server
    parameters:
    imagePullSecrets:
        - name: ngc-docker-reg-secret
    secrets:
    ngc-api-key-secret: k8sSecret/ngc-api-key-secret/NGC_CLI_API_KEY
    openai-key-secret: k8sSecret/openai-key-secret/OPENAI_API_KEY
    nvidia-api-key-secret: k8sSecret/nvidia-api-key-secret/NVIDIA_API_KEY