RAG LangChain Bot

This is an example chatbot that showcases RAG executed directly in the Plugin server using LangChain. It uses NVIDIA AI Catalog models and Milvus vector stores to perform RAG.

The RAG Langchain bot showcases the following ACE Agent features:

• Integrating a custom RAG pipeline with ACE Agent

• Installing custom dependencies in Plugin server

• Returning a streaming text response from Plugin server

• Can be deployed in either Chat Engine Server Architecture or Plugin Server Architecture

Prerequisites

1. Start the Milvus Vector Store using the Docker compose file.

   docker compose -f samples/nv_langchain_bot/docker-compose-milvus.yml up -d
   

2. Install the pymilvus, pypdf, and langchain-nvidia-ai-endpoints packages to prepare for ingestion.

   pip install langchain-nvidia-ai-endpoints==0.0.1 pypdf==3.10.0
   

3. Create an NVIDIA API Catalog API key.

   1. Visit NVIDIA API Catalog website and login/sign-up: NVIDIA API Catalog. This gives you 1000 credits to use for free.

   2. Select any model to visit the model usage page. On the right side of this page, you should see a Python code sample to use the model. Click Get API key button in the upper right corner to generate an API key.

4. Ingest the sample URL by moving into the plugins directory and running the code in ingest.py. Replace the URL in ingest.py with a PDF link that matches your use case.

Docker-based bot deployment

1. Set the NVIDIA_API_KEY environment variable with your NVIDIA AI Catalog key before launching the bot.

   export NVIDIA_API_KEY=...
   

2. Copy the requirements from nv_langchain_bot/plugins/requirements_dev.txt into deploy/docker/dockerfiles/plugin_server.Dockerfile.

   ##############################
   # Install custom dependencies
   ##############################
   RUN pip3 install \
       langchain==0.0.322 \
       langchain-community==0.0.16 \
       langchain-nvidia-ai-endpoints==0.0.1 \
       pypdf==3.10.0 \
       pymilvus==2.3.1
   

3. Prepare the environment for the Docker compose commands.

   export BOT_PATH=./samples/nv_langchain_bot/
   source deploy/docker/docker_init.sh
   

4. Deploy the ACE Agent microservices. Deploy the Chat Engine, Plugin server, and NLP server microservices.

   docker compose -f deploy/docker/docker-compose.yml up --build chat-bot -d
   

5. Try out the bot using a web browser, you can deploy a sample frontend application with only textual chat support using the following command.

   docker compose -f deploy/docker/docker-compose.yml up frontend
   

6. Interact with the bot using the URL http://<workstation IP>:9001/.

Sample question - How many Americans receive Social Security benefits?