RAG LangChain Bot#
This is an example chatbot that showcases RAG executed directly in the Plugin server using LangChain. It uses NVIDIA AI Catalog models and Milvus vector stores to perform RAG.
The RAG Langchain bot showcases the following ACE Agent features:
Integrating a custom RAG pipeline with ACE Agent
Installing custom dependencies in Plugin server
Returning a streaming text response from Plugin server
Can be deployed in either Chat Engine Server Architecture or Plugin Server Architecture
Prerequisites
Start the Milvus Vector Store using the Docker compose file.
docker compose -f samples/nv_langchain_bot/docker-compose-milvus.yml up -d
Install the
pymilvus
,pypdf
, andlangchain-nvidia-ai-endpoints
packages to prepare for ingestion.pip install langchain-nvidia-ai-endpoints==0.0.1 pypdf==3.10.0
Create an NVIDIA API Catalog API key.
Visit NVIDIA API Catalog website and login/sign-up: NVIDIA API Catalog. This gives you 1000 credits to use for free.
Select any model to visit the model usage page. On the right side of this page, you should see a Python code sample to use the model. Click
Get API key
button in the upper right corner to generate an API key.
Ingest the sample URL by moving into the
plugins
directory and running the code iningest.py
. Replace the URL iningest.py
with a PDF link that matches your use case.
Docker-based bot deployment
Set the
NVIDIA_API_KEY
environment variable with your NVIDIA AI Catalog key before launching the bot.export NVIDIA_API_KEY=...
Copy the requirements from
nv_langchain_bot/plugins/requirements_dev.txt
intodeploy/docker/dockerfiles/plugin_server.Dockerfile
.############################## # Install custom dependencies ############################## RUN pip3 install \ langchain==0.0.322 \ langchain-community==0.0.16 \ langchain-nvidia-ai-endpoints==0.0.1 \ pypdf==3.10.0 \ pymilvus==2.3.1
Prepare the environment for the Docker compose commands.
export BOT_PATH=./samples/nv_langchain_bot/ source deploy/docker/docker_init.sh
Deploy the ACE Agent microservices. Deploy the Chat Engine, Plugin server, and NLP server microservices.
docker compose -f deploy/docker/docker-compose.yml up --build chat-bot -d
Try out the bot using a web browser, you can deploy a sample frontend application with only textual chat support using the following command.
docker compose -f deploy/docker/docker-compose.yml up frontend
Interact with the bot using the URL
http://<workstation IP>:9001/
.
Sample question - How many Americans receive Social Security benefits?