Abstract

Enterprise RAG is only as effective as its ability to retrieve the right enterprise context at the right time. As organizations move from proof-of-concept to production, optimizing the retrieval path becomes critical to balancing accuracy, latency, and throughput while delivering responsive experiences for AI applications and agents.

Building on Enterprise RAG deployments based on NVIDIA’s RAG Blueprint using the Enterprise RAG Deployment Guide v2.0, and assuming enterprise data has already been ingested using the Enterprise RAG Ingestion Scaling Guide v2.0, this document provides guidance and insights for scaling, sizing, and tuning the retrieval pipeline. The recommendations in this guide help organizations optimize retrieval performance to meet the latency, throughput, and accuracy requirements of specific enterprise use cases on the NVIDIA Enterprise Reference Architecture for NVIDIA RTX PRO™ 6000 Blackwell Server Edition and NVIDIA H200 NVL systems.

This document is the third in a three-part series covering the deployment, ingestion, and retrieval stages of Enterprise RAG:

• Enterprise RAG Deployment Guide v2.0

• Enterprise RAG Ingestion Scaling Guide v2.0

• Enterprise RAG Retrieval Scaling and Sizing Guide v2.0