Enterprise RAG Ingestion Scaling Guide

NVIDIA Enterprise Reference Architecture

• Abstract
• RAG Ingestion Pipeline Workflow
  • NV-Ingest Client Workflow
    • Client Side Concurrency
    • Client Side Batching
  • NV-Ingest Server Workflow
    • Server Side Concurrency
    • Document Concurrency Example
    • Text and Metadata Extraction
    • Multimodal Data Extraction
  • Vector DB Bulk Upload
  • Evaluate Accuracy
  • Performance Metrics and Tracing
    • Key Ingestion Performance Metrics
    • NV-Ingest Extraction Document Trace
    • NV-Ingest Latency and Queue Charts
    • NV-Ingest Metrics Monitoring: Request Rates, Resource GPU/CPU Utilization
• RAG Ingestion Performance Benchmarking
  • Benchmarking Methodology
  • Basic Validation - RAG Ingestor-server (NV-Ingest) Text-only Data Ingestion
  • Multimodal - NeMo Retriever (NV-Ingest) Extraction NIM Microservices
    • Scaling Multimodal Ingestion
    • NV-Ingest Scale 1X
    • NV-Ingest Scale 2X
    • NV-Ingest Scale 4X
    • Linear Scaling for Ingestion - Strong vs Weak Scaling
    • Multimodal Ingestion Performance Results
      • Minimum Baseline - Strong Scaling vs Weak Scaling
      • Run:ai Ingestion Performance with RTX PRO 6000
      • TCO - Multimodal NV-Ingest with Run:ai - Cost per 1 Million Vectors
      • MIG Ingestion Performance with RTX PRO 6000
      • TCO - Multimodal NV-Ingest with MIG - Cost per 1 Million Vectors
  • Text-only - NeMo Retriever (NV-Ingest) Ingestion
    • Performance Results - NV-Ingest Text
    • TCO - Text-only NV-Ingest - Cost per 10 Million Vectors
  • Wiki Text Bulk Ingest - Parquet Text-only Data
    • CPU: Wiki Text Bulk Ingest
    • GPU: Wiki Text Bulk Ingest
    • Wiki Text - Ingestion Performance Tests and Results
    • TCO - Wiki Text Ingest - Cost Per 10 Million Vectors
    • Optimizing Wiki Text Bulk Ingestion
• RAG Ingestion Sizing
  • Ingestion Sizing — T-Shirt Sizing Guide (Extended)
• Summary - Ingestion Best Practices
  • Ingestion Pipeline - Time Distribution
  • Text Chunking Strategy
  • Embedding and VDB Best Practices
  • Observability and Monitoring
  • Multimodal Ingestion Best Practices
  • GPU Sharing - Run:ai or MIG
  • Scaling NV-Ingest and NIM Microservices
  • Right Sizing NIM Fractions - Strong vs Weak Scaling
  • Concurrency - Document (client) vs Worker (server)
  • Data Preprocessing - Reduce E2E Ingestion Time
  • Text-only NV-Ingest Best Practices
  • Wiki Text Bulk Ingest Best Practices
    • Vector DB: GPU (cuVS) vs CPU Bulk Ingestion
    • Parallelization and Batch Sizing
    • Wiki Text Scaling Performance
  • Ingestion Sizing - T-Shirt Sizing Guide
• References

Notices

• Notices
  • NVIDIA Enterprise Reference Architecture Notice & Disclaimers