Introduction#
The current enterprise landscape is characterized by a rapidly escalating volume of proprietary, internal data—often siloed within numerous systems like document repositories, knowledge bases, and collaborative platforms. Simultaneously, the demand for sophisticated, evidence-based decision-making is driving enterprises toward deploying advanced AI systems. However, traditional Large Language Model (LLM) deployments face significant hurdles:
Data Siloing & Limited RAG: Internal knowledge is untapped because LLMs use public data. Basic RAG lacks multi-step reasoning across diverse, proprietary sources.
Lack of Agency & Complex Reasoning: LLMs struggle with goal-oriented tasks needing planning and iterative refinement. Users need a “Research Agent” for synthesizing reports from internal and external data.
Deployment Complexity & Scalability: Deploying powerful agentic AI requires significant GPUs and complex orchestration. Lack of sizing guidance leads to high costs and bottlenecks.
NVIDIA Blueprints simplify the deployment and management of complex AI systems offering pre-validated architecture that automates dependency management between various components, and uses Helm for easy deployment and scaling. The NVIDIA AI-Q Research Agent blueprint directly addresses these challenges by providing a robust, scalable, and agentic framework. AI-Q uses RAG for retrieval, can consume multiple data sources, reason between different data sources, search the web for any additional research and provide a detailed report on a given subject. AI-Q provides the open blueprint and reference architecture for building next‑generation agents, while NeMo Agent Toolkit is the underlying control layer, that integrates with other frameworks like LangChain, LlamaIndex, etc., and connects, profiles, and optimizes AI agents across frameworks and infrastructure..
Scope#
This paper covers how to scale, size and optimize AI-Q Research Agent and covers the following components on Enterprise Reference Architecture. Out of scope: AI-Q allows a human in the loop feature, this feature was not used when conducting performance tests for AI-Q.
Table 1: Scope of NVIDIA Blueprints, NVIDIA NIM versions along with GPUs tested
NVIDIA Components |
NVIDIA GPUs |
|---|---|
AI-Q Blueprint v.1.2.0 w/ Meta Llama 70B NIM v latest |
RTX PRO 6000 BSE |
RAG Blueprint v.2.3.0 With Nemotron Super 49B 1.5 NIM v 1.14.0 |
RTX PRO 6000 BSE |
RTX PRO 6000 BSE |
Target Audience#
This guide is meant to help NVIDIA partners architecting agentic solutions for deep research based on existing enterprise data. This guide helps determine Infrastructure and sizing requirements for cluster-level deployments. The guide can be used for both new and existing deployments to determine the capacity and scale needed based on Deep Research workload. It can also be used by the following Personas as they architect their Enterprise solutions.
Enterprise Architects: Enterprise Architects tasked with designing and defining servers, GPU’s and Networking gear to determine what Infrastructure resources will be needed to support Deep Research agents.
MLOps Engineer: MLOps Engineers can use this to define Infrastructure requirements as they talk to Infrastructure/Cloud teams to carve out resources to run Agentic workflows for Deep Research.
Platform Engineer: Platform Engineers can use this guide to determine how to design their Container Environment around Kubernetes, and also what kind of resources will be needed by the cluster to support Deep research agents.