Quickstart Guide

Please refer to this guide as you build an RAG application with RTX Virtual Workstation.

Prerequisites

Before you get started, go through the following to acquire the necessary software and hardware components.

Required Software

• Hypervisor: vGPU supported hypervisors

• NVIDIA vGPU Software: vGPU version 17.4 or later

• Linux VM Operating System: Ubuntu 24.04 or Ubuntu 22.04

• Download NVIDIA AI Workbench for Ubuntu here (AI vWS backend)

Warning

• Install AI Workbench from the same account that will use AI Workbench.

• Before downloading NVIDIA AI Workbench, please read the NVIDIA AI Product Agreement, the NVIDIA AI Workbench Shared Security Model, and our Data Privacy Policy. By downloading, installing, or using the NVIDIA AI Workbench software, you agree to the terms of the NVIDIA AI Product Agreement (EULA). If you do not agree to the terms of the EULA, you are not authorized to download, install, or use NVIDIA AI Workbench.

• Activate, download, and install your RTX Virtual Workstation licenses

• Select and download a large language model (LLaMa 3-8B is recommended) that can be sourced from Hugging Face or GitHub for building an RAG application

Important

Don’t have a license yet? Request a free 90-day evaluation license

Required Hardware

NVIDIA vGPU Certified Systems equipped with NVIDIA L40S, L40, L4, A40, A10, or T4. View the list of NVIDIA vGPU Certified Servers.

• Minimum requirements of Linux VM configuration: 8 vCPU, 32 GB system memory, 120 GB storage

• Recommended vGPU profile for Linux VM: 16Q (4-bit model) or 24Q (8 bit model)