Quickstart Guide

Please refer to this guide as you fine-tune your large language model with RTX Virtual Workstation.

Before you get started, go through the following to acquire the necessary software and hardware components.

Required Software

• Hypervisor: vGPU supported hypervisors

• NVIDIA vGPU Software: vGPU version 17.4 or later

• Linux VM Operating System: Ubuntu 24.04 or Ubuntu 22.04

• Download NVIDIA AI Workbench for Ubuntu here

  Warning

  • Install AI Workbench from the same account that will use AI Workbench.

  • Before downloading NVIDIA AI Workbench, please read the NVIDIA AI Product Agreement, the NVIDIA AI Workbench Shared Security Model, and our Data Privacy Policy. By downloading, installing, or using the NVIDIA AI Workbench software, you agree to the terms of the NVIDIA AI Product Agreement (EULA). If you do not agree to the terms of the EULA, you are not authorized to download, install, or use NVIDIA AI Workbench.

  
  

• Activate, download, and install your RTX Virtual Workstation licenses

• Request access to the large language model, LLaMa 3-8B on Hugging Face and generate an Access Token

Required Hardware

NVIDIA vGPU Certified Systems are equipped with NVIDIA L40S, L40, and L4. View the list of NVIDIA vGPU Certified Servers.

Fine-tuning in this deployment guide (4-bit quantization with Llama-3-8B-instruct)

• Minimum requirements: 16 vCPU, 24 GB system memory, 120 GB storage, 16Q Profile

• Recommended: 32 vGPU, 128 GB system memory, 120 GB storage, 16Q Profile

For more detailed information on supported models, quantization bits, and hardware requirements, you can reference the Llamafactory project on GitHub.