Quickstart Guide#
Please refer to this guide as you fine-tune your large language model with RTX Virtual Workstation.
Prerequisites#
Before you get started, go through the following to acquire the necessary software and hardware components.
Required Software#
Hypervisor: vGPU supported hypervisors
NVIDIA vGPU Software: vGPU version 17.4 or later
Linux VM Operating System: Ubuntu 24.04 or Ubuntu 22.04
Download NVIDIA AI Workbench for Ubuntu here
Warning
Install AI Workbench from the same account that will use AI Workbench.
Before downloading NVIDIA AI Workbench, please read the NVIDIA AI Product Agreement, the NVIDIA AI Workbench Shared Security Model, and our Data Privacy Policy. By downloading, installing, or using the NVIDIA AI Workbench software, you agree to the terms of the NVIDIA AI Product Agreement (EULA). If you do not agree to the terms of the EULA, you are not authorized to download, install, or use NVIDIA AI Workbench.
Activate, download, and install your RTX Virtual Workstation licenses
Request access to the large language model, LLaMa 3-8B on Hugging Face and generate an Access Token
Important
Don’t have an NVIDIA vGPU license yet? Request a free 90-day evaluation license
Required Hardware#
NVIDIA vGPU Certified Systems are equipped with NVIDIA L40S, L40, and L4. View the list of NVIDIA vGPU Certified Servers.
Fine-tuning in this deployment guide (4-bit quantization with Llama-3-8B-instruct)#
Minimum requirements: 16 vCPU, 24 GB system memory, 120 GB storage, 16Q Profile
Recommended: 32 vGPU, 128 GB system memory, 120 GB storage, 16Q Profile
For more detailed information on supported models, quantization bits, and hardware requirements, you can reference the Llamafactory project on GitHub.