NVIDIA Run:ai on DGX Cloud Documentation

Run:ai on DGX Cloud Manual

• 1. Product Overview
  • 1.1. Cluster Architecture
  • 1.2. Shared Responsibility Model
    • 1.2.1. NVIDIA Responsibilities
    • 1.2.2. End User Responsibilities
  • 1.3. Cluster Ingress and Egress
  • 1.4. Cluster User Scopes and User Roles
    • 1.4.1. User Scopes
      • 1.4.1.1. NVIDIA Run:ai Departments
      • 1.4.1.2. NVIDIA Run:ai Projects
    • 1.4.2. Cluster Users
      • 1.4.2.1. Customer Admin Role
      • 1.4.2.2. Customer User Roles
  • 1.5. Next Steps
• 2. Cluster Onboarding Guide
  • 2.1. Introduction
  • 2.2. Admin Steps
    • 2.2.1. Overview of Personas in Run:ai on DGX Cloud
    • 2.2.2. Accessing the Run:ai on DGX Cloud Cluster
    • 2.2.3. Setting up the NVIDIA Run:ai and Kubernetes CLI
    • 2.2.4. Creating a Department in NVIDIA Run:ai
    • 2.2.5. Creating a Project in NVIDIA Run:ai
    • 2.2.6. Assigning User Roles in NVIDIA Run:ai
    • 2.2.7. Creating an Environment
    • 2.2.8. Creating a Data Source
  • 2.3. User Steps
    • 2.3.1. Running Sample Workloads
      • 2.3.1.1. Running a Single Node NCCL Test
        • 2.3.1.1.1. Prerequisites
        • 2.3.1.1.2. Instructions
      • 2.3.1.2. Running a Multi-Node NCCL Test
        • 2.3.1.2.1. Prerequisites
        • 2.3.1.2.2. Instructions
      • 2.3.1.3. (Optional) Running a Single Node NeMo Interactive Workload
      • 2.3.1.4. (Optional) Downloading Data into the Data Source
  • 2.4. Conclusion
• 3. Cluster Administrator Guide
  • 3.1. Cluster Handover
  • 3.2. Accessing your Cluster
    • 3.2.1. Accessing the NVIDIA Run:ai UI
  • 3.3. Managing Users
    • 3.3.1. Creating Users
      • 3.3.1.1. Assigning Roles in NVIDIA Run:ai
      • 3.3.1.2. Access Rule Creation in NVIDIA Run:ai
  • 3.4. Managing Departments and Projects
    • 3.4.1. Departments
      • 3.4.1.1. Modifying the Default Department
      • 3.4.1.2. Creating a Department
    • 3.4.2. Projects
      • 3.4.2.1. Creating a Project
      • 3.4.2.2. Editing a Project
      • 3.4.2.3. Updating Access to Projects
  • 3.5. Managing Compute Resources
    • 3.5.1. Creating a New Compute Resource
  • 3.6. Managing Your Storage Utilization (CLI)
  • 3.7. Restricting Container Image Access
  • 3.8. Managing Cluster Updates and Upgrades
• 4. Cluster User Guide
  • 4.1. Logging Into Your Cluster
    • 4.1.1. Accessing the NVIDIA Run:ai UI
    • 4.1.2. Overview
  • 4.2. Understanding Your Role and Scope
    • 4.2.1. Learning Your User Role(s)
  • 4.3. Managing Workloads
  • 4.4. Managing Environments
    • 4.4.1. Creating a New Environment
    • 4.4.2. Editing an Environment
  • 4.5. Managing Data Storage on the Cluster
    • 4.5.1. Data Sources
      • 4.5.1.1. PVC
        • 4.5.1.1.1. Creating a PVC
      • 4.5.1.2. Git
        • 4.5.1.2.1. Creating a Git Data Source
      • 4.5.1.3. ConfigMap
    • 4.5.2. Scratch/Ephemeral Storage
  • 4.6. Setting Up Credentials for Accessing Registries and Data
    • 4.6.1. Accessing Your NGC Org
    • 4.6.2. Setting Up Your NGC API Key
    • 4.6.3. Adding NGC Credentials to the NVIDIA Run:ai on DGX Cloud Cluster
      • 4.6.3.1. Adding a Docker Registry Credential
      • 4.6.3.2. Adding a Generic Secret Credential
    • 4.6.4. Adding Git Credentials
  • 4.7. Setting Up Templates
• 5. CLI/API Setup Guide
  • 5.1. Accessing the NVIDIA Run:ai CLI
    • 5.1.1. CLI Prerequisites
    • 5.1.2. Downloading the NVIDIA Run:ai CLI
    • 5.1.3. Setting up Your Kubernetes Configuration File
  • 5.2. Kubernetes Usage for Researchers
  • 5.3. Advanced Kubernetes Usage for Admins
  • 5.4. Setting up the REST API on NVIDIA Run:ai on DGX Cloud
    • 5.4.1. References
• 6. Storage User Guide
  • 6.1. Storage Classes
    • 6.1.1. Enterprise Storage Profiles
  • 6.2. Object Storage
    • 6.2.1. Creating a New AWS Object Storage Resource
      • 6.2.1.1. Multiple Pods and S3 Bucket Combinations for AWS
      • 6.2.1.2. Mount Semantics for AWS S3
    • 6.2.2. Destroying an Object Storage Resource
  • 6.3. Volume Protection
  • 6.4. Data Volumes in Run:ai on DGX Cloud

Tutorials

• 1. Data Download Examples
  • 1.1. Download Data From S3 in a Training Workload
    • 1.1.1. Prerequisites and Requirements
    • 1.1.2. Creating the Credentials
    • 1.1.3. Creating the Data Source
    • 1.1.4. Creating the Training Workload
    • 1.1.5. Cleaning up the Environment
  • 1.2. Using Your Data From Amazon S3
    • 1.2.1. Example: Using S3 to copy data to PVC within NVIDIA Run:ai
    • 1.2.2. Example: Using data directly within AWS Services
• 2. Interactive Workload Examples
  • 2.1. Interactive NeMo Workload Job
    • 2.1.1. Prerequisites and Requirements
    • 2.1.2. Creating the Data Source
    • 2.1.3. Creating the Workspace
    • 2.1.4. Running the Notebook
    • 2.1.5. Persisting the Model to Storage
    • 2.1.6. Cleaning up the Environment
  • 2.2. RAPIDS and Polars Workspace
    • 2.2.1. Prerequisites and Requirements
    • 2.2.2. Creating the Workspace
    • 2.2.3. Downloading the Polars Notebook
    • 2.2.4. Running the Polars Notebook
    • 2.2.5. Cleaning up the Environment
  • 2.3. Running Visual Studio Code Server
    • 2.3.1. Prerequisites and Requirements
    • 2.3.2. Creating the Workspace via CLI on a local workstation
    • 2.3.3. Cleaning up the Environment
  • 2.4. Building and Running an Interactive PyTorch Job with Visual Studio Code
    • 2.4.1. Prerequisites and Requirements
    • 2.4.2. Creating the Training Script
    • 2.4.3. Creating and Using a Custom Docker Container
    • 2.4.4. Run the Custom Container on Run:ai on DGX Cloud Using the NVIDIA Run:ai CLI
    • 2.4.5. Connect to the Running Workload Locally Using Visual Studio (via SSH)
    • 2.4.6. Cleaning up the Environment
  • 2.5. Using WandB with a Workspace
    • 2.5.1. Prerequisites and Requirements
    • 2.5.2. Creating the Credentials
    • 2.5.3. Creating the Workspace
    • 2.5.4. Connecting to WandB in your Jupyter Notebook
    • 2.5.5. Cleaning up the Environment
  • 2.6. Using BioNeMo Framework for ESM-2nv Data Preprocessing and Model Training
    • 2.6.1. Prerequisites and Requirements
    • 2.6.2. Creating the Data Source
    • 2.6.3. Creating an Interactive Workspace for Preprocessing
    • 2.6.4. Using Workspace to Preprocess Data for ESM-2nv
    • 2.6.5. Pretraining
      • 2.6.5.1. Single GPU Training in Jupyter Notebook
      • 2.6.5.2. Multi-Node Distributed Pretraining
• 3. Distributed Training Workload Examples
  • 3.1. Distributed PyTorch Training Job
    • 3.1.1. Requirements
    • 3.1.2. Creating a Custom Docker Container
    • 3.1.3. Creating a Data Source
    • 3.1.4. Creating the Workload via the UI
    • 3.1.5. Monitoring the Job
    • 3.1.6. Getting the Checkpoint
    • 3.1.7. Cleaning up the Environment
  • 3.2. Using Zero Quota Projects for Lower Priority Workloads
    • 3.2.1. Requirements
    • 3.2.2. Creating the Maximum Quota Project via the UI
    • 3.2.3. Creating the Zero Quota Project via the UI
    • 3.2.4. Creating the Lower Priority Interactive Workload via the UI
    • 3.2.5. Creating the Higher Priority Training Workload via the UI
    • 3.2.6. Cleaning up the Environment
• 4. NeMo End-to-End Workflow Example
  • 4.1. Requirements
  • 4.2. Initial Setup
    • 4.2.1. Create a Service Account
    • 4.2.2. Create a Persistent Volume Claim (PVC)
    • 4.2.3. Create a Hugging Face Account
    • 4.2.4. Accept the Llama-3.1-8B License
    • 4.2.5. Create a Weights & Biases Account
    • 4.2.6. Set Up VS Code Locally
  • 4.3. Prepare the Data
    • 4.3.1. Set Up the Script
    • 4.3.2. Launch Data Preparation
  • 4.4. Pre-Train the Model
    • 4.4.1. Set Up the Environment
    • 4.4.2. Launch the Pre-Training Job
    • 4.4.3. View the Project Dashboard on Weights & Biases
  • 4.5. Deploy the Model for Inference
    • 4.5.1. Send Requests to the Deployed Model
    • 4.5.2. Clean Up
• 5. Inference Examples
  • 5.1. Using the UI
    • 5.1.1. Prerequisites and Requirements
    • 5.1.2. Create an Environment
    • 5.1.3. Optional: Create a PVC Datastore
    • 5.1.4. Create Workload
• 6. MLOps Integration Examples
  • 6.1. Argo Workflows Integration
    • 6.1.1. Prerequisites and Requirements
    • 6.1.2. Installing Argo Workflows CLI
    • 6.1.3. Configure Target Namespace
    • 6.1.4. Installing Argo Workflows (Admin Only)
    • 6.1.5. Basic Argo Workflows Usage
      • 6.1.5.1. Hello World
      • 6.1.5.2. Hello GPU
    • 6.1.6. Access Argo Workflows Web Interface
    • 6.1.7. Clean Up
  • 6.2. Ray Cluster Integration
    • 6.2.1. Prerequisites and Requirements
    • 6.2.2. Deploy a KubeRay Operator
    • 6.2.3. Deploy a KubeRay Cluster
    • 6.2.4. Submit a Ray Job
      • 6.2.4.1. Execute a Ray job in the head pod
      • 6.2.4.2. Submit a Ray job to the Ray Cluster via ray job submission SDK
    • 6.2.5. Cleanup

Appendix

• AWS-Based Run:ai on DGX Cloud Cluster Configuration
  • Amazon Elastic Fabric Adapters
    • Troubleshooting
  • (Optional) Private Access
    • AWS PrivateLink
      • Introduction
      • Assumptions
      • Configuration
        • Validation
        • PrivateLink Endpoints
        • DNS
      • Usage
      • Resources
        • Required Permissions
    • AWS Transit Gateway
      • Introduction
      • Assumptions
      • Getting Started
      • (Example) AWS Transit Gateway Private Access Deployment
• Common Issues and Debugging Tips
  • Common Issues and Resolutions
• Release Notes
  • NVIDIA Run:ai on DGX Cloud (Current)
    • Feature
    • Changes
    • Removed
    • SaaS
  • 1.3.1
    • Feature
    • Fixed
    • Known Issues
  • 1.3.0
    • Feature
    • Fixed
    • Known Issues
  • 1.2.2
    • Feature
    • Fixed
• Notices
  • Notice
  • Trademarks
  • Copyright
• Documentation Feedback