Multi-Node Training for AI on Kubernetes (VMware Tanzu)

AI Practitioner

• Overview
• Multi-Node Training with Tanzu Overview
  
  • Distributed Training
    
    • Model Parallelism
    • Data Parallelism
  • Horovod - Framework support for Data Parallelism
    
    • What is Message Passing Interface (MPI)?
    • What is the NVIDIA Collective Communications Library (NCCL)?
    • Hardware Used For Distributed Training
    • GPUDirect RDMA
  • How to write distributed training workloads on GPUs using Horovod
• Kubernetes Overview
  
  • Why use Kubernetes for Multi Node workflows?
  • What are the GPU and MPI operators?
  • Kubernetes with VMware Tanzu
• Step #1: Single-Node Training
  
  • Lab Overview
• Step #2: Multi-Node Training
• Next Steps

Notices

• Legal
  
  • Agreements
  • Privacy Policy
  • Notice
  • Trademarks
  • Copyright