NVIDIA AI Enterprise Solution Guide

This solution guide outlines the creation of an AI pipeline on NVIDIA AI Enterprise by leveraging a Natural Language Processing use case example

NVIDIA AI Enterprise

• Enterprise-Grade AI Software Platform
  • Enterprise-Grade AI Software Platform From NVIDIA
  • NVIDIA AI Software Supported
  • NVIDIA AI Workflows
  • Production-Ready Pretrained Models
  • NVIDIA NIM
  • Supported Hardware and Software

Triton Inference Server

• Overview
  • Triton Inference Server
    • What is TensorRT?
    • Triton Inference Server Architecture
• User Personas
• IT Administrator
  • Creating an Ubuntu 20.04 Virtual Machine
  • Install NVIDIA Driver, Docker, and NVIDIA Container Toolkit
  • Additional Application Configuration
    • Configuring the VM for BERT Model Training and Inference
  • Create a template from the VM
    • Create a Guest Customization Specification
    • Create the Virtual Machine Template
• AI Practitioner
  • Train the BERT QA Model
  • Export Model to Triton Inference Server Format
• DevOps Engineer
  • Check Triton Inference Server Health
  • Validation
    • Getting the Triton Client Libraries and Examples
    • Using Triton gRPC client to run Inference

Appendix

• Benchmarking Triton Inference Server
  • Triton BERT Large Benchmarks
• Scaling Triton Inference Server
  • Vertical Scaling on vSphere VMs
  • Horizontal Scaling on vSphere VMs (Advanced)
    • Deploying a load balancer for Horizontal Scaling
    • Running Scale-Out Inference Using a Load Balancer
  • Deploying Triton Inference Server on Kubernetes
    • What is Kubernetes?
    • What is the NVIDIA GPU Operator?
    • IT Administrator Workflow
      • Clone VM - Standard OS Ubuntu 20.04
      • Install Kubernetes
    • DevOps Engineer Workflow
      • Install GPU Operator
      • Deploy Triton Inference Server on Kubernetes
        • Access an NFS share to hold all Trained GPU TensorFlow models
        • Manage the Storage
        • Create a Persistent Volume (PV) for Kubernetes to access the NFS server
        • Create a Persistent Volume Claim (PVC)
        • Create a Triton Inference Server Kubernetes Deployment
        • Check the health of Kubernetes deployment
    • Autoscaling the Triton Inference Server deployment with Kubernetes
      • Horizontal Pod auto scaler
        • Custom Metrics server
        • NVIDIA DCGM Exporter Service
        • Prometheus Server
        • Install Prometheus Adapter
        • Verify if the Custom Metrics are Available to the Metrics Server
        • Create a Horizontal Pod Auto Scaler Object
        • Generating Load to Show Auto Scaling

Support

• Support and Services

Notices

• Notices
  • Notice
  • Trademarks
  • Copyright