NIM Operator Deployment

This guide walks you through deploying NVIDIA cuOpt as a Kubernetes service using the NIM Operator.

Overview

The NIM Operator simplifies the deployment and management of NVIDIA NIM microservices on Kubernetes. This deployment method provides:

• Automated lifecycle management - The operator handles scaling, updates, and health checks

• Native Kubernetes integration - Uses standard K8s patterns (CRDs, services, ingress)

• GPU resource management - Leverages the GPU Operator for optimal GPU utilization

• Built-in monitoring - Prometheus metrics and service monitors

Supported GPUs

The following NVIDIA GPUs are supported:

GPU	Architecture	Notes
A100	Ampere	Recommended for production
H100	Hopper	High performance
H200	Hopper	High performance with extended memory
B200	Blackwell	Latest generation
RTX 6000 Pro	Ada Lovelace	Workstation GPU

Quick Start

1. Complete all prerequisites

2. Set your NGC API Key:

   export NGC_API_KEY=<your-ngc-api-key>
   

3. Run the deployment script:

   ./deploy.sh
   

4. Verify the deployment:

   kubectl get nimservice -n nim-service
   kubectl get pods -n nim-service
   

Documentation Contents

• Prerequisites
  • Kubernetes Cluster Setup
  • GPU Operator Installation
  • Storage Provisioner
  • NIM Operator Installation
  • NGC API Key
  • Verification Checklist
  • Next Steps
• Deployment Guide
  • Automated Deployment
  • Manual Deployment
  • Verifying the Deployment
  • Testing the Deployment
  • Cleanup
  • Troubleshooting
  • Next Steps
• Configuration Guide
  • Image Configuration
  • Resource Configuration
  • Environment Variables
  • Storage Configuration
  • Networking Configuration
  • Scaling Configuration
  • Health Probes
  • Monitoring Configuration
  • Full Configuration Example

Additional Resources

• NVIDIA cuOpt Documentation

• NIM Operator Documentation

• GPU Operator Documentation