NVIDIA Cloud Functions

NVIDIA Cloud Functions

• Overview
  • Function Types
  • Getting Started
• Quickstart
  • Clone, Build and Push the Docker Image to NGC Private Registry
  • Create & Deploy the Function Using the Cloud Functions UI
  • Create & Deploy the Function Using the Cloud Functions API
  • Create & Deploy the Function Using the NGC CLI
  • Invoke the Function
• Elastic NIM
  • About NIM
  • Prerequisites
    • Setup
  • NGC Authentication
  • Download the NIM
    • Export the NGC API Key
    • NGC CLI Tool
    • Docker Login to NGC
    • List Available NIMs
  • Upload NIM to Private Registry
  • Create the Function
  • Validating the Function Deployment
  • Troubleshooting
• API
  • OpenAPI Specification
  • Authorization
    • Generate an NGC Personal API Key
    • API Key Scopes and Domains
  • Using the NVCF Invocation API
    • HTTP (Polling)
      • Polling After Initial Invocation
      • Large Responses (302 Status Code)
    • HTTP Streaming
      • Advantages of HTTP Streaming
    • gRPC
  • Statuses and Errors
    • Inference Container Status Codes and Responses
    • NVCF API Status Codes
    • Common Function Invocation Errors
• Function Lifecycle
  • Key Concepts
  • Function States
  • Workflow
  • Function Lifecycle Endpoints
    • Function Creation, Management & Deployment
      • Function Metadata
    • Function Invocation
    • Asset Management
    • Visibility, Cluster Groups & GPUs
• Function Creation
  • Working with NGC Private Registry
    • Generate an NGC Personal API Key
    • Download & Configure the NGC CLI
    • Authenticate with NGC Docker Registry
    • (Optional) Push a Container to the NGC Private Registry
    • Best Practices with NGC Docker Registry and Cloud Functions
      • Container Versioning
      • Usage of NGC Teams
  • Container-Based Function Creation
    • Resources
    • Container Endpoints
    • Composing a FastAPI Container
      • Create the “requirements.txt” File
      • Implement the Server
      • Create the Dockerfile
      • Build the Container & Create the Function
    • Composing a PyTriton Container
      • Create the “requirements.txt” File
      • Create the “run.py” File
      • Create the “Dockerfile”
      • Build the Docker Image
      • Push the Docker Image
      • Create the Function
      • Additional Examples
    • Triton-based Container Configuration
      • Configuration
    • Creating Functions with NGC Models & Resources
    • Creating gRPC-based Functions
      • Prerequisites
      • gRPC Function Creation via UI
      • gRPC Function Creation via CLI
      • gRPC Function Creation via API
      • gRPC Function Invocation
    • Available Container Variables
    • Adding Partial Response (Progress)
      • Storing Partial and Complete Outputs
      • Creating a Progress File
      • Best Practices
  • Helm-Based Function Creation
    • Prerequisites
    • Secret Management
    • Create a Helm-based Function
    • Limitations
    • Helm Chart Overrides
• Secret Management
  • Adding Secrets During Function Creation
    • Using the UI
    • Using the API
  • Managing Secrets for Existing Functions
    • Function States and Secret Management Rules
    • Using the UI
    • Using the API
  • Listing Secrets in the API
  • Using Secrets In Inference Container
    • Secrets in Container-based Functions
    • Secrets in Helm Chart Functions
  • Considerations and Limitations
• Function Deployment
  • Deployment Validation
  • Deploying a Function
    • Key Concepts
      • Function Queueing
      • Autoscaling and Instance Counts
    • Deploy via the UI
    • Deploy via API
    • Deploy via CLI
    • Delete a Deployment
  • Deployment Failures
• Function Management
  • Function Versioning
  • Function Versioning Best Practices
    • Prerequisites
    • Function Update Flow (A/B Testing)
• Function Monitoring
  • Troubleshooting
  • Logging and Metrics
    • Emit and View Inference Container Logs
      • How to Add Logs to Your Inference Container
    • View Function Metrics
    • Instrument with OpenTelemetry
  • Logging and Metrics (Internal)
• Function Permissions
  • Listing Functions via API
• Asset Management
  • Creating an Asset ID and Pre-signed Upload URL
  • Uploading Assets to Cloud Storage
  • Listing Assets
  • Deleting Assets
  • Specifying Assets when invoking a Function
  • Using Assets with Custom Containers
  • Retrieving Asset Directory and Asset ID in Triton Python Backend
• Cluster Setup & Management
  • Prerequisites
    • Supported Kubernetes Versions
    • Considerations
  • Register the Cluster
    • Configuration
    • Advanced Settings
      • Caching Support
    • Install the Cluster Agent
  • View & Validate Cluster Setup
    • Verify Cluster Agent Installation via UI
    • Verify Cluster Agent Installation via Terminal
  • Deregister the Cluster
    • Delete the Cluster via UI
    • Delete the Cluster Agent and the NVCA Operator
  • Cluster Agent Monitoring and Reliability
    • Monitoring Data
      • Metrics
      • Logs
      • Tracing
    • Cluster Key Rotation
  • Advanced: NVCA Operator Configuration Options
    • Node Affinity
    • NVCA Operator Parameters
    • NGC Configuration
    • Node Selector Configuration
    • OpenTelemetry Configuration
  • Advanced: Manual Instance Configuration
• NGC Account Access
  • Login as the Organization Owner
• FAQ