NVIDIA Cloud Functions

NVIDIA Cloud Functions

• Overview
  • Function Types
  • Getting Started
• Functions Quickstart
  • Clone, Build and Push the Docker Image to NGC Private Registry
  • Create & Deploy the Function Using the Cloud Functions UI
  • Create & Deploy the Function Using the Cloud Functions API
  • Create & Deploy the Function Using the NGC CLI
  • Invoke the Function
• Tasks Quickstart
  • Clone, Build, and Push the Docker Image to NGC Private Registry
  • Create & Launch the Task Using the Cloud Functions UI
  • Create & Launch the Task Using the NGC CLI
  • Understanding the Tasks Sample
  • Verifying Task Results
  • Next Steps
• Elastic NIM
  • Benefits of Running NIM on NVCF
  • Prerequisites
• Create and Deploy the Function
  • Managing Your Function using the NGC CLI
    • Export NGC_API_KEY Environment Variable
    • List All Functions
    • Get Function Information (Specify Version)
    • Delete a Function
    • Test the NIM Function with Sample Input
  • Deployment Best Practices
  • Troubleshooting Guide
• Function Lifecycle
  • Key Concepts
  • Function States
  • Workflow
  • Function Lifecycle Endpoints
    • Function Creation, Management & Deployment
      • Function Metadata
    • Function Invocation
    • Asset Management
    • Visibility, Cluster Groups & GPUs
• Function Creation
  • Working with NGC Private Registry
    • Generate an NGC Personal API Key
    • Download & Configure the NGC CLI
    • Authenticate with NGC Docker Registry
    • (Optional) Push a Container to the NGC Private Registry
    • Best Practices with NGC Docker Registry and Cloud Functions
      • Container Versioning
      • Usage of NGC Teams
  • Container-Based Function Creation
    • Resources
    • Container Endpoints
    • Composing a FastAPI Container
      • Create the “requirements.txt” File
      • Implement the Server
      • Create the Dockerfile
      • Build the Container & Create the Function
    • Composing a PyTriton Container
      • Create the “requirements.txt” File
      • Create the “run.py” File
      • Create the “Dockerfile”
      • Build the Docker Image
      • Push the Docker Image
      • Create the Function
      • Additional Examples
    • Triton-based Container Configuration
      • Configuration
    • Creating Functions with NGC Models & Resources
    • Creating gRPC-based Functions
      • Prerequisites
      • gRPC Function Creation via UI
      • gRPC Function Creation via CLI
      • gRPC Function Creation via API
      • gRPC Function Invocation
    • Creating Low Latency Streaming (LLS AKA GameStreamSDK/WebRTC) Functions
      • Building the Streaming Server Application
      • Creating the LLS Streaming Function
      • Connecting to a streaming function with a client
        • Intermediary Proxy
        • Web Browser Client
    • Available Container Variables
    • Adding Partial Response (Progress)
      • Storing Partial and Complete Outputs
      • Creating a Progress File
      • Best Practices
  • Helm-Based Function Creation
    • Prerequisites
    • Secret Management
    • Create a Helm-based Function
    • Limitations
    • Helm Chart Overrides
• Secret Management
  • Adding Secrets During Function Creation
    • Using the UI
    • Using the API
  • Managing Secrets for Existing Functions
    • Function States and Secret Management Rules
    • Using the UI
    • Using the API
  • Listing Secrets in the API
  • Using Secrets In Inference Container
    • Secrets in Container-based Functions
    • Secrets in Helm Chart Functions
  • Considerations and Limitations
• Function Deployment
  • Deployment Validation
  • Deploying a Function
    • Key Concepts
      • Function Queueing
      • Autoscaling and Instance Counts
    • Deploy via the UI
    • Deploy via API
    • Deploy via CLI
    • Delete a Deployment
  • Deployment Failures
• Manually Deploy a NIM
  • Prerequisites
  • Create NGC Personal API Key
• Download the NIM
  • Export the NGC API Key
  • List Available NIMs
• Upload NIM to Private Registry
• Create the Function
• Validating the Function Deployment
• Troubleshooting
• Function Management
  • Function Versioning
  • Function Versioning Best Practices
    • Prerequisites
    • Function Update Flow (A/B Testing)
• Function Monitoring
  • Troubleshooting
  • Logging and Metrics
    • Emit and View Inference Container Logs
      • How to Add Logs to Your Inference Container
    • View Function Metrics
    • Instrument with OpenTelemetry
• Function Permissions
  • Listing Functions via API
  • Sharing a Function to Another NCA ID
• Function Invocation
  • Using the NVCF Invocation API
    • HTTP Streaming
      • Advantages of HTTP Streaming
    • HTTP (Polling)
      • Polling After Initial Invocation
      • Large Responses (302 Status Code)
    • gRPC
  • Statuses and Errors
    • Inference Container Status Codes and Responses
    • Common Function Invocation Errors
• Asset Management
  • Creating an Asset ID and Pre-signed Upload URL
  • Uploading Assets to Cloud Storage
  • Listing Assets
  • Deleting Assets
  • Specifying Assets when invoking a Function
  • Using Assets with Custom Containers
  • Retrieving Asset Directory and Asset ID in Triton Python Backend
• Tasks
  • Introduction
  • Task Management and Execution
    • Lifecycle of a Task
    • Environments
    • REST API Endpoints
      • Create Task
      • Container Argument Examples
      • List Tasks
      • Retrieve Task Details
      • Delete Task
      • Cancel Task
      • List Task Events
      • List Task Results
      • Update Task Secrets
  • Environment Variables Available in Task Container
  • Models and Resources
  • Task Secrets
  • Task Results
    • Handling Task Results
      • Result Handling Strategy: UPLOAD
      • Result Handling Strategy: NONE
    • progress File Format
    • Intermediate and Final Results
  • Task Runtime Duration
• Helm Chart Based Tasks
  • Prerequisites
  • 1. Essential Helm Chart Values
  • 2. Progress Updates
  • 3. Container Image Pull Secret Management
  • 4. Sample Helm Chart values.yaml
• Create a Helm-based Task
  • Creation from UI
  • Creation from API
  • Helm Chart Values Overrides
  • Limitations
  • 1. Disk Size
  • 2. Security Constraints
  • Restrictions
• Cluster Setup & Management
  • Prerequisites
    • Supported Kubernetes Versions
    • Considerations
  • Register the Cluster
    • Configuration
    • Advanced Settings
      • Caching Support
    • Install the Cluster Agent
  • View & Validate Cluster Setup
    • Verify Cluster Agent Installation via UI
    • Verify Cluster Agent Installation via Terminal
  • Upgrade the Cluster Agent
    • Operator Upgrade Required
    • NVCA-Only Upgrade
  • Deregister the Cluster
    • Prerequisites
    • Delete the Cluster via UI
    • Delete the Cluster Agent and the NVCA Operator
  • Cluster Agent Monitoring and Reliability
    • Monitoring Data
      • Metrics
      • Logs
      • Tracing
    • Cluster Key Rotation
  • Network Configuration
    • Key Network Requirements
    • Network Policy Customization via ConfigMap
    • Network Policy Customization via clusterNetworkCIDRs Flag
  • Advanced: NVCA Operator Configuration Options
    • Node Selection for Cloud Functions
    • Managing Feature Flags
    • Enable Helm Shared Storage
  • Advanced: Manual Instance Configuration
    • Manual Instance Type Configuration
      • Prerequisites
      • Configuration Fields
  • Cloud Provider Specific Requirements
    • Oracle Cloud Infrastructure (OCI)
• NGC Account Access
  • Login as the Organization Owner
• Cloud Tasks and Cloud Functions Member Roles
• API
  • OpenAPI Specification
  • Authorization
    • Generate an NGC Personal API Key
    • API Key Scopes and Domains
    • NVCF API Status Codes
• FAQ
• Support
• Service Status