Is this page helpful?

NVIDIA Cloud Functions

NVIDIA Cloud Functions

• Overview
  • Function Types
  • Getting Started
• Functions Quickstart
  • Clone, Build and Push the Docker Image to NGC Private Registry
  • Create & Deploy the Function Using the Cloud Functions UI
  • Create & Deploy the Function Using the Cloud Functions API
  • Create & Deploy the Function Using the NGC CLI
  • Invoke the Function
• Tasks Quickstart
  • Clone, Build, and Push the Docker Image to NGC Private Registry
  • Create & Launch the Task Using the Cloud Functions UI
  • Create & Launch the Task Using the NGC CLI
  • Understanding the Tasks Sample
  • Verifying Task Results
  • Next Steps
• Elastic NIM
  • Benefits of Running NIM on NVCF
  • Prerequisites
• Create and Deploy the Function
  • Managing Your Function using the NGC CLI
    • Export NGC_API_KEY Environment Variable
    • List All Functions
    • Get Function Information (Specify Version)
    • Delete a Function
    • Test the NIM Function with Sample Input
  • Deployment Best Practices
  • Troubleshooting Guide
• Function Lifecycle
  • Key Concepts
  • Function States
  • Workflow
  • Function Lifecycle Endpoints
    • Function Creation, Management & Deployment
      • Function Metadata
    • Function Invocation
    • Visibility, Cluster Groups & GPUs
• Function Creation
  • Working with NGC Private Registry
    • Generate an NGC Personal API Key
    • Download & Configure the NGC CLI
    • Authenticate with NGC Docker Registry
    • (Optional) Push a Container to the NGC Private Registry
    • Best Practices with NGC Docker Registry and Cloud Functions
      • Container Versioning
      • Usage of NGC Teams
      • Security
  • Container-Based Function Creation
    • Resources
    • Container Endpoints
    • Composing a FastAPI Container
      • Create the “requirements.txt” File
      • Implement the Server
      • Create the Dockerfile
      • Build the Container & Create the Function
    • Composing a PyTriton Container
      • Create the “requirements.txt” File
      • Create the “run.py” File
      • Create the “Dockerfile”
      • Build the Docker Image
      • Push the Docker Image
      • Create the Function
      • Additional Examples
    • Creating Functions with NGC Models & Resources
    • Creating gRPC-based Functions
      • Prerequisites
      • gRPC Function Creation via UI
      • gRPC Function Creation via CLI
      • gRPC Function Creation via API
      • gRPC Function Invocation
    • Creating Low Latency Streaming (LLS AKA GameStreamSDK/WebRTC) Functions
      • Building the Streaming Server Application
      • Creating the LLS Streaming Function
      • Connecting to a streaming function with a client
        • Intermediary Proxy
        • Web Browser Client
    • Available Container Variables
    • Environment Variables
  • Helm-Based Function Creation
    • Prerequisites
    • Secret Management
    • Create a Helm-based Function
    • Limitations
  • 1. Asset caching
  • 2. Inference
  • 3. Security Constraints
    • Helm Chart Overrides
• Secret Management
  • Adding Secrets During Function Creation
    • Using the UI
    • Using the API
  • Managing Secrets for Existing Functions
    • Function States and Secret Management Rules
    • Using the UI
    • Using the API
  • Listing Secrets in the API
  • Using Secrets In Inference Container
    • Secrets in Container-based Functions
    • Secrets in Helm Chart Functions
  • Considerations and Limitations
• Function Deployment
  • Deployment Validation
  • Deploying a Function
    • Key Concepts
    • Function Queueing
    • Autoscaling and Instance Counts
    • Deploy via the UI
    • Deploy via API
    • Deploy via CLI
    • Delete a Deployment
  • Function Deployment Status
    • Overview
    • Example Use Cases
    • Deployment Stages Details
    • Deployment Status Dashboard
      • Viewing Deployment Status
  • Deployment Failures
• Manually Deploy a NIM
  • Prerequisites
  • Create NGC Personal API Key
• Download the NIM
  • Export the NGC API Key
  • List Available NIMs
• Upload NIM to Private Registry
• Create the Function
• Validating the Function Deployment
• Troubleshooting
• Function Management
  • Function Versioning
  • Function Versioning Best Practices
    • Prerequisites
    • Function Update Flow (A/B Testing)
• Function Permissions
  • Listing Functions via API
  • Sharing a Function to Another NCA ID
• Function Invocation
  • Using the NVCF Invocation API
    • HTTP Streaming
      • Advantages of HTTP Streaming
    • gRPC
  • Statuses and Errors
    • Inference Container Status Codes and Responses
    • Common Function Invocation Errors
• Third-Party Registries
  • Overview
    • Docker Hub
      • Prerequisites
        • Docker Hub Account Requirements
        • Personal Access Token Setup
        • Image Architecture Requirements
      • Adding Registry Credentials
        • In the UI
        • In the CLI
        • In the NGC API
        • Base64 Encoding Credentials
      • Using Credentials
      • References
        • Docker CLI Push Example
        • Helm Charts
    • AWS ECR
      • Prerequisites
        • AWS Account Requirements
        • IAM Permissions for Private ECR
        • IAM Permissions for Public ECR
        • Creating AWS Access Keys
        • Registry Hostname Patterns
        • Image Architecture Requirements
      • Adding Registry Credentials
        • In the UI
        • In the CLI
        • In the NGC API
        • Base64 Encoding Credentials
      • Using Credentials
      • Troubleshooting
        • Common Issues and Solutions
      • References
        • AWS ECR Documentation
        • Related Documentation
  • Common Requirements
    • Image Architecture Requirements
    • Authentication Requirements
• Tasks
  • Introduction
  • Task Management and Execution
    • Lifecycle of a Task
    • Environments
    • REST API Endpoints
      • Create Task
      • Container Argument Examples
      • List Tasks
      • Retrieve Task Details
      • Delete Task
      • Cancel Task
      • List Task Events
      • List Task Results
      • Update Task Secrets
  • Environment Variables Available in Task Container
  • Models and Resources
  • Task Secrets
  • Task Results
    • Handling Task Results
      • Result Handling Strategy: UPLOAD
      • Result Handling Strategy: NONE
    • progress File Format
    • Intermediate and Final Results
  • Task Runtime Duration
• Helm Chart Based Tasks
  • Prerequisites
  • 1. Essential Helm Chart Values
  • 2. Progress Updates
  • 3. Container Image Pull Secret Management
  • 4. Sample Helm Chart values.yaml
• Create a Helm-based Task
  • Creation from UI
  • Creation from API
  • Helm Chart Values Overrides
  • Limitations
  • 1. Disk Size
  • 2. Asset caching
  • 3. Security Constraints
• Cluster Setup & Management
  • Prerequisites
    • Supported Kubernetes Versions
    • Considerations
  • Register the Cluster
    • Configuration
    • Advanced Settings
      • Caching Support
      • NVLink-optimized Clusters
    • Cluster Maintenance Mode
      • Cordon Maintenance
      • Cordon And Drain Maintenance
    • Install the Cluster Agent
  • View & Validate Cluster Setup
    • Verify Cluster Agent Installation via UI
    • Verify Cluster Agent Installation via Terminal
  • Upgrade the Cluster Agent
    • Operator Upgrade Required
    • NVCA-Only Upgrade
    • Force NVCA Rollout
  • Deregister the Cluster
    • Prerequisites
    • Delete the Cluster via UI
    • Delete the Cluster Agent and the NVCA Operator
  • Cluster Agent Monitoring and Reliability
    • Monitoring Data
      • Metrics
      • Logs
      • Tracing
    • Cluster Key Rotation
      • Frequently Asked Questions
  • Network Configuration
    • Key Network Requirements
    • Network Policy Customization via ConfigMap
    • Network Policy Customization via clusterNetworkCIDRs Flag
  • Advanced: Additional Configuration Options
    • CSI Volume Mount Options
    • Node Selection for Cloud Functions
    • GPU Product Name Override
    • Managing Feature Flags
    • Enable Helm Shared Storage
  • Advanced: Helm-Managed Cluster Configuration
    • Switching to Helm-Managed Mode
    • Configuration Parameters
    • Updating Configuration
    • Upgrading NVCA Version
    • Switching Back to NGC-Managed Mode
    • Limitations
    • Tips
    • Advanced Parameter Reference
      • NVCA Operator Parameters
      • NGC Configuration Parameters
      • Node Selector Configuration
      • OpenTelemetry Configuration
      • Network Policy Configuration
      • Example Usage
  • Advanced: Manual Instance Configuration
    • Manual Instance Type Configuration
      • Prerequisites
      • Configuration Fields
  • Cloud Provider Specific Notes
    • Oracle Cloud Infrastructure (OCI)
    • AWS
• NGC Account Access
  • Login as the Organization Owner
• Cloud Tasks and Cloud Functions Member Roles
• Observability Guide
  • Overview
  • NGC UI Observability
    • Overview
      • How to Access
    • Logs
    • Viewing Metrics
  • External Observability
    • Ports
    • Configuration
    • Resource Management
    • Security
  • How to Set Up External Observability on a BYOC Cluster
    • BYOC Steps
  • Error Handling
  • Appendix A: Terminology
  • Appendix B: Available Metrics
    • CPU Metrics
      • Memory Metrics
      • Filesystem Metrics
      • Network Metrics
      • Kubernetes State Metrics
      • GPU Metrics
      • NVCF Worker Service Metrics
      • NVCA Instance Type Metrics
      • OpenTelemetry Collector Metrics
      • Resource Attributes
  • Appendix C: Adding Custom Application Metrics/Logs/Traces
• Debuggability Guide
  • Overview
  • Real-Time Logs
    • Key Benefits
    • Getting Started
  • Remote Command Execution
    • Key Benefits
    • Getting Started
  • NGC CLI Requirements and Examples
    • CLI Version Requirements
    • Detailed CLI Examples
  • Supported Commands
  • Security
  • Troubleshooting
    • Common Error Codes
    • Required Permissions
  • Limitations
  • Appendix A: Terminology
• API
  • OpenAPI Specification
  • Authorization
    • Generate an NGC Personal API Key
    • API Key Scopes and Domains
    • NVCF API Status Codes
• FAQ
• Support
• Service Status