Cloud FunctionsThis guide provides instructions for installing Low Latency Streaming (LLS) components for self-hosted NVCF deployments. LLS enables real-time streaming capabilities for simulation workloads.
This guide covers manual installation of LLS resources, image mirroring, and NLB resources.
Low Latency Streaming (LLS) provides real-time streaming capabilities for self-hosted NVCF deployments. LLS consists of:
Self-hosted mode uses your ECR registry for container images and customer-managed infrastructure.
The following diagram illustrates the request and streaming flow for self-hosted NVCF LLS, showing how a browser application invokes a function and establishes direct streaming channels to worker nodes:
The architecture demonstrates a nine-step process:
The following diagram shows the detailed architecture of a self-hosted NVCF LLS deployment in AWS VPC with EKS:
Key components and data flows:
Control Plane Components (EKS):
GPU Nodes (EKS):
AWS VPC:
Data Flow Channels:
Before installing LLS, ensure:
nvcf-onprem organizationConfigure AWS Credentials
LLS installation requires AWS CLI access to create NLB resources, security groups, and manage EC2 instances. Verify your AWS credentials are configured:
If this command fails, configure your AWS credentials before proceeding using your organization’s standard AWS authentication flow.
Set NGC API Key
Before proceeding, set your NGC API key as an environment variable:
Configurable UDP Streaming Port
The UDP port for WebRTC streaming traffic defaults to 5004 and is exported as
LLS_UDP_PORT in Step 2.1. This variable is used throughout
the guide for the NLB security group rule, NLB listener, and Helm values configuration.
To use a different port, change the export LLS_UDP_PORT= value in Step 2.1 and set
env.lbPort in your Helm values file (Step 3) to the same value.
LLS requires specific container images and Helm charts to be available in your registry.
The following artifacts must be mirrored for LLS deployment:
Container Images:
streaming-proxy - Streaming Proxy Container(RP Container)Helm Charts:
gdn-streaming - GDN Streaming Proxy Helm chartOptional (for streaming workloads):
usd-composer)For detailed instructions on pulling these artifacts from NGC and pushing to your registry, see self-hosted-image-mirroring. The LLS-specific artifacts section lists exactly what you need.
When mirroring to ECR, the repository path must match your Helm values configuration. Ensure your registryName in Step 4 includes the same repository path you used when mirroring the chart.
LLS requires a Network Load Balancer (NLB) for streaming traffic.
Create a security group with the following rules:
Inbound:
Outbound:
You must create the NLB in the same region and availability zone as your GPU node pool.
The following commands create all required NLB resources. Copy and paste each section, replacing the placeholder values at the start.
2.1 Set Environment Variables
2.2 Get VPC and Public Subnet
2.3 Create Security Group
2.4 Configure Security Group Rules
2.5 Create Network Load Balancer
Choose one of the following options:
Option A: Dynamic IP (simpler)
Option B: Static EIP (predictable IP)
Use this if you need a predictable ingress IP for firewall rules:
2.6 Wait for NLB to Become Active
2.7 Create Target Group
2.8 Create Listener
2.9 Verify and Save Outputs
Save the Target Group ARN — you will need it for the adding streaming proxy container nodes as targets to the target group lls-step3-helm-values.
Create a custom-values.yaml file with your configuration.
Use the load balancer DNS name and availability zone from Step 2:
<placeholder> values with your actual configurationenv.lbPort is the public UDP port for streaming trafficenv.lbDns is the public load balancer dns nameavailabilityZones.zones is the availability zone for streaming proxy container which must be the same as the availability zone of the Load BalancernodeSelector.node-type is the node type where streaming proxy container will be deployedimage.repository is the streaming proxy container image repositoryimage.tag is the streaming proxy container image tagresources.requests.cpu is the streaming proxy pod cpu requestresources.requests.memory is the streaming proxy pod memory requestresources.limits.cpu is the streaming proxy pod cpu limitresources.limits.memory is the streaming proxy pod memory limit4.1 Deploy the LLS operator using Helm:
4.2 Add RP Container Node IP and Port to Load Balancer’s Target Group:
4.3 Add security rule to cluster node to accept streaming traffic on nodeport:
Verify the LLS components are running:
Check Streaming proxy pods (should show 1/1 Running):
Expected output:
Check container logs (filter out verbose stack traces):
The following table summarizes all AWS and Kubernetes resources created for LLS, who creates them, and how they are cleaned up:
The NLB and its associated resources created in lls-step2-nlb must be manually deleted.
Save your NLB cleanup script as lls-nlb-cleanup.sh, then run:
The script performs the following steps in dependency order:
Safety: The script identifies resources using the tags that were set during lls-step2-nlb. It will only delete resources with both lls:cluster=<your-cluster-name> and lls:managed-by=customer tags, ensuring it won’t accidentally delete other NLBs in your account.
If the NLB security group deletion still fails with DependencyViolation after the script removes the node SG rule, the NLB may not have fully deleted yet. The script will automatically retry. If it still fails after several attempts, wait a few more minutes and run the script again.
Verify all LLS resources have been removed:
NLB Security Group:
Streaming Proxy Security Group:
The streaming proxy pod has liveness and readiness probes configured to the /v1/health endpoint at port 8000.
The health check is used to determine if the streaming proxy pod is healthy. The health check is configured to check the /v1/health endpoint at port 8000 of the streaming proxy pod.