Advanced Usage
Persistent Storage
The NIM service chart supports two optional persistent volume claims for storage that survives pod restarts and helm uninstall. Both are disabled by default and are annotated with helm.sh/resource-policy: keep so they are retained when the Helm release is removed.
For the operator, equivalent settings are available under spec.parameters.nimModelCache and spec.parameters.serverLogs on the NvidiaLipsyncMediaFunction custom resource. For details, refer to Operator Configuration.
Model Cache
Caches NGC model artifacts locally to avoid re-downloading on every deployment. The persistent volume claim name is <appName>-model-cache.
The chart mounts the model cache at /opt/nim/.cache. The LipSync operator sets NIM_CACHE_PATH to that path on the NIM pod when model cache is enabled on the custom resource.
Server Log Files
Persists time-stamped NIM server log files under /var/log/lipsync. The persistent volume claim name is <appName>-server-logs.
When serverLogs is enabled, the chart sets AI4M_NIM_LOG_PATH to /var/log/lipsync.
StorageClass
By default, storageClassName is set to "", which uses the cluster’s default StorageClass. To use a specific StorageClass, set storageClassName to the name of an existing StorageClass in your cluster under the nimModelCache or serverLogs block.
If no default StorageClass is configured in your cluster and storageClassName is left empty, the PVC remains in Pending state. In that case, set storageClassName to a valid StorageClass from your cluster.
List StorageClasses in your cluster:
If a chart-managed PVC stays Pending, uncomment storageClassName in your values file (refer to the Model Cache and Server Log Files examples) and set it to an RWO-capable class from that list. For example:
The model cache persistent volume claim requires a StorageClass that supports ReadWriteOnce access mode. When using a shared filesystem, ensure that only one pod writes to the cache concurrently.
LipSync NIM Integration with Active Speaker Detection for Multi-Speaker Use Cases
Multi-speaker lip sync requires integration with the Active Speaker Detection (ASD) NIM. The ASD NIM identifies active speakers in the video stream and provides per-frame bounding boxes, speaker IDs, and confidence scores. The LipSync NIM uses this ancillary data to apply lip sync to the active speaker.
NMOS integration currently has known issues, and multi-speaker integration is supported only in ST 2110 static mode. Refer to NMOS Multi-Speaker Limitation.
Enabling Ancillary Input for ASD Bounding Boxes
To receive bounding box ancillary data from the ASD NIM, enable ancillary input in the LipSync NIM configuration as shown. (The hostIp and multicast settings must match your deployment configuration.)
Debug Visualization
The output.boundingBoxEnabled flag is optional and intended for debugging and visualization only:
When enabled:
- A green bounding box is drawn around the currently active speaker.
- A blue bounding box is drawn for a detected speaker who is inactive or not speaking.
This visualization helps verify that ASD metadata is being received and interpreted correctly by the LipSync NIM.
ASD NIM Configuration
Ensure that the ASD NIM ancillary output IP and port match the LipSync NIM ancillary input configuration. For example, the default LipSync Helm configuration uses the following values:
- IP address:
234.5.8.9 - Port:
8001
For the binary layout of the ancillary application payload, refer to Ancillary Data Payload (SMPTE ST 2110-40).
Ancillary Data Payload (SMPTE ST 2110-40)
When ancillary input is enabled, the LipSync NIM ingests per-frame active speaker tracking bounding boxes carried over the SMPTE ST 2110-40 ancillary data channel.
The application payload follows a fixed binary layout with size of 312 bytes, supporting up to 12 bounding boxes per frame.
Binary Layout
Using a Custom Input Asset
To provide your own .ts transport stream without baking it into the sender image, use a PersistentVolumeClaim, copy assets into it, and then enable the optional sender.assetsPVC settings in the nvidia-lipsync-h4m-sample Helm chart and set sender.video.inputFile to a path under that mount (for example, /mnt/sender-assets/...).
On Red Hat OpenShift, use oc instead of kubectl in the following steps.
Step 1: Create a PVC
This example uses your cluster’s default StorageClass. If the PVC stays Pending, set storageClassName to a storage class recommended for your cluster (for example, <storage-class>). You can list available classes with the following command:
Step 2: Upload the Asset to the PVC
Launch a temporary pod that mounts the PVC, copy files, and then delete the pod:
Step 3: Point Helm at the Mounted Path
Enable the claim on the sender and set inputFile under mountPath. (Defaults are in values*.yaml for the sample chart.)
Upgrade or install the release; then confirm that the sender process sees the file:
Use the sender deployment name from sender.appName in your values file (for example nvidia-lipsync-sender-nmos or nvidia-lipsync-sender-st2110).
You should see a line such as INPUT_FILE=/mnt/sender-assets/... matching your sender.video.inputFile. (The sender script logs this at startup.)
Troubleshooting
End-to-End Demo Chart and NIM Service Chart
Kubernetes Operator
On Red Hat OpenShift, replace kubectl with oc.
See Also
- Configuration Reference: Full set of Helm values and operator custom resource fields.
- Observability: Operator controller metrics.
- Limitations and Known Behaviors