Limitations and Known Behaviors#
Session Restart Requirement#
In ST 2110 static and NMOS operating modes, any interruption in the input media stream—such as sender restarts, sender pod replacements, NMOS link/unlink operations, or any loss of continuity in one or more sender streams (for example, gaps in the input or a stream stopping transmission)—is treated as the end of the current session.
To restore correct operation in both modes, the entire processing pipeline (sender → LipSync NIM → receiver) must be redeployed. Restarting individual components is not sufficient.
NMOS Connection Order#
Establish the downstream connections first: Connect the sender audio and LipSync NIM output to the receiver. After these receiver connections are configured, connect the sender audio and video streams to the LipSync NIM input.
Following this sequence ensures proper stream alignment and helps prevent muxing inconsistencies.
For NMOS connections in the Connection Manager web UI, refer to NMOS Workflow.
NMOS Multi-Speaker Limitation#
Multiple NIM integrations for multi-speaker use cases have known issues with NMOS in the current release. Currently, only ST 2110 static mode is supported.
Latency#
The LipSync model uses a 30-frame look-ahead buffer, resulting in a fixed, end-to-end latency of approximately 1 second at 30 fps with the default settings.
SRT Stream#
Before launching the pods in ST 2110 static mode and connecting nodes via NMOS, ensure that the SRT stream is already started from the receiver side (such as when using a client like VLC).
If the pipeline is started before the SRT stream is active, it might result in only audio being received while the video is not rendered. In such cases, keep the SRT stream running and restart the pipeline to restore normal operation.
Model Cache Persistence#
The model cache persistent volume claim requires a storage class that supports ReadWriteOnce access mode. When using a shared filesystem, ensure only one pod writes to the cache concurrently to avoid corruption.
Operator Chart#
The Operator chart deploys only the controller. Sender and receiver sample workloads are provided by the nvidia-lipsync-h4m-sample umbrella chart, not by the operator alone.
Deleting the NvidiaLipsyncMediaFunction custom resource definition while instances exist, or without uninstalling operands first, can strand resources. Follow the uninstall order—custom resources first, and then the Helm release—as described in Uninstall.
The media-function image is configured when the operator is installed, so CR reconciled by that operator use the same validated controller and media-function image combination. This ensures the operator and workload are always supported together and avoids compatibility issues. Refer to Operator Configuration.