Is this page helpful?

End-to-End Demo Chart#

Overview#

The nvidia-lipsync-h4m-sample Helm chart deploys a complete LipSync demo pipeline on Holoscan for Media. This includes the following components:

  • Sender pipeline.

  • LipSync NIM service (nvidia-lipsync-h4m-service).

  • Receiver pipeline (ST 2110 or NMOS).

  • SRT output for preview.

This chart provides a ready-to-run setup for evaluating and demonstrating LipSync capabilities within a Holoscan for Media environment.

Installation#

Prerequisites#

Complete all prerequisite steps (Rivermax license, image pull and model pull secrets, GPU node, Multus attachment for SMPTE ST 2110) before running helm install. For details, refer to Getting Started.

If not already done, add the Helm repository and pull the chart:

helm pull nim-repo/nvidia-lipsync-h4m-sample --version 1.0.0

Helm Installation#

The chart includes a default values.yaml and can be deployed from the .tgz with -f. For the sample ST 2110 or NMOS pipelines, copy the YAML from Helm Configuration Parameters into values-st2110.yaml or values-nmos.yaml, adjust fields, and run the matching install command.

Default deployment (NMOS mode, 1080p):

Important

The default values.yaml ships with example values for cluster-specific fields such as node selector (example-gpu-node), image pull secret, network name, and scheduler. You must update these values to match your cluster before deploying. The easiest approach is to use the global overrides so that each value is specified once for all components (sender, receiver, and NIM service). For details, refer to Global Overrides.

With global overrides:

helm upgrade --install lipsync-h4m-sample \
  nvidia-lipsync-h4m-sample-1.0.0.tgz \
  --set global.nodeSelector.hostname=<gpu-node-name> \
  --set global.image.secret=<image-pull-secret> \
  --set global.network.name=<multus-net-attach-def> \
  --set global.schedulerName=<scheduler-name> \
  --set global.service.type=NodePort \
  --set nvidia-lipsync-h4m-service.ngcApiKeySecret.name=<model-pull-secret>

Alternatively, set values per component when the sender, receiver, and NIM service run on different nodes or use different pull secrets:

helm upgrade --install lipsync-h4m-sample \
  nvidia-lipsync-h4m-sample-1.0.0.tgz \
  --set sender.nodeSelector.hostname=<gpu-node-name> \
  --set receiver.nodeSelector.hostname=<gpu-node-name> \
  --set nvidia-lipsync-h4m-service.nodeSelector.hostname=<gpu-node-name> \
  --set sender.image.secret=<image-pull-secret> \
  --set receiver.image.secret=<image-pull-secret> \
  --set nvidia-lipsync-h4m-service.image.secret=<image-pull-secret> \
  --set nvidia-lipsync-h4m-service.ngcApiKeySecret.name=<model-pull-secret>

For all available Helm values, refer to the Configuration Reference.

After installation, check the status of the pods:

kubectl get pods

Example output:

NAME                                                    READY   STATUS
nvidia-lipsync-nim-nmos-556d95b4f6-4m2cj                1/1     Running
nvidia-lipsync-receiver-nmos-f6ddc9f59-v5w5l            1/1     Running
nvidia-lipsync-sender-nmos-6586c98f9d-vn9tg             1/1     Running

Confirm that the sender, LipSync NIM, and receiver pods all show 1/1 in the READY column before continuing.

Check the pod state:

kubectl get pods -o wide
kubectl describe pod <pod-name>

Look for Pulling image in the events (normal—wait longer) or CrashLoopBackOff / ErrImagePull (actionable: check secrets and node resources).

For NMOS, refer to Chrome Remote Desktop for UI access and setup steps.

For NMOS connection order and NMOS known issues, refer to Limitations and Known Behaviors.

High-Speed Network Configuration#

For ST 2110, confirm the high-speed network attachment name in the cluster, and then set the same network.name on the sender, receiver, and nvidia-lipsync-h4m-service so it matches that attachment (for example, the name from a NetworkAttachmentDefinition or what the platform documents).

Inline overrides (append to helm install):

  --set sender.network.name=media-a-tx-net \
  --set receiver.network.name=media-a-tx-net \
  --set nvidia-lipsync-h4m-service.network.name=media-a-tx-net

Or in values YAML:

sender:
  network:
    name: "media-a-tx-net"
receiver:
  network:
    name: "media-a-tx-net"
nvidia-lipsync-h4m-service:
  network:
    name: "media-a-tx-net"

Replace media-a-tx-net with the attachment name used in the cluster.

Persistent Storage#

To persist model cache and service logs across pod restarts or re-deployments, enable nimModelCache, serverLogs, or both in the values file. Both Persistent Volume Claims (PVCs) are optional and disabled by default.

  • nimModelCache stores downloaded models locally, preventing re-downloads on subsequent runs.

  • serverLogs persists service logs for debugging and analysis.

nvidia-lipsync-h4m-service:
  nimModelCache:
    enabled: true
  serverLogs:
    enabled: true

Sender Sample Media#

Input Media Files#

The sender expects an MPEG Transport Stream (.ts) file containing specific media streams required for lip synchronization.

Stream Composition#

Each input .ts file must contain the following streams:

  • Video stream: Source video to be processed by the LipSync NIM.

  • Original audio stream: The original audio associated with the video (used as reference audio).

  • Translated audio stream: The target audio (such as a different language) to be synchronized with the video.

Codecs#

  • Video codec: H.264

  • Audio codec: Opus, 48 kHz, mono (both the original and translated audio streams)

Bundled Sample Files#

The sender container includes sample .ts files:

/workspace/assets/

Pattern

Resolution

Frame Rate

sample1_*.ts

1080p

30

sample2_*.ts

4K

30000/1001

Language Variants#

  • _de: German

  • _es: Spanish

  • _fr: French

Use the same resolution and frame rate (FPS) for the sender, LipSync NIM service (nvidia-lipsync-h4m-service), and receiver.

Ensure that the resolution and frame rate specified in Helm (sender.video and related settings) match the selected input file.

sender:
  video:
    inputFile: /workspace/assets/sample1_de.ts
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1

Helm Configuration Parameters#

Copy the appropriate configuration into values-st2110.yaml or values-nmos.yaml, adjust fields for your environment, and pass it to helm upgrade --install with -f.

For a full reference of all Helm keys and pipeline tuning parameters, refer to the Configuration Reference.

ST 2110 (Static Mode) Configuration (values-st2110.yaml)#

sender:
  enabled: true
  appName: nvidia-lipsync-sender-st2110
  scriptName: sender-st2110-2languages
  nodeSelector:
    hostname: example-gpu-node
  network:
    name: media-a-tx-net
  schedulerName: topo-aware-scheduler
  image:
    repository: nvcr.io/nim/nvidia/lipsync-h4m-sample-sender
    tag: "1.0.0"
    secret: ngc-secret-key
  # Mount PVC sender-assets (upload .ts via asset-loader)
  assetsPVC:
    enabled: false
    claimName: sender-assets
    mountPath: "/mnt/sender-assets"
    readOnly: true
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
    # Sample TS files (uncomment one to use):
    inputFile: /workspace/assets/sample1_de.ts
    # inputFile: /workspace/assets/sample1_es.ts
    # inputFile: /workspace/assets/sample1_fr.ts
    # inputFile: /workspace/assets/sample2_de.ts
    # inputFile: /workspace/assets/sample2_es.ts
    # inputFile: /workspace/assets/sample2_fr.ts
    # inputFile: /mnt/sender-assets/custom_sample.ts
  hostIp: "234.5.8.9"
  st2110Ports:
    video: 5001
    audio1: 6001
    audio2: 7001
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32514

receiver:
  enabled: true
  appName: nvidia-lipsync-receiver-st2110
  scriptName: receiver-to-srt-st2110
  nodeSelector:
    hostname: example-gpu-node
  network:
    name: media-a-tx-net
  schedulerName: topo-aware-scheduler
  image:
    repository: nvcr.io/nim/nvidia/lipsync-h4m-sample-receiver
    tag: "1.0.0"
    secret: ngc-secret-key
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
    audioSampleRate: 48000
  hostIp: "234.5.8.9"
  st2110Port:
    video: 5002
    audio: 7001
  srtPort:
    internal: 8999
    external: 32511
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32515

nvidia-lipsync-h4m-service:
  enabled: true
  appName: nvidia-lipsync-nim-st2110
  containerSecurityContext:
    runAsUser: 1000
    runAsGroup: 1000
    runAsNonRoot: true
    allowPrivilegeEscalation: true
  nodeSelector:
    hostname: example-gpu-node
  network:
    name: media-a-tx-net
  schedulerName: topo-aware-scheduler
  image:
    repository: nvcr.io/nim/nvidia/lipsync-h4m-nim
    tag: "1.0.0"
    secret: ngc-secret-key
  ngcApiKeySecret:
    name: model-download-api-key
    key: NGC_API_KEY
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
  nmos:
    enabled: false
    hostname: lipsync-nim.local
    description: Nvidia LipSync NIM
    label: Nvidia-LipSync-NIM
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32513
  nim:
    enabled: 1
  logging:
    level: 3
  input:
    video:
      sessionName: video_in
      localInterfaceName: net1
      hostIp: "234.5.8.9"
      hostPort: "5001"
      hostNumSubnetBits: 24
    audio:
      sessionName: audio_in
      localInterfaceName: net1
      hostIp: "234.5.8.9"
      hostPort: "7001"
      hostNumSubnetBits: 24
    ancillaryData:
      enabled: false
      sessionName: ancillary_data_in
      localInterfaceName: net1
      hostIp: "234.5.8.9"
      hostPort: "8001"
      hostNumSubnetBits: 24
  output:
    video:
      sessionName: video_out
      localInterfaceName: net1
      hostIp: "234.5.8.9"
      hostPort: "5002"
      hostNumSubnetBits: 24
    boundingBoxEnabled: false
  model:
    headMovementSpeed: 0
  nimModelCache:
    enabled: false
    size: 10Gi
    storageClassName: ""
  serverLogs:
    enabled: false
    size: 5Gi
    storageClassName: ""

NMOS Configuration (values-nmos.yaml)#

sender:
  enabled: true
  appName: nvidia-lipsync-sender-nmos
  scriptName: sender-nmos-2languages
  nodeSelector:
    hostname: example-gpu-node
  network:
    name: media-a-tx-net
  schedulerName: topo-aware-scheduler
  image:
    repository: nvcr.io/nim/nvidia/lipsync-h4m-sample-sender
    tag: "1.0.0"
    secret: ngc-secret-key
  # Mount PVC sender-assets (upload .ts via asset-loader)
  assetsPVC:
    enabled: false
    claimName: sender-assets
    mountPath: "/mnt/sender-assets"
    readOnly: true
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
    # Sample TS files (uncomment one to use):
    inputFile: /workspace/assets/sample1_de.ts
    # inputFile: /workspace/assets/sample1_es.ts
    # inputFile: /workspace/assets/sample1_fr.ts
    # inputFile: /workspace/assets/sample2_de.ts
    # inputFile: /workspace/assets/sample2_es.ts
    # inputFile: /workspace/assets/sample2_fr.ts
    # inputFile: /mnt/sender-assets/custom_sample.ts
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32514

receiver:
  enabled: true
  appName: nvidia-lipsync-receiver-nmos
  scriptName: receiver-to-srt-nmos
  nodeSelector:
    hostname: example-gpu-node
  network:
    name: media-a-tx-net
  schedulerName: topo-aware-scheduler
  image:
    repository: nvcr.io/nim/nvidia/lipsync-h4m-sample-receiver
    tag: "1.0.0"
    secret: ngc-secret-key
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
    audioSampleRate: 48000
  srtPort:
    internal: 8999
    external: 32512
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32515

nvidia-lipsync-h4m-service:
  enabled: true
  appName: nvidia-lipsync-nim-nmos
  containerSecurityContext:
    runAsUser: 1000
    runAsGroup: 1000
    runAsNonRoot: true
    allowPrivilegeEscalation: true
  nodeSelector:
    hostname: example-gpu-node
  network:
    name: media-a-tx-net
  schedulerName: topo-aware-scheduler
  image:
    repository: nvcr.io/nim/nvidia/lipsync-h4m-nim
    tag: "1.0.0"
    secret: ngc-secret-key
  ngcApiKeySecret:
    name: model-download-api-key
    key: NGC_API_KEY
  nmos:
    enabled: true
    hostname: lipsync-nim.local
    description: Nvidia LipSync NIM
    label: Nvidia-LipSync-NIM
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32513
  nim:
    enabled: 1
  logging:
    level: 3
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
  audio:
    pcmFormat: S24BE
    samplingRate: 48000
    numChannels: 1
  input:
    video:
      sessionName: video_in
    audio:
      sessionName: audio_in
    ancillaryData:
      enabled: false
      sessionName: ancillary_data_in
  output:
    video:
      sessionName: video_out
    boundingBoxEnabled: false
  model:
    headMovementSpeed: 0
  nimModelCache:
    enabled: false
    size: 10Gi
    storageClassName: ""
  serverLogs:
    enabled: false
    size: 5Gi
    storageClassName: ""

Install#

ST 2110:

helm install lipsync-h4m-sample nvidia-lipsync-h4m-sample-1.0.0.tgz \
  -f values-st2110.yaml

NMOS:

helm install lipsync-h4m-sample nvidia-lipsync-h4m-sample-1.0.0.tgz \
  -f values-nmos.yaml

Note: Connect the receiver first by establishing sender audio and LipSync NIM output to receiver. After the receiver connection is in place, connect the sender audio and video streams to the LipSync NIM. For more information, refer to Limitations and Known Behaviors.

View Output via SRT#

kubectl get nodes -o wide

Use the internal IP address from the output. With the sample values in this guide, ST 2110 output is on port 32511 and NMOS on 32512—for example, srt://<internal-ip>:32511 or srt://<internal-ip>:32512 in VLC or ffplay.

ffplay "srt://<internal-ip>:32511"   # ST 2110
ffplay "srt://<internal-ip>:32512"   # NMOS

Disabling Components#

You can disable individual pipeline components during helm install or helm upgrade without removing the release by setting their enabled flag to false.

--set sender.enabled=false
--set receiver.enabled=false
--set nvidia-lipsync-h4m-service.enabled=false

This stops only the selected component; the rest of the pipeline continues to run.

Uninstall#

helm uninstall lipsync-h4m-sample

Note: If serverLogs or nimModelCache were enabled, their PVCs are retained after uninstall (resource-policy: keep). Delete manually if no longer needed:

kubectl delete pvc <appName>-server-logs <appName>-model-cache

Verify and Check Logs#

helm status lipsync-h4m-sample
kubectl get pods -o wide

Replace the deploy names in kubectl logs with sender.appName, receiver.appName, and nvidia-lipsync-h4m-service.appName from the values file. (ST 2110 and NMOS use different names.)

kubectl logs deploy/nvidia-lipsync-sender-nmos
kubectl logs deploy/nvidia-lipsync-nim-nmos
kubectl logs deploy/nvidia-lipsync-receiver-nmos

On Red Hat OpenShift, replace kubectl with oc. For log access, refer to Observability.

End-to-End Setup and Validation#

  1. Create values-st2110.yaml or values-nmos.yaml from Starter Configuration Files and run the matching Install command.

  2. Confirm all pods show READY 1/1 with kubectl get pods.

  3. Open the SRT preview; refer to View Output via SRT. For VLC steps, NMOS wiring, and screenshots, refer to Verification.

  4. Run helm uninstall lipsync-h4m-sample.

# Install sample release (NMOS; for ST 2110, use -f values-st2110.yaml)
helm install lipsync-h4m-sample nvidia-lipsync-h4m-sample-1.0.0.tgz -f values-nmos.yaml
# Confirm all pods show READY 1/1
kubectl get pods
# Node IPs (use internal IP address in the SRT URL)
kubectl get nodes -o wide
# Example SRT: srt://<internal-ip>:32512 (NMOS) or srt://<internal-ip>:32511 (ST 2110)
# Remove the Helm release
helm uninstall lipsync-h4m-sample

For troubleshooting, refer to Advanced Usage.