> This page is for NVIDIA Active Speaker Detection on Holoscan for Media, version 1.0.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/maxine/active-speaker-detection-h4m/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/maxine/llms.txt.

# End-to-End Demo Chart

## Overview

The `nvidia-active-speaker-detection-h4m-sample` Helm chart deploys a complete Active Speaker Detection demo pipeline on Holoscan for Media. This includes:

- Sender pipeline.
- Active Speaker Detection NIM service (`nvidia-active-speaker-detection-h4m-service`).
- Receiver pipeline (SMPTE ST 2110 or NMOS).
- SRT output for preview.

The NIM analyzes live video and audio streams to identify the active speaker in a multi-person scene. Detection results are transmitted as bounding box metadata via SMPTE ST 2110-40 ancillary data.

## Installation

### Prerequisites

Complete all prerequisite steps (Rivermax license, image pull and model pull secrets, high-speed network attachment) before running `helm install`. For details, refer to [Getting Started](/nim/maxine/active-speaker-detection-h4m/getting-started).

### Pull the Chart

If not already done, add the Helm repository and pull the chart:

```bash
helm pull nim-repo/nvidia-active-speaker-detection-h4m-sample --version 1.0.0
```


For the full repository setup, refer to [Pull Helm Charts](/nim/maxine/active-speaker-detection-h4m/installation/index#pull-helm-charts).

### Helm Installation

The chart includes a default `values.yaml`. For SMPTE ST 2110 *static* or NMOS pipelines, copy the configuration from [Starter Configuration Files](#starter-configuration-files) into `values-st2110.yaml` or `values-nmos.yaml`, adjust fields, and run the matching install command.

<Warning>
The default `values.yaml` ships with example values for cluster-specific fields such as node selector (`example-gpu-node`), image pull secret, network name, and scheduler. You *must* update these values to match your cluster before deploying. The easiest approach is to use the `global` overrides so that each value is specified once for all components (sender, receiver, and NIM service). For details, refer to [Global Overrides](/nim/maxine/active-speaker-detection-h4m/configuration/common#global-overrides).
</Warning>

Minimal deployment using `global` overrides (NMOS mode, 1080p):

```bash
helm upgrade --install nvidia-asd-h4m-sample \
  nvidia-active-speaker-detection-h4m-sample-1.0.0.tgz \
  --set global.nodeSelector.hostname=<gpu-node-name> \
  --set global.image.secret=<image-pull-secret> \
  --set global.network.name=<multus-net-attach-def> \
  --set global.schedulerName=<scheduler-name> \
  --set nvidia-active-speaker-detection-h4m-service.ngcModelDownload.secretName=<model-pull-secret>
```


Alternatively, set values per component when the sender, receiver, and NIM service run on different nodes or use different pull secrets:

```bash
helm upgrade --install nvidia-asd-h4m-sample \
  nvidia-active-speaker-detection-h4m-sample-1.0.0.tgz \
  --set sender.nodeSelector.hostname=<sender-node> \
  --set receiver.nodeSelector.hostname=<receiver-node> \
  --set nvidia-active-speaker-detection-h4m-service.nodeSelector.hostname=<gpu-node> \
  --set sender.image.secret=<image-pull-secret> \
  --set receiver.image.secret=<image-pull-secret> \
  --set nvidia-active-speaker-detection-h4m-service.image.secret=<image-pull-secret> \
  --set nvidia-active-speaker-detection-h4m-service.ngcModelDownload.secretName=<model-pull-secret>
```


For all available Helm values, refer to the [Configuration Reference](/nim/maxine/active-speaker-detection-h4m/configuration).

#### Recommended: Two-Phase Deploy

First, deploy the receiver and NIM service. When the pipeline is ready, enable the sender. This procedure prevents frames from accumulating in receiver queues before the NIM is ready.

```bash
# Phase 1: receiver + NIM service only
helm upgrade --install nvidia-asd-h4m-sample \
  nvidia-active-speaker-detection-h4m-sample-1.0.0.tgz \
  --set sender.enabled=false

kubectl rollout status deployment/<receiver-appName> --timeout=180s
kubectl rollout status deployment/<nim-appName> --timeout=180s

# Phase 2: enable sender
helm upgrade --install nvidia-asd-h4m-sample \
  nvidia-active-speaker-detection-h4m-sample-1.0.0.tgz

kubectl rollout status deployment/<sender-appName> --timeout=180s
```


On Red Hat OpenShift, replace `kubectl` with `oc`.

<Note>
A rollout status timeout is not a failure; it means the pod did not become ready within the allotted time. The NIM pod might still be pulling its image or initializing. If a timeout occurs, check the pod state before taking action:

```bash
kubectl get pods -o wide
kubectl describe pod <pod-name>
```

Look for `Pulling image` in the events (normal—wait longer) or `CrashLoopBackOff` / `ErrImagePull` (actionable: check secrets and node resources).
</Note>

After installation, verify all pods are running:

```bash
kubectl get pods
```

Confirm that the sender, NIM service, and receiver pods all show `READY 1/1` before continuing.

---

## Sender Sample Media

### Input Media Files

The sender expects an MPEG Transport Stream (`.ts`) file containing video and audio streams for active speaker detection.

### Stream Composition

Each input `.ts` file must contain the following streams:

- **Video stream:** Source video with one or more visible speakers.
- **Audio streams:** One mono audio track per speaker (for example, two speakers require two audio tracks). Each audio stream should contain only a single speaker's audio.

### Codecs

- **Video codec:** H.264
- **Audio codec:** Opus, 48 kHz, mono

### Bundled Sample Files

The sender container includes a sample `.ts` file:

`/workspace/assets/sample_asset.ts`

| File | Resolution | Frame Rate | Audio Tracks |
|------|------------|------------|--------------|
| `sample_asset.ts` | 1080p (1920×1080) | 30 fps | 2 |

Use the same resolution and frame rate for the sender, NIM service (`nvidia-active-speaker-detection-h4m-service`), and receiver.

<Note>
Each input audio stream must contain only a single speaker's audio. During silence, audio samples must be zero. If background noise is present, enable audio thresholding via `useAudioThresholdToDetectActiveAudioStream` and `audioThresholdDb` in the pipeline configuration. For details, refer to [Configuration Reference](/nim/maxine/active-speaker-detection-h4m/configuration/pipeline#active-speaker-detection-pipeline-tuning).
</Note>

---

## Receiver Type

The receiver supports two operating modes, controlled by `receiver.receiverType`:

| Mode | Description |
|------|-------------|
| `ancillary` | Receives bounding box metadata via SMPTE ST 2110-40 ancillary data. Use to consume detection results programmatically. |
| `video` | Receives the output video with bounding box overlays re-streamed via SRT. Requires `testFrameOverlayMode: true` on the NIM service. |

Example of switching the receiver to video mode:

```yaml
receiver:
  receiverType: "video"

nvidia-active-speaker-detection-h4m-service:
  pipeline:
    testFrameOverlayMode: true
```

<Note>
In `ancillary` mode, the `ancillaryInputParams` section is used. In `video` mode, the `videoInputParams` section is used instead.
</Note>

---

## Starter Configuration Files

Copy the appropriate configuration into `values-st2110.yaml` or `values-nmos.yaml`, adjust fields for your environment, and pass it to `helm upgrade --install` with `-f`.

For a full reference of all Helm keys and pipeline tuning parameters, refer to the [Configuration Reference](/nim/maxine/active-speaker-detection-h4m/configuration).

### SMPTE ST 2110 Configuration — Static (values-st2110.yaml)

```yaml
global:
  nodeSelector:
    hostname: ""
  image:
    secret: ""
  network:
    name: ""
  schedulerName: ""
  service:
    type: ""

sender:
  enabled: true
  appName: nvidia-active-speaker-detection-sender-st2110
  replicas: 1
  nodeSelector:
    hostname: example-gpu-node
  image:
    repository: nvcr.io/nim/nvidia/active-speaker-detection-h4m-sample-sender
    tag: "1.0.0"
    secret: ngc-api-key
  network:
    name: "media-a-tx-net"
  inputAssets:
    videoFile: /workspace/assets/sample_asset.ts
    numOfAudioTrack: 2
    videoFramerateNum: 30
    videoFramerateDen: 1
    videoWidth: 1920
    videoHeight: 1080
  st2110Ports:
    senderRemoteIp: 234.5.8.26
    video: 7005
    audio: 7006
  nmos:
    enabled: false
    hostname: asd-sender.local
    description: NVIDIA Active Speaker Detection Sender
    label: Nvidia-ASD-Sender
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32524

receiver:
  enabled: true
  appName: nvidia-active-speaker-detection-receiver-st2110
  replicas: 1
  nodeSelector:
    hostname: example-gpu-node
  image:
    repository: nvcr.io/nim/nvidia/active-speaker-detection-h4m-sample-receiver
    tag: "1.0.0"
    secret: ngc-api-key
  network:
    name: "media-a-tx-net"
  receiverType: "ancillary"
  nmos:
    enabled: false
    hostname: asd-receiver.local
    description: NVIDIA Active Speaker Detection Receiver
    label: Nvidia-ASD-Receiver
  videoInputParams:
    videoReceiverRemoteIp: "234.5.8.27"
    videoReceiverPort: 7005
    videoWidth: 1920
    videoHeight: 1080
    videoFramerateNum: 30
    videoFramerateDen: 1
  ancillaryInputParams:
    hostIp: "234.5.8.27"
    hostPort: 7006
    framerateNum: 30
    framerateDen: 1
  srtPort:
    internal: 8888
    external: 30889
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32525

nvidia-active-speaker-detection-h4m-service:
  enabled: true
  appName: nvidia-active-speaker-detection-nim-st2110
  replicas: 1
  nodeSelector:
    hostname: example-gpu-node
  image:
    repository: nvcr.io/nim/nvidia/active-speaker-detection-h4m-nim
    tag: "1.0.0"
    secret: ngc-api-key
  nimLogs:
    enabled: true
    mountPath: /workspace/nim-logs
  network:
    name: "media-a-tx-net"
  nmos:
    enabled: false
    hostname: active-speaker-detection-nim.local
    description: Nvidia Active Speaker Detection NIM
    label: Nvidia-Active-Speaker-Detection-NIM
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
  audio:
    pcmFormat: S24BE
    samplingRate: 48000
    numChannels: 1
  input:
    video:
      sessionName: video_in
      localInterfaceName: net1
      hostIp: "234.5.8.26"
      hostPort: "7005"
      hostNumSubnetBits: 24
      flushFrameCountThreshold: 3
    audio:
      sessionName: audio_in
      localInterfaceName: net1
      hostIp: 234.5.8.26
      hostPort: "7006"
      hostNumSubnetBits: 24
      numStreams: 2
  output:
    ancillaryData:
      sessionName: ancillary_out
      localInterfaceName: net1
      hostIp: 234.5.8.27
      hostPort: "7006"
      hostNumSubnetBits: 24
  logging:
    level: 3
  pipeline:
    testFrameOverlayMode: false
    outputFrameBufferSize: 30
    useAudioThresholdToDetectActiveAudioStream: false
    audioThresholdDb: -40.0
    syncTolerance: 0.5986
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32526
```


### NMOS Configuration (values-nmos.yaml)

```yaml
global:
  nodeSelector:
    hostname: ""
  image:
    secret: ""
  network:
    name: ""
  schedulerName: ""
  service:
    type: ""

sender:
  enabled: true
  appName: nvidia-active-speaker-detection-sender-nmos
  replicas: 1
  nodeSelector:
    hostname: example-gpu-node
  image:
    repository: nvcr.io/nim/nvidia/active-speaker-detection-h4m-sample-sender
    tag: "1.0.0"
    secret: ngc-api-key
  network:
    name: "media-a-tx-net"
  inputAssets:
    videoFile: /workspace/assets/sample_asset.ts
    numOfAudioTrack: 2
    videoFramerateNum: 30
    videoFramerateDen: 1
    videoWidth: 1920
    videoHeight: 1080
  nmos:
    enabled: true
    hostname: asd-sender.local
    description: Nvidia Active Speaker Detection Sender
    label: Nvidia-ASD-Sender
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32524

receiver:
  enabled: true
  appName: nvidia-active-speaker-detection-receiver-nmos
  replicas: 1
  nodeSelector:
    hostname: example-gpu-node
  image:
    repository: nvcr.io/nim/nvidia/active-speaker-detection-h4m-sample-receiver
    tag: "1.0.0"
    secret: ngc-api-key
  receiverType: "ancillary" # or "video"
  videoInputParams:
    videoFramerateNum: 30
    videoFramerateDen: 1
    videoWidth: 1920
    videoHeight: 1080
  nmos:
    enabled: true
    hostname: asd-receiver.local
    description: Nvidia Active Speaker Detection Receiver
    label: Nvidia-ASD-Receiver
  ancillaryInputParams:
    hostIp: "234.5.8.27"
    hostPort: 7006
    framerateNum: 30
    framerateDen: 1
  videoInputParams:
    videoReceiverRemoteIp: "234.5.8.26"
    videoReceiverPort: 7005
    videoWidth: 1920
    videoHeight: 1080
    videoFramerateNum: 30
    videoFramerateDen: 1
  srtPort:
    internal: 8888
    external: 30889
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32525

nvidia-active-speaker-detection-h4m-service:
  enabled: true
  appName: nvidia-active-speaker-detection-feature-nmos
  replicas: 1
  nodeSelector:
    hostname: example-gpu-node
  image:
    repository: nvcr.io/nim/nvidia/active-speaker-detection-h4m-nim
    tag: "1.0.0"
    secret: ngc-api-key
  nimLogs:
    enabled: true
    mountPath: /workspace/nim-logs
  network:
    name: "media-a-tx-net"
  nmos:
    enabled: true
    hostname: active-speaker-detection-nim.local
    description: Nvidia Active Speaker Detection NIM
    label: Nvidia-Active-Speaker-Detection-NIM
  video:
    width: 1920
    height: 1080
    framerateNum: 30
    framerateDen: 1
  audio:
    pcmFormat: S24BE
    samplingRate: 48000
    numChannels: 1
  input:
    video:
      sessionName: video_in
      flushFrameCountThreshold: 3
    audio:
      sessionName: audio_in
      numStreams: 2
  output:
    video:
      sessionName: video_out
    ancillaryData:
      sessionName: ancillary_out
      localInterfaceName: net1
      hostIp: 234.5.8.27
      hostPort: "7006"
      hostNumSubnetBits: 24
  logging:
    level: 3
  pipeline:
    testFrameOverlayMode: false
    outputFrameBufferSize: 30
    useAudioThresholdToDetectActiveAudioStream: false
    audioThresholdDb: -40.0
    syncTolerance: 0.5986
  service:
    enabled: true
    type: NodePort
    port: 9010
    nodePort: 32526
```


On Holoscan for Media clusters, the NMOS Connection Manager web UI is typically reached from a browser on the cluster network. For remote graphical access, refer to [Chrome Remote Desktop](/nim/maxine/active-speaker-detection-h4m/getting-started#chrome-remote-desktop) in [Getting Started](/nim/maxine/active-speaker-detection-h4m/getting-started).

<Note>
In NMOS mode, connect receivers to the NIM service *before* connecting the sender via the NMOS Connection Manager UI. In SMPTE ST 2110 mode (static), ensure that the "sender → NIM service → receiver IP address and port" chain is consistent across all values.
</Note>

### Install

SMPTE ST 2110 (static):

```bash
helm upgrade --install nvidia-asd-h4m-sample \
  nvidia-active-speaker-detection-h4m-sample-1.0.0.tgz \
  -f values-st2110.yaml
```


<Note>
NMOS is enabled by default. To deploy in fixed SMPTE ST 2110 mode (static), append the following:

  ```bash
  --set sender.nmos.enabled=false \
  --set receiver.nmos.enabled=false \
  --set nvidia-active-speaker-detection-h4m-service.nmos.enabled=false
  ```
</Note>

NMOS:

```bash
helm upgrade --install nvidia-asd-h4m-sample \
  nvidia-active-speaker-detection-h4m-sample-1.0.0.tgz \
  -f values-nmos.yaml
```


---

## View Receiver Output

The receiver logs the parsed ancillary payload as JSON. To view it, run the following command:  

```bash
oc logs deploy/<receiver-deployment>
```

A typical log line looks like the following:

```json
{
  "detections": [
    {
      "audio_id": 1,
      "bbox": {"h": 259.10, "w": 193.48, "x": 1364.90, "y": 381.73},
      "det_idx": 0,
      "is_speaking": true,
      "score": 0.9589,
      "track_id": 0
    },
    {
      "audio_id": 0,
      "bbox": {"h": 170.55, "w": 135.42, "x": 391.97, "y": 350.03},
      "det_idx": 1,
      "is_speaking": false,
      "score": 0.9071,
      "track_id": 1
    }
  ],
  "frame_idx": 560,
  "rtp_timestamp": 75101117997,
  "timestamp": "07:25:35.585747"
}
```

| Field | Description |
|-------|-------------|
| `detections` | Array of detected faces in the frame. |
| `audio_id` | Index of the diarized audio stream associated with this face (`-1` if unassigned). |
| `bbox` | Bounding box in pixels: `x`, `y` (top-left corner), `w` (width), `h` (height). |
| `det_idx` | Detection index within the frame. |
| `is_speaking` | `true` if this face is the active speaker. |
| `score` | Detection confidence score (0–1). |
| `track_id` | Persistent face track identifier across frames. |
| `frame_idx` | Zero-based frame counter. |
| `rtp_timestamp` | RTP timestamp of the corresponding input video frame. |
| `timestamp` | Wall-clock time of the frame (`HH:MM:SS.ffffff`). |

---

## View Output via SRT

Get the internal IP of the node:

```bash
kubectl get nodes -o wide
```

The SRT output is on port `30889` by default. To view the stream in VLC, refer to [Open an SRT Stream in VLC](/nim/maxine/active-speaker-detection-h4m/end-to-end-verification#open-an-srt-stream-in-vlc).

---

## Disabling Components

Individual pipeline components can be disabled during `helm install` or `helm upgrade` without removing the release:

```bash
--set sender.enabled=false
--set receiver.enabled=false
--set nvidia-active-speaker-detection-h4m-service.enabled=false
```

Only the selected component stops; the rest of the pipeline continues to run.

---

## Verify

```bash
helm status nvidia-asd-h4m-sample
kubectl get pods -o wide
```

Verify configured Helm values:

```bash
helm get values nvidia-asd-h4m-sample
```

On Red Hat OpenShift, replace `kubectl` with `oc`. For log access, refer to [Observability](/nim/maxine/active-speaker-detection-h4m/observability#logs).

---

## Uninstall

```bash
helm uninstall nvidia-asd-h4m-sample
```

---

## End-to-End Verification

1. Create `values-st2110.yaml` or `values-nmos.yaml` from [Starter Configuration Files](#starter-configuration-files) and run the matching [Install](#install) command.
2. Confirm all pods show `READY 1/1` with `kubectl get pods`.
3. Open the SRT preview; refer to [View Output via SRT](#view-output-via-srt). For VLC steps, NMOS wiring, and screenshots, refer to [Verification](/nim/maxine/active-speaker-detection-h4m/end-to-end-verification).
4. Run `helm uninstall nvidia-asd-h4m-sample`.

For troubleshooting, refer to [Advanced Usage](/nim/maxine/active-speaker-detection-h4m/advanced-usage#troubleshooting).