> This page is for NVIDIA LipSync NIM, version 1.4.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.4.0 (Latest) (default): https://docs.nvidia.com/nim/maxine/lipsync/1.4.0/llms.txt
> - 1.3.0: https://docs.nvidia.com/nim/maxine/lipsync/1.3.0/llms.txt
> - 1.2.0: https://docs.nvidia.com/nim/maxine/lipsync/1.2.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/maxine/llms.txt.

# Getting Started

This page shows how to authenticate with NGC, launch the LipSync NIM container, and configure its runtime.

## Prerequisites

To ensure that you have the supported hardware and software stack, check the [Support Matrix](/nim/maxine/lipsync/support-matrix).

### NGC Authentication

An NGC API key is required to pull the container image and download models. Complete the following steps to create the key, export it, and log in to the NVIDIA Container Registry.

#### Generate an API Key

LipSync NIM is available only through the [AI for Media Private Access Program](https://developer.nvidia.com/topics/ai/generative-ai/ai-for-media/private-access). Joining the Private Access Program gives you an NGC API key with the permissions required for this NIM.

You can generate a key at [https://org.ngc.nvidia.com/setup/api-keys](https://org.ngc.nvidia.com/setup/api-keys) after joining the Private Access Program.

When creating an NGC API Personal key, ensure that at least **NGC Catalog** is selected from the **Services Included** dropdown. You can include more services if this key is to be reused for other purposes.

<Note>
Personal keys allow you to configure an expiration date, revoke or delete the key using an action button, and rotate the key as needed. For more information about key types, refer to [NGC API Keys](https://docs.nvidia.com/ngc/gpu-cloud/ngc-user-guide/index.html#ngc-api-keys) in the NGC User Guide.
</Note>

#### Export the NGC API Key

Pass the value of the API key to the `docker run` command in the next section as the `NGC_API_KEY` environment variable to download the appropriate models and resources when starting the NIM.

If you are not familiar with how to create the `NGC_API_KEY` environment variable, the simplest way is to export it in your terminal:

```bash
export NGC_API_KEY=<value>
```

Run one of the following commands to make the key available at startup:

```bash
# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc
```

<Note>
Other, more secure options include saving the value in a file, so that you can retrieve with `cat $NGC_API_KEY_FILE`, or using a [password manager](https://www.passwordstore.org/).
</Note>

#### Docker Login to NGC

To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:

```bash
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
```

Use `$oauthtoken` as the username and `NGC_API_KEY` as the password. The `$oauthtoken` username is a special name that indicates that you will authenticate with an API key and not a user name and password.

<Note>
Before pulling the container image, you must accept the Governing Terms on the [LipSync NIM](https://catalog.ngc.nvidia.com/orgs/nim/teams/nvidia/containers/lipsync) container page in NGC. This page is accessible only after enrollment in the [AI for Media Private Access Program](https://developer.nvidia.com/topics/ai/generative-ai/ai-for-media/private-access).
</Note>

## Launching the NIM Container

The following command launches the LipSync NIM container with the gRPC service. The NIM detects the GPU architecture of the host and downloads the matching model automatically. (For a list of parameters, refer to [Runtime Parameters for the Container](#runtime-parameters-for-the-container).)

```bash
docker run -it --rm --name=lipsync-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NV_AI4M_MAX_CONCURRENCY_PER_GPU=1 \
  -e NIM_HTTP_API_PORT=8000 \
  -e NIM_GRPC_API_PORT=8001 \
  -p 8000:8000 \
  -p 8001:8001 \
  -p 9002:9002 \
  nvcr.io/nim/nvidia/lipsync:latest
```

<Note>
Port `9002` publishes the Triton Inference Server metrics endpoint, which
[Observability](/nim/maxine/lipsync/observability) uses. It is not required for inference; omit it if you do
not intend to scrape metrics.
</Note>

Mount a host directory at `NIM_CACHE_PATH` (for example
`-v $HOME/.cache/nim:/opt/nim/.cache`) to persist model artifacts across runs and avoid
re-downloading them on every launch.

<Note>
The flag `--gpus all` is used to assign all available GPUs to the NIM container.
To assign specific GPUs to the NIM container (if your machine has multiple GPUs), use `--gpus '"device=0,1,2..."'`.
</Note>

On bare-metal Linux systems with PCIe topology and multiple GPUs, we recommend setting IOMMU to passthrough. Refer to [PCIe Multi-GPU Systems](/nim/maxine/lipsync/advanced-usage/multi-gpu).

If the NIM launch is successful, you get a response similar to the following.

```bash
I1027 22:31:44.952125 123 grpc_server.cc:2560] "Started GRPCInferenceService at 127.0.0.1:9001"
I1027 22:31:44.952247 123 http_server.cc:4755] "Started HTTPService at 127.0.0.1:9000"
I1027 22:31:44.993329 123 http_server.cc:358] "Started Metrics Service at 127.0.0.1:9002"
Triton server is ready
[INFO AI4M BASE LOGGER 2025-10-27 22:31:46.097 PID:207] Using threading mode for gRPC service
[INFO AI4M BASE LOGGER 2025-10-27 22:31:46.097 PID:207] Starting threading gRPC service with 1 threads
[INFO AI4M BASE LOGGER 2025-10-27 22:31:46.105 PID:207] Using Insecure Server Credentials
[INFO AI4M BASE LOGGER 2025-10-27 22:31:46.107 PID:207] Listening to 0.0.0.0:8001
```

<Note>
By default, the LipSync NIM gRPC service is hosted on port `8001`. You must use this port for inferencing requests. The port is configurable via the `NIM_GRPC_API_PORT` environment variable.
</Note>

### Selecting a Language-Specific Model

The LipSync NIM includes a generic, language-agnostic model (used by default) and fine-tuned models for specific languages. You select a language-specific model at NIM container launch time by using the `NIM_TAGS_SELECTOR` environment variable. Each container loads exactly one model.

| `NIM_TAGS_SELECTOR` | Model used |
| ------------------- | ---------- |
| _(not set)_ | Generic, language-agnostic model (default) |
| `language=de` | German |
| `language=es` | Spanish |
| `language=fr` | French |

Only `de`, `es`, and `fr` are supported. Any other value causes the container to fail at startup; omit the variable to use the generic model.

Profile selection stays automatic when you request a language: the NIM selects the profile that matches both your GPU architecture and the requested language.

The following example launches the NIM with the German model:

```bash
docker run -it --rm --name=lipsync-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_TAGS_SELECTOR="language=de" \
  -e NIM_HTTP_API_PORT=8000 \
  -e NIM_GRPC_API_PORT=8001 \
  -p 8000:8000 \
  -p 8001:8001 \
  nvcr.io/nim/nvidia/lipsync:latest
```

## Environment Variables

The following table describes the environment variables that can be passed into a NIM as a `-e` argument added to a `docker run` command. The `NV_AI4M_LS_*` variables that govern buffering and timeouts for the input streams are grouped and explained in [Input Stream Handling](/nim/maxine/lipsync/advanced-usage#input-stream-handling).

| ENV | Required? | Default | Notes |
| --- | --------- | ------- | ----- |
| `NGC_API_KEY` | Yes | None | You must set this variable to the value of your personal NGC API key. |
| `NIM_CACHE_PATH` | Optional | `/opt/nim/.cache` | Location (in container) where the container caches model artifacts. |
| `NIM_GRPC_API_PORT` | No | `8001` | Publish the gRPC `Lipsync` service on the prescribed port inside the container. Be sure to adjust the port passed to the `-p`/`--publish` flag of `docker run` to reflect that (for example, `-p $NIM_GRPC_API_PORT:$NIM_GRPC_API_PORT`). The left side of the colon (`:`) is the port of your host address and does *not* need to match `$NIM_GRPC_API_PORT`. The right side of the colon (`:`) is the port inside the container, which *must* match `NIM_GRPC_API_PORT` (or `8001` if not set). |
| `NIM_HTTP_API_PORT` | No | `8000` | Publish the NIM HTTP service to the prescribed port inside the container. Supported endpoints are `/v1/license` (returns the license information), `/v1/metadata` (returns metadata including asset information, license information, model information, and version), and `/v1/metrics` (exposes Prometheus metrics via an ASGI app endpoint). Refer to [HTTP API Endpoints](#http-api-endpoints).  |
| `NIM_MODEL_PROFILE` | Optional | None | Pins the NIM to one specific model profile. Leave it unset to let the NIM select the profile that matches the detected GPU architecture. The earlier name `NIM_MANIFEST_PROFILE` is still honored as a fallback when this variable is unset. For details and important caveats, refer to [Model Manifest Profile](/nim/maxine/lipsync/advanced-usage#model-manifest-profile). |
| `NIM_TAGS_SELECTOR` | Optional | None | Selects the language-specific model to load, in the form `language=<code>`; `<code>` is `de`, `es`, or `fr`. If not set, the generic, language-agnostic model is used. For details, refer to [Selecting a Language-Specific Model](#selecting-a-language-specific-model). |
| `NIM_SSL_MODE`      | No  | disabled | Set SSL security on the endpoints to `tls` or `mtls`. Defaults to unsecured endpoint. |
| `NIM_SSL_CA_CERTS_PATH` | No  | None | Set the path to CA root certificate inside the NIM. This is required only when `NIM_SSL_MODE` is `mtls`. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_CA_CERTS_PATH` can be set to `/opt/nim/crt/ssl_ca_cert.pem`. |
| `NIM_SSL_CERT_PATH` | No  | None | Set the path to the server's public SSL certificate inside the NIM. This is required only when an SSL mode is enabled. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_CERT_PATH` can be set to `/opt/nim/crt/ssl_cert_server.pem`. |
| `NIM_SSL_KEY_PATH`  | No  | None | Set the path to the server's private key inside the NIM. This is required only when an SSL mode is enabled. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_KEY_PATH` can be set to `/opt/nim/crt/ssl_key_server.pem`. |
| `NV_AI4M_LS_DEBUG_MODE` | No | `0` | Set to `1` to overlay the frame number, lipsync effect status, and speaker bounding boxes on each output frame. Intended for troubleshooting. Refer to [Debug Mode](/nim/maxine/lipsync/advanced-usage#debug-mode). |
| `NV_AI4M_MAX_CONCURRENCY_PER_GPU` | No | `1` | Number of concurrent inference requests the NIM server supports per GPU. Higher values consume more GPU memory and can cause out-of-memory errors. Buffer caps and coverage timeouts apply per request, so review them when you raise this value. Refer to [Input Stream Handling](/nim/maxine/lipsync/advanced-usage#input-stream-handling). |
| `NV_AI4M_LS_FIRST_BYTES_TIMEOUT_S` | No | `30` | Seconds to wait for the beginning of the video, which selects streaming or transactional mode. A client that opens a request and then sends no video fails with `DEADLINE_EXCEEDED`. Refer to [Streaming and Transactional Modes](/nim/maxine/lipsync/advanced-usage#streaming-and-transactional-modes). |
| `NV_AI4M_LS_VIDEO_COVERAGE_TIMEOUT_S` | No | `300` | Seconds to wait for the video needed to produce one output frame. Refer to [Coverage Timeouts](/nim/maxine/lipsync/advanced-usage#coverage-timeouts). |
| `NV_AI4M_LS_AUDIO_COVERAGE_TIMEOUT_S` | No | `30` | Seconds to wait for the speech audio needed to produce one output frame. Refer to [Coverage Timeouts](/nim/maxine/lipsync/advanced-usage#coverage-timeouts). |
| `NV_AI4M_LS_SPEAKER_INFO_COVERAGE_TIMEOUT_S` | No | `30` | Seconds to wait for the speaker information needed to produce one output frame. Applies only when you supply speaker data. Refer to [Coverage Timeouts](/nim/maxine/lipsync/advanced-usage#coverage-timeouts). |
| `NV_AI4M_LS_BACKGROUND_COVERAGE_TIMEOUT_S` | No | `30` | Seconds to wait for the background audio needed to produce one output frame. On expiry the background audio drops to silence instead of failing the request. Refer to [Coverage Timeouts](/nim/maxine/lipsync/advanced-usage#coverage-timeouts). |
| `NV_AI4M_LS_UNIT_COVERAGE_TIMEOUT_S` | No | Sum of the four values above | Overall cap on the time spent assembling one output frame across every input. Set to `0` to disable. Refer to [Coverage Timeouts](/nim/maxine/lipsync/advanced-usage#coverage-timeouts). |
| `NV_AI4M_LS_TRANSACTIONAL_COVERAGE_TIMEOUT_S` | No | `600` | Per-frame budget used in transactional mode, replacing the four per-stream values. Refer to [Coverage Timeouts](/nim/maxine/lipsync/advanced-usage#coverage-timeouts). |
| `NV_AI4M_LS_MAX_VIDEO_INPUT_BUFFER_MB` | No | `2048` | Cap, in MB, on encoded video held while the video decoder is behind. Refer to [Buffer Caps](/nim/maxine/lipsync/advanced-usage#buffer-caps). |
| `NV_AI4M_LS_MAX_AUDIO_INPUT_BUFFER_MB` | No | `512` | Cap, in MB, on encoded audio held while the audio decoder is behind. Speech and background audio each get this allowance. Refer to [Buffer Caps](/nim/maxine/lipsync/advanced-usage#buffer-caps). |
| `NV_AI4M_LS_MAX_SPEAKER_INFO_INPUT_BUFFER_FRAMES` | No | `1000000` | Cap, in `SpeakerInfoPerFrame` entries, on buffered speaker information. Refer to [Buffer Caps](/nim/maxine/lipsync/advanced-usage#buffer-caps). |
| `NV_AI4M_LS_MAX_VIDEO_QUEUE_SIZE` | No | `60` | Bound on the queue of decoded video frames waiting for inference. Refer to [Buffer Caps](/nim/maxine/lipsync/advanced-usage#buffer-caps). |

## Runtime Parameters for the Container

The following table describes the docker run flags used to launch the LipSync NIM container.

| Flags | Description |
|-|-|
| `-it` | `--interactive` + `--tty` (see [docker container run](https://docs.docker.com/reference/cli/docker/container/run/)). |
| `--rm` | Delete the container after it stops (see [docker container run](https://docs.docker.com/reference/cli/docker/container/run/)). |
| `--name=container-name` | Give a name to the NIM container. Use any preferred value. |
| `--runtime=nvidia` | Ensure NVIDIA drivers are accessible in the container. |
| `--gpus all` | Expose NVIDIA GPUs inside the container. If you are running on a host with multiple GPUs, you need to specify which GPU to use. You can also specify multiple GPUs. For more information about mounting specific GPUs, see [GPU Enumeration](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html#gpu-enumeration). |
| `--shm-size=8GB` | Allocate host memory for multi-process communication. |
| `-e NV_AI4M_MAX_CONCURRENCY_PER_GPU` | Number of concurrent inference requests to be supported by the NIM server per GPU (default: 1). Higher values consume more GPU memory and can cause out-of-memory errors. |
| `-e NGC_API_KEY=$NGC_API_KEY` | Provide the container with the token necessary to download adequate models and resources from NGC. See [NGC Authentication](#ngc-authentication). |
| `-p <host_port>:<container_port>` | Ports published by the container are directly accessible on the host port. |
| `-e NV_AI4M_LS_DEBUG_MODE` | Environment variable to enable debug mode that overlays frame number, lipsync effect status and bounding boxes for each frame. Enable by setting it to 1. (default: 0). |

## HTTP API Endpoints

In addition to the gRPC `Lipsync` service on `NIM_GRPC_API_PORT`, the NIM serves three
read-only HTTP endpoints on `NIM_HTTP_API_PORT` (default `8000`). Publish that port with
`-p 8000:8000` to reach them from the host.

| Endpoint | Method | Purpose |
| -------- | ------ | ------- |
| `/v1/license` | GET | Returns the license text shipped in the container. |
| `/v1/metadata` | GET | Returns asset information, license information, model information, and version. |
| `/v1/metrics` | GET | Exposes Prometheus metrics via an ASGI app endpoint. |

The following example queries the `/v1/metadata` endpoint from the host:

```bash
curl -s http://localhost:8000/v1/metadata
```

An abbreviated response:

```json
{
  "assetInfo": [""],
  "licenseInfo": {
    "name": "LICENSE",
    "path": "/opt/nim/LICENSE",
    "type": "file"
  },
  "modelInfo": [
    {
      "modelUrl": "ngc://nim/nvidia/lipsync:r22-sm89-en"
    }
  ]
}
```

<Note>
`/v1/metrics` on the HTTP port is distinct from the Triton metrics endpoint on port `9002`
described in [Observability](/nim/maxine/lipsync/observability). They are different services on different
ports.
</Note>

## Stopping the Container

The following command can be used to stop the container.

```bash
docker stop lipsync-nim
```