Getting Started#

Prerequisites#

To ensure that you have the supported hardware and software stack, check the Support Matrix.

NGC Authentication#

Generate an API Key#

An NGC API key is required to access NGC resources. You can generate a key at https://org.ngc.nvidia.com/setup/api-keys.

When creating an NGC API Personal key, ensure that at least NGC Catalog is selected from the Services Included dropdown. You can include more services if this key is to be reused for other purposes.

Note

Personal keys allow you to configure an expiration date, revoke or delete the key using an action button, and rotate the key as needed. For more information about key types, refer to NGC API Keys in the NGC User Guide.

Export the NGC API Key#

To download the appropriate models and resources when starting the NIM, use the NGC_API_KEY environment variable to pass the value of the API key to the docker run command in the next section.

The simplest way to create the NGC_API_KEY environment variable is to export it in your terminal:

export NGC_API_KEY=<value>

Run one of the following commands to make the key available at startup:

# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc

Note

Other, more secure options include saving the value in a file, so that you can retrieve with cat $NGC_API_KEY_FILE, or using a password manager.

Docker Login to NGC#

To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

Launching the NIM Container#

The following command launches the Active Speaker Detection NIM container with the gRPC service. (For a list of parameters, see Runtime Parameters for the Container.)

# Choose manifest profile ID based on target architecture.
export MANIFEST_PROFILE_ID=<enter_manifest_profile_id>

Then run the NIM launch command:

docker run -it --rm --name=active-speaker-detection-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=$MANIFEST_PROFILE_ID \
  -e NV_AI4M_MAX_CONCURRENCY_PER_GPU=1 \
  -e NIM_HTTP_API_PORT=8000 \
  -e NIM_GRPC_API_PORT=8001 \
  -p 8000:8000 \
  -p 8001:8001 \
  nvcr.io/nim/nvidia/active-speaker-detection:latest

Note

The flag --gpus all is used to assign all available GPUs to the NIM container. To assign specific GPUs to the NIM container (in case of multiple GPUs available in your machine), use --gpus '"device=0,1,2..."'.

Model Manifest Profiles#

The following table lists manifest profile IDs that you can specify in MANIFEST_PROFILE_ID.

GPU Architecture (compute capability)	Manifest Profile ID
Blackwell (cc 12.0/10.0)	dcc949a9d8d1c5e492e0954c79f716244574329d6e4b661abf81fc9d7f7158a5
Ada (cc 8.9)	78a24c0c4c1858aef26c964f64ac6b6a4b8b75197e23e7ee243d71fc6939ff98
Ampere (cc 8.6)	e1bba1aa38a23d4b89f3fdfc5a6707cda027e946e4993d230799c41f8dded8c8
Turing (cc 7.5)	52cab9095cb4dcefd3bca0484b1939046d4b44a3d2cf8201e84509fc712fba53

Note

MANIFEST_PROFILE_ID is an optional parameter. If the manifest profile ID is not supplied, the NIM automatically selects a matching profile ID based on the target hardware architecture.

However, if MANIFEST_PROFILE_ID is used, ensure that the associated GPU architecture is compatible with the target hardware. If an incorrect manifest profile ID is used, a deserialization error occurs on inference.

If the NIM launch is successful, you get a response similar to the following.

I1027 22:31:44.952125 123 grpc_server.cc:2560] "Started GRPCInferenceService at 127.0.0.1:9001"
I1027 22:31:44.952247 123 http_server.cc:4755] "Started HTTPService at 127.0.0.1:9000"
I1027 22:31:44.993329 123 http_server.cc:358] "Started Metrics Service at 127.0.0.1:9002"
Triton server is ready
[INFO AI4M BASE LOGGER 2026-03-11 10:55:57.487 PID:195] Listening to 0.0.0.0:8001

Note

By default, the Active Speaker Detection NIM gRPC service is hosted on port 8001. You must use this port for inferencing requests. The port is configurable via the NIM_GRPC_API_PORT environment variable.

Environment Variables#

The following table describes the environment variables that can be passed into a NIM as a -e argument added to a docker run command:

ENV	Required?	Default	Notes
`NGC_API_KEY`	Yes	None	You must set this variable to the value of your personal NGC API key.
`NIM_CACHE_PATH`	Optional	`/opt/nim/.cache`	Location (in container) where the container caches model artifacts.
`NIM_GRPC_API_PORT`	No	`8001`	Publish the NIM gRPC service to the prescribed port inside the container. Be sure to adjust the port passed to the `-p`/`--publish` flag of `docker run` accordingly (for example, `-p $NIM_GRPC_API_PORT:$NIM_GRPC_API_PORT`). The left side of the colon (`:`) is the port of your host address and does not need to match `$NIM_GRPC_API_PORT`. The right side of the colon (`:`) is the port inside the container, which must match `NIM_GRPC_API_PORT` (or `8001` if not set).
`NIM_HTTP_API_PORT`	No	`8000`	Publish the NIM HTTP service to the prescribed port inside the container. Supported endpoints are `/v1/license` (returns the license information), `/v1/metadata` (returns metadata including asset information, license information, model information, and version), and `/v1/metrics` (exposes Prometheus metrics via an ASGI app endpoint).
`NIM_MANIFEST_PROFILE`	Optional	None	You must set this model profile to be able to download the specific model type supported on your GPU. For more about `NIM_MANIFEST_PROFILE`, refer to Model Manifest Profiles.
`NIM_SSL_MODE`	No	disabled	Set SSL security on the endpoints to `tls` or `mtls`. Defaults to unsecured endpoint.
`NIM_SSL_CA_PATH`	No	None	Set the path to CA root certificate inside the NIM. This is required only when `NIM_SSL_MODE` is `mtls`. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_CA_PATH` can be set to `/opt/nim/crt/ssl_ca_cert.pem`.
`NIM_SSL_CERT_PATH`	No	None	Set the path to the server’s public SSL certificate inside the NIM. This is required only when an SSL mode is enabled. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_CERT_PATH` can be set to `/opt/nim/crt/ssl_cert_server.pem`.
`NIM_SSL_KEY_PATH`	No	None	Set the path to the server’s private key inside the NIM. This is required only when an SSL mode is enabled. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_KEY_PATH` can be set to `/opt/nim/crt/ssl_key_server.pem`.
`NV_AI4M_ASD_MAX_SPEAKERS`	No	`5`	Maximum number of audio speaker streams supported. Controls both the gRPC service and the Triton backend.
`NV_AI4M_MAX_CONCURRENCY_PER_GPU`	No	`1`	Number of concurrent inference requests to be supported by the NIM server per GPU. Total concurrency = `NV_AI4M_MAX_CONCURRENCY_PER_GPU` × number of GPUs.

Runtime Parameters for the Container#

Flags	Description
`-it`	`--interactive` + `--tty` (see docker container run).
`--rm`	Delete the container after it stops (see docker container run).
`--name=container-name`	Give a name to the NIM container. Use any preferred value.
`--runtime=nvidia`	Ensure NVIDIA drivers are accessible in the container.
`--gpus all`	Expose NVIDIA GPUs inside the container. If you are running on a host with multiple GPUs, you need to specify which GPU to use. You can also specify multiple GPUs. For more information about mounting specific GPUs, see GPU Enumeration.
`--shm-size=8GB`	Allocate host memory for multi-process communication.
`-e NV_AI4M_MAX_CONCURRENCY_PER_GPU`	Controls the number of concurrent inference requests processed simultaneously per GPU. Default: 1. Higher values enable parallel request processing but might reduce individual request performance due to resource sharing.
`-e NV_AI4M_ASD_MAX_SPEAKERS`	Maximum number of audio speaker streams. Default: 4. This setting controls both the gRPC service and the Triton backend.
`-e NGC_API_KEY=$NGC_API_KEY`	Provide the container with the token necessary to download adequate models and resources from NGC. See NGC Authentication.
`-p <host_port>:<container_port>`	Ports published by the container are directly accessible on the host port.

Stopping the Container#

Use the following commands to stop the container:

docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME