Configuration

Riva TTS NIM uses docker containers under the hood. Each NIM has its own Docker container and there are several ways to configure it. The remainder of this section describes how to configure a NIM container.

GPU Selection

Passing --gpus all to docker run is not supported in environments with two or more GPUs.

In environments with a combination of GPUs, expose specific GPU inside the container using either:

• the --gpus flag (ex: --gpus='"device=1"')

• the environment variable NVIDIA_VISIBLE_DEVICES (ex: -e NVIDIA_VISIBLE_DEVICES=1)

The device ID(s) to use as input(s) are listed in the output of nvidia-smi -L:

GPU 0: Tesla H100 (UUID: GPU-b404a1a1-d532-5b5c-20bc-b34e37f3ac46)
GPU 1: NVIDIA GeForce RTX 3080 (UUID: GPU-b404a1a1-d532-5b5c-20bc-b34e37f3ac46)

Refer to the NVIDIA Container Toolkit documentation for more instructions.

Shared memory flag

Tokenization uses Triton’s Python backend capabilities that scales with the number of CPU cores available. You may need to increase the available shared memory given to the microservice container.

Example providing 1g of shared memory:

docker run ... --shm-size=1g ...

Environment Variables

The following table describes the environment variables that can be passed into a NIM as a -e argument added to a docker run command:

ENV	Required?	Default	Notes
NGC_API_KEY	Yes	None	You must set this variable to the value of your personal NGC API key.
NIM_CACHE_PATH	Yes	/opt/nim/.cache	Location (in container) where the container caches model artifacts.
NIM_HTTP_API_PORT	Yes	9000	Publish the NIM service to the prescribed port inside the container. Make sure to adjust the port passed to the -p/--publish flag of docker run to reflect that (ex: -p $NIM_HTTP_API_PORT:$NIM_HTTP_API_PORT). The left-hand side of this : is your host address:port, and does NOT have to match with $NIM_HTTP_API_PORT. The right-hand side of the : is the port inside the container which MUST match NIM_HTTP_API_PORT (or 9000 if not set). Supported endpoints are /v1/license (Returns the license information), /v1/metadata (Returns metadata including asset information, license information, model information, and version) and /v1/metrics (Exposes Prometheus metrics via an ASGI app endpoint).
NIM_GRPC_API_PORT	Yes	50051	Publish the Riva TTS service over GRPC to the prescribed port inside the container. Make sure to adjust the port passed to the -p/--publish flag of docker run to reflect that (ex: -p $NIM_GRPC_API_PORT:$NIM_GRPC_API_PORT). The left-hand side of this : is your host address:port, and does NOT have to match with $NIM_GRPC_API_PORT. The right-hand side of the : is the port inside the container which MUST match NIM_GRPC_API_PORT (or 50051 if not set).
NIM_SSL_MODE	No	"DISABLED"	Main switch to enable SSL/TLS or skip environment variables NIM_SSL_KEY_PATH, NIM_SSL_CERTS_PATH, NIM_SSL_CA_CERTS_PATH. Possible values: (1) "DISABLED" - no HTTPS, (2) "TLS" - HTTPS with only server-side TLS (client certificate not required), (3) "MTLS" - HTTPS with mTLS (client certificate required). If "TLS" is used, then NIM_SSL_CERTS_PATH, NIM_SSL_KEY_PATH are required. If "MTLS" is used, then NIM_SSL_CERTS_PATH, NIM_SSL_KEY_PATH, NIM_SSL_CA_CERTS_PATH are required.
NIM_SSL_KEY_PATH	Required if NIM_SSL_MODE is enabled	None	Path to the server’s TLS private key file (required for TLS HTTPS). It’s used to decrypt incoming messages and sign outgoing ones.
NIM_SSL_CERTS_PATH	Required if NIM_SSL_MODE is enabled	None	Path to the server’s certificate file (required for TLS HTTPS). It contains the public key and server identification information.
NIM_SSL_CA_CERTS_PATH	Required if NIM_SSL_MODE="MTLS"	None	Path to a the CA (certificate Authority) certificate.

Volumes

The following table describes the paths inside the container into which local paths can be mounted.

Container path	Required	Notes	Docker argument example
/opt/nim/.cache (or NIM_CACHE_PATH if present)	Not required, but if this volume is not mounted, the container will do a fresh download of the model each time it is brought up.	This is the directory within which models are downloaded inside the container. It is very important that this directory could be accessed from inside the container. This can be achieved by adding the option -u $(id -u) to the docker run command. For example, to use ~/.cache/nim as the host machine directory for caching models, first do mkdir -p ~/.cache/nim before running the docker run ... command.	-v ~/.cache/nim:/opt/nim/.cache -u $(id -u)