Runtime Parameters for Speech NIM Containers

When you run a Speech NIM container with docker run, the following flags and environment variables are commonly used. Service-specific examples are in the ASR, TTS, and NMT tutorials.

API Protocols and Ports

Each Speech NIM container exposes two ports. Multiple protocols share the HTTP port. Clients pick a protocol by the path or upgrade they request, not by connecting to a different port. Use this table to point each client at the correct port before running the deploy command:

Protocol	Default Port	Container Environment Variable	Used By
HTTP/REST	9000	NIM_HTTP_API_PORT	curl and OpenAPI clients (for example, /v1/audio/translations, /v1/text/translations, /v1/metrics, /v1/health/ready).
WebSocket	9000	NIM_HTTP_API_PORT	Realtime TTS streaming (realtime_tts_client.py) and the ASR WebSocket client. WebSocket upgrades over the same HTTP port.
gRPC	50051	NIM_GRPC_API_PORT	The Riva Python, C++, and Go clients and any other gRPC consumer.

Publish both ports in docker run (-p 9000:9000 -p 50051:50051). Targeting a WebSocket client at 50051, or a gRPC client at 9000, is the most common cause of connection refused on first run.

Triton’s internal ports (8000, 8001, 8002) are used by the inference stack inside the container and do not need to be published.

Common Flags and Variables

Flag or variable	Description
-it	Interactive + TTY (refer to Docker docs).
--rm	Remove the container when it stops (refer to Docker docs).
--name=<container-name>	Name for the container. Use any value you prefer; often set using CONTAINER_ID.
--runtime=nvidia	Make NVIDIA drivers available inside the container.
--gpus '"device=0"'	Expose GPU 0. On multi-GPU hosts, specify the device. Refer to GPU enumeration.
--shm-size=8GB	Shared memory for multi-GPU communication.
-e NGC_API_KEY	Token for downloading models and resources from NGC. Refer to NGC Access Setup.
-e NIM_HTTP_API_PORT=<port>	HTTP API port inside the container. Default 9000. Avoid 8000.
-e NIM_GRPC_API_PORT=<port>	gRPC API port inside the container. Default 50051.
-p 9000:9000	Map host port (left) to container HTTP port (right).
-p 50051:50051	Map host port (left) to container gRPC port (right).
-e NIM_TAGS_SELECTOR=<key=value,...>	Select a model/profile. Key-value pairs, for example, name=parakeet-1-1b-ctc-en-us,mode=all. Refer to the support matrix and service tutorials.

Optional: Realtime WebSocket Authentication

NIMs that expose the realtime WebSocket API support two independent authentication tiers, both off by default so that existing deployments are unaffected:

Variable	When required	Description
REALTIME_AUTH_MINT_API_KEY	Optional	Static key required on POST /v1/realtime/{transcription,synthesis}_sessions. Setting the key is what enables enforcement; there is no separate toggle. When unset, the endpoint is unauthenticated.
REALTIME_AUTH_REQUIRE_TOKEN	Optional	false (default) or true. When true, every /v1/realtime WebSocket handshake must present a valid ephemeral client_secret.value.

Your backend holds the long-lived key and exchanges it for a short-lived, single-use token that a browser presents on the WebSocket handshake, so no long-lived secret needs to reach the client and no WebSocket proxy is required. Because the two variables are independent, you can protect the session-creation endpoint first and require tokens on the socket later, once your clients have migrated.

    -e REALTIME_AUTH_REQUIRE_TOKEN=true \
    -e REALTIME_AUTH_MINT_API_KEY \

Passing -e REALTIME_AUTH_MINT_API_KEY without a value makes Docker inherit it from your shell, keeping the secret out of your shell history and the host process list.

For the token flow, client code, and error handling, refer to the realtime API references for ASR and TTS.

Optional: TLS/mTLS

Some NIMs support HTTPS using environment variables:

Variable	When required	Description
NIM_SSL_MODE	Optional	DISABLED (default), TLS (server-side only), or MTLS (client cert required).
NIM_SSL_KEY_PATH	When SSL is enabled	Path to the server TLS private key.
NIM_SSL_CERTS_PATH	When SSL is enabled	Path to the server certificate.
NIM_SSL_CA_CERTS_PATH	When NIM_SSL_MODE=MTLS	Path to the CA certificate.

Check the support matrix and service docs for which images support SSL.