Is this page helpful?

Getting Started#

Authenticate with NGC, launch the Eye Contact NIM container, and set environment variables and runtime flags before you send inference requests.

Prerequisites#

Check the Support Matrix to make sure that you have the supported hardware and software stack.

NGC Authentication#

An NGC API key is required to pull the container image and download models. Complete the following steps to create the key, export it, and log in to the NVIDIA Container Registry.

Generate an API Key#

An NGC API key is required to access NGC resources. You can generate a key at NGC API Keys.

When creating an NGC API Personal key, ensure that at least “NGC Catalog” is selected from the “Services Included” dropdown. More Services can be included if this key is to be reused for other purposes.

Note

Personal keys allow you to configure an expiration date, revoke or delete the key using an action button, and rotate the key as needed. For more information about key types, please refer the NGC User Guide.

Export the API Key#

Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.

If you are not familiar with how to create the NGC_API_KEY environment variable, the simplest way is to export it in your terminal:

export NGC_API_KEY=<value>

Run one of the following commands to make the key available at startup:

# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc

Note

Other, more secure options include saving the value in a file, so that you can retrieve with cat $NGC_API_KEY_FILE, or using a password manager.

Docker Login to NGC#

To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

Launching the NIM Container#

The following command launches the Eye Contact NIM container with the gRPC service. For a list of parameters, refer to Runtime Parameters for the Container.

# Run the container
docker run -it --rm --name=maxine-eye-contact-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_HTTP_API_PORT=8000 \
  -e NV_AI4M_MAX_CONCURRENCY_PER_GPU=1 \
  -p 8000:8000 \
  -p 8001:8001 \
  nvcr.io/nim/nvidia/maxine-eye-contact:latest

There is also an option to cache the model manifest locally. For more details, refer to Model Caching.

Note

The flag --gpus all assigns all available GPUs to the Docker container. To assign specific GPUs to the Docker container (in case of multiple GPUs available in your machine), use --gpus '"device=0,1,2..."'.

If the command runs successfully, you get a response similar to the following.

I1126 09:22:21.048202 31 grpc_server.cc:2558] "Started GRPCInferenceService at 127.0.0.1:9001"
I1126 09:22:21.048377 31 http_server.cc:4704] "Started HTTPService at 127.0.0.1:9000"
I1126 09:22:21.089295 31 http_server.cc:362] "Started Metrics Service at 127.0.0.1:9002"
Maxine GRPC Service: Listening to 0.0.0.0:8001

Note

By default, the Eye Contact gRPC service is hosted on port 8001. You must use this port for inferencing requests.

Environment Variables#

The following table describes the environment variables that can be passed into a NIM as a -e argument added to a docker run command:

ENV	Required?	Default	Notes
`NGC_API_KEY`	Yes	None	You must set this variable to the value of your personal NGC API key.
`NIM_CACHE_PATH`	Optional	`/opt/nim/.cache`	Location (in container) where the container caches model artifacts.
`NIM_HTTP_API_PORT`	No	`8000`	Publish the NIM service to the prescribed port inside the container. Be sure to adjust the port passed to the `-p`/`--publish` flag of `docker run` to reflect that (for example, `-p $NIM_HTTP_API_PORT:$NIM_HTTP_API_PORT`). The left side of the colon (`:`) is the port of your host address and does not need to match `$NIM_HTTP_API_PORT`. The right side of the colon (`:`) is the port inside the container, which must match `NIM_HTTP_API_PORT` (or `8000` if not set). Supported endpoints are `/v1/license` (returns the license information), `/v1/metadata` (returns metadata, including asset information, license information, model information, and version), and `/v1/metrics` (exposes Prometheus metrics via an ASGI app endpoint).
`NIM_MODEL_PROFILE`	Optional	None	If unset, the NIM selects a profile that matches the GPU compute capability. Set this only to pin a specific profile. A profile that does not match the GPU causes a deserialization error on inference. Refer to Model Profiles and Selection for more information.
`NIM_SSL_MODE`	No	disabled	Set SSL security on the endpoints to `tls` or `mtls`. Defaults to unsecured endpoint.
`NIM_SSL_CA_PATH`	No	None	Set the path to CA root certificate inside the NIM. This is required only when `NIM_SSL_MODE` is `mtls`. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_CA_PATH` can be set to `/opt/nim/crt/ssl_ca_cert.pem`.
`NIM_SSL_CERT_PATH`	No	None	Set the path to the server’s public SSL certificate inside the NIM. This is required only when an SSL mode is enabled. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_CERT_PATH` can be set to `/opt/nim/crt/ssl_cert_server.pem`.
`NIM_SSL_KEY_PATH`	No	None	Set the path to the server’s private key inside the NIM. This is required only when an SSL mode is enabled. For example, if the SSL certificates are mounted at `/opt/nim/crt` in the NIM, `NIM_SSL_KEY_PATH` can be set to `/opt/nim/crt/ssl_key_server.pem`.

Runtime Parameters for the Container#

The following table describes the docker run flags used to launch the Eye Contact NIM container.

Flags	Description
`-it`	`--interactive` + `--tty` (see docker container run in Docker Docs)
`--rm`	Delete the container after it stops (see docker container run in Docker Docs)
`--name=container-name`	Give a name to the NIM container. Use any preferred value.
`--runtime=nvidia`	Ensure NVIDIA drivers are accessible in the container.
`--gpus all`	Expose NVIDIA GPUs inside the container. If you are running on a host with multiple GPUs, you need to specify which GPU to use. You can also specify multiple GPUs. For more information about mounting specific GPUs, refer to GPU Enumeration.
`--shm-size=8GB`	Allocate host memory for multi-process communication.
`-e NGC_API_KEY=$NGC_API_KEY`	Provide the container with the token necessary to download adequate models and resources from NGC. Refer to NGC Authentication.
`-e NV_AI4M_MAX_CONCURRENCY_PER_GPU`	Number of concurrent inference requests to be supported by the NIM server per GPU.
`-e AI4M_MAX_INPUT_FILE_SIZE_MB`	Environment variable to configure maximum file size (in MB) supported by the server (no default).
`-p <host_port>:<container_port>`	Ports published by the container are directly accessible on the host port.

Stopping the Container#

Use the following commands to stop the container:

docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME