Getting Started
Prerequisites
Check the Support Matrix to make sure that you have the supported hardware and software stack.
NVIDIA NIM for Studio Voice can run on Linux or on Windows (under WSL2).
Running on Windows
You can run NVIDIA NIM for Studio Voice on RTX in a Windows-based system with Windows Subsystem for Linux (WSL) after taking the following additional steps.
Support for NIMs on WSL is in Public Beta.
-
Refer to the NVIDIA NIM on WSL documentation for setup instructions.
-
Refer to the NIM Model Profile Table to ensure that hardware and software requirements are met.
By default, WSL has access to half of system RAM. To change the memory available for WSL, create
.wslconfigin the home directoryC:\Users\<UserName>with the following content:Restart WSL instances to apply the configuration:
For any issues related to WSL setup, refer to Troubleshooting in the NVIDIA NIM on WSL2 documentation.
For any issues related to port forwarding, refer to Setting Up Port Forwarding.
-
Use the
podmancommand examples in the following sections.
NGC Authentication
Generate an API Key
An NGC API key is required to access NGC resources. You can generate a key at NGC API Keys.
When creating an NGC API Personal key, ensure that at least “NGC Catalog” is selected from the “Services Included” dropdown. More Services can be included if this key is to be reused for other purposes.
Personal keys allow you to configure an expiration date, revoke or delete the key using an action button, and rotate the key as needed. For more information about key types, please refer the NGC User Guide.
Export the API Key
Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.
If you are not familiar with how to create the NGC_API_KEY environment variable, the simplest way is to export it in your terminal:
Run one of the following commands to make the key available at startup:
Other, more secure options include saving the value in a file, so that you can retrieve with cat $NGC_API_KEY_FILE, or using a password manager.
Docker Login to NGC
To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:
docker
podman
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.
Starting the NIM Container
The following command launches the Studio Voice NIM container with the gRPC service. Find reference to runtime parameters for the container here.
Transactional NIM:
docker
podman
Streaming NIM:
docker
podman
The flag --gpus all is used to assign all available GPUs to the docker container. This fails on multiple GPU unless all GPUs are same. To assign specific GPU to the docker container (in case of different multiple GPUs available in your machine) use --gpus '"device=0,1,2..."'
Please note, required VRAM resource for running the NIM, varies between 2.3 and 3.7 GB VRAM depending on the model profile.
If the command runs successfully, you will get an output ending similar to the following:
By default, the Studio Voice gRPC service is hosted on port 8001. You must use this port for inferencing requests.
Selecting a Model Profile
Note that NIM_MODEL_PROFILE is an optional parameter. If NIM_MODEL_PROFILE is not provided, the NIM automatically selects a matching NIM_MODEL_PROFILE based on the target hardware architecture.
If STREAMING is set to true, the NIM automatically selects a profile that is compatible with your GPU of Model Type 48k-LL.
If STREAMING is set to false, the NIM automatically selects a profile that is compatible with your GPU of Model Type 48k-HQ.
However, if NIM_MODEL_PROFILE is used, ensure that the associated GPU architecture is compatible with the target hardware. If an incorrect NIM_MODEL_PROFILE is used, a deserialization error will occur on inference.
If a non-supported GPU is used for launching the NIM, you get an error:
nimlib.exceptions.NIMProfileIDNotFound: Could not match a profile in manifest at /opt/nim/etc/default/model_manifest.yaml.
For more information about NIM_MODEL_PROFILE, refer to the NIM Model Profile Table.
Environment Variables
The following table describes the environment variables that can be passed into a NIM as a -e argument added to a docker run command:
Runtime Parameters for the Container
Stopping the Container
The following commands can be used to stop the container.