Getting Started

Check the Support Matrix to make sure that you have the supported hardware and software stack.

NGC Authentication

Generate an API key

An NGC API key is required to access NGC resources and a key can be generated here: https://org.ngc.nvidia.com/setup/personal-keys.

When creating an NGC API Personal key, ensure that at least “NGC Catalog” is selected from the “Services Included” dropdown. More Services can be included if this key is to be reused for other purposes.

Export the API key

Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.

If you’re not familiar with how to create the NGC_API_KEY environment variable, the simplest way is to export it in your terminal:

            

            
export NGC_API_KEY=<value>
        

Run one of the following commands to make the key available at startup:

            

            
# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc
        

Docker Login to NGC

To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:

            

            
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
        

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

The following command launches a container with the generic (non-optimized) model that can work on any of the supported GPUs. GPU-specific optimized models are available for select GPUs. To use optimized models, refer the table of Supported Models and specify NIM_MANIFEST_PROFILE according to your GPU.

            

            
export CONTAINER_NAME=fastpitch-hifigan-tts
docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=3c8ee3ee-477f-11ef-aa12-1b4e6406fad5 \
  -e NIM_HTTP_API_PORT=9000 \
  -e NIM_GRPC_API_PORT=50051 \
  -p 9000:9000 \
  -p 50051:50051 \
  nvcr.io/nim/nvidia/fastpitch-hifigan-tts:1.0.0
        

1. Open a new terminal and run the following command to check if the service is ready to handle inference requests:

            

            
curl -X 'GET' 'http://localhost:9000/v1/health/ready'
        

If the service is ready, you get a response similar to the following.

            

            
{"ready":true}
        

2. Install the Riva Python client

Riva uses gRPC APIs. You can download proto files from Riva gRPC Proto files and compile them to a target language using Protoc compiler. You can find Riva clients in C++ and Python languages at the following locations.

• Python Client Repository

• C++ Client Repository

Install Riva Python client

            

            
sudo apt-get install python3-pip
pip install -r https://raw.githubusercontent.com/nvidia-riva/python-clients/main/requirements.txt
pip install --force-reinstall git+https://github.com/nvidia-riva/python-clients.git
        

Download Riva sample client

            

            
git clone https://github.com/nvidia-riva/python-clients.git
        

3. Run Text-to-Speech (TTS) inference:

            

            
python3 python-clients/scripts/tts/talk.py --server 0.0.0.0:50051 --text "Hello, this is a speech synthesizer." --voice "English-US.Female-1" --language-code en-US --output output.wav
        

On running the above command, the synthesized audio file named output.wav will be created. Available voices can be queried using --list-voices option.

            

            
python3 python-clients/scripts/tts/talk.py --server 0.0.0.0:50051 --list-voices
        

The sample client supports the following options to make a transcription request to the gRPC endpoint.

• --text - Text input to synthesize, supports providing SSML tags with text. See customization for further information.

• --language-code - A language of input text. Currently, only “en-US” is supported in Riva TTS NIM.

• --list-voices - List available voices. This argument should be used exclusively just to query the available voices before running inference.

• --voice - A voice name to use. You can determine the value from the output of --list-voices option.

On initial startup, the container will download the model from NGC. You can skip this download step on future runs by caching the model locally using a cache directory as in the example below.

            

            
# Create the cache directory on the host machine
export LOCAL_NIM_CACHE=~/.cache/nim_tts
mkdir -p "$LOCAL_NIM_CACHE"
chmod 777 $LOCAL_NIM_CACHE

# Run the container with the cache directory mounted in the appropriate location
docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=3c8ee3ee-477f-11ef-aa12-1b4e6406fad5 \
  -e NIM_HTTP_API_PORT=9000 \
  -e NIM_GRPC_API_PORT=50051 \
  -p 9000:9000 \
  -p 50051:50051 \
  -v "$LOCAL_NIM_CACHE:/home/nvs/.cache/nim" \
  nvcr.io/nim/nvidia/fastpitch-hifigan-tts:1.0.0
        

The following commands stop the container by stopping and removing the running docker container.

            

            
docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME