Getting Started

Check the Support Matrix to make sure that you have the supported hardware and software stack.

NGC Authentication

Generate an API key

An NGC API key is required to access NGC resources and a key can be generated here: https://org.ngc.nvidia.com/setup/personal-keys.

When creating an NGC API Personal key, ensure that at least “NGC Catalog” is selected from the “Services Included” dropdown. More Services can be included if this key is to be reused for other purposes.

Export the API key

Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.

If you’re not familiar with how to create the NGC_API_KEY environment variable, the simplest way is to export it in your terminal:

            

            
export NGC_API_KEY=<value>
        

Run one of the following commands to make the key available at startup:

            

            
# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc
        

Docker Login to NGC

To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:

            

            
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
        

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

The following command launches a container with the generic (non-optimized) model which can work on any of the supported GPUs. GPU specific optimized models are available for select GPUs. For using optimized models, refer to the table of Supported Models and specify NIM_MANIFEST_PROFILE according to your GPU.

            

            
export CONTAINER_NAME=parakeet-ctc-1.1b-asr
docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=9136dd64-4777-11ef-9f27-37cfd56fa6ee \
  -e NIM_HTTP_API_PORT=9000 \
  -e NIM_GRPC_API_PORT=50051 \
  -p 9000:9000 \
  -p 50051:50051 \
  nvcr.io/nim/nvidia/parakeet-ctc-1.1b-asr:1.0.0
        

1. Open a new terminal and run the following command to check if the service is ready to handle inference requests:

            

            
curl -X 'GET' 'http://localhost:9000/v1/health/ready'
        

If the service is ready, you get a response similar to the following.

            

            
{"ready":true}
        

2. Install the Riva Python client.

Riva uses gRPC APIs. You can download proto files from Riva gRPC Proto files and compile them to a target language using Protoc compiler. You can find Riva clients in C++ and Python languages at the following locations.

• Python Client Repository

• C++ Client Repository

Install Riva Python client

            

            
sudo apt-get install python3-pip
pip install -r https://raw.githubusercontent.com/nvidia-riva/python-clients/main/requirements.txt
pip install --force-reinstall git+https://github.com/nvidia-riva/python-clients.git
        

Download Riva sample client

            

            
git clone https://github.com/nvidia-riva/python-clients.git
        

3. Run Speech-to-Text (STT) inference.

In case you do not have a speech file available, you can use a sample speech file provided in the docker container launched in previous section.

            

            
# Copy sample wav file from running NIM container to host machine
docker cp $CONTAINER_NAME:/opt/riva/wav/en-US_sample.wav .
        

Riva ASR supports Mono, 16-bit audio in WAV, OPUS and FLAC formats.

• Streaming mode: Input speech file is streamed to the service chunk-by-chunk.

            

            
python3 python-clients/scripts/asr/transcribe_file.py --server 0.0.0.0:50051 --input-file en-US_sample.wav --language-code en-US
        

• Offline mode: Input speech file is sent to the service in one shot.

            

            
python3 python-clients/scripts/asr/transcribe_file_offline.py --server 0.0.0.0:50051 --input-file en-US_sample.wav --language-code en-US
        

Above commands will generate a transcript as shown below.

            

            
## what is natural language processing
        

The sample client supports the following options for making a transcription requests to the gRPC endpoint.

• --input-file - A path to a local file to stream.

• --language-code - Language of the input audio. Currently, only “en-US” is supported in Riva ASR NIM.

• --automatic-punctuation - Whether the transcript should be automatically punctuated.

• --boosted-lm-words - Words to boost when decoding. See customization for further information.

• --boosted-lm-score - Value by which to boost words. Recommended range for the boost score is 20 to 100. Refer to the customization page for more information.

On initial startup, the container will download the model from NGC. You can skip this download step on future runs by caching the model locally using a cache directory as in the example below.

            

            
# Create the cache directory on the host machine
export LOCAL_NIM_CACHE=~/.cache/nim_asr
mkdir -p "$LOCAL_NIM_CACHE"
chmod 777 $LOCAL_NIM_CACHE

# Run the container with the cache directory mounted in the appropriate location
docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=9136dd64-4777-11ef-9f27-37cfd56fa6ee \
  -e NIM_HTTP_API_PORT=9000 \
  -e NIM_GRPC_API_PORT=50051 \
  -p 9000:9000 \
  -p 50051:50051 \
  -v "$LOCAL_NIM_CACHE:/home/nvs/.cache/nim" \
  nvcr.io/nim/nvidia/parakeet-ctc-1.1b-asr:1.0.0
        

The following commands stop the container by stopping and removing the running docker container.

            

            
docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME