Running Inference#

Verify Server Health#

Perform a health check on the gRPC endpoint.

  1. Install grpcurl from the fullstorydev/grpcurl repo.

    Example commands to run on Ubuntu:

    wget https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_amd64.deb
sudo dpkg -i grpcurl_1.9.1_linux_amd64.deb

  2. Download the health checking proto:

    wget https://raw.githubusercontent.com/grpc/grpc/master/src/proto/grpc/health/v1/health.proto

  3. Run the health check on <server_ip> or localhost:

    grpcurl --plaintext --proto health.proto <server_ip>:8001 grpc.health.v1.Health/Check

    If the service is ready, you get a response similar to the following:

    { "status": "SERVING" }

Note

To use grpcurl with an SSL enabled server, avoid using the --plaintext argument. Instead, use --cacert with a CA certificate, --key with a private key, or --cert with a certificate file. For more details, refer to grpcurl --help.

Running Inference via Script#

1. Clone the Repository#

Download the BNR Python client code by cloning the NVIDIA Maxine NIM Clients repository:

git clone https://github.com/NVIDIA-Maxine/nim-clients.git

# Go to the 'bnr' folder
cd nim-clients/bnr/

2. Install Dependencies#

sudo apt-get install python3-pip
pip install -r requirements.txt

3. Run the Python Client#

You can use the sample client script in the BNR GitHub repo to send a gRPC request to the hosted NIM server:

  1. Go to the scripts directory.

    cd scripts

  2. Ensure that the server is launched in the correct mode.

    • To use the client in Streaming mode, launch the NIM in Streaming mode.

    • To use the client in Transactional mode, launch the NIM in Transactional mode.

  3. Run the command to send the gRPC request.

    python bnr.py --target <server_ip:port> --input <input_audio_file_path> --output <output_audio_file_path>

Transactional mode: The following example command processes the packaged sample audio file and generates a bnr_48k_output.wav file in the current folder.

python bnr.py --target 127.0.0.1:8001 --input ../assets/bnr_48k_input.wav --output bnr_48k_output.wav

Streaming mode: The following example command streams the raw audio data from the sample audio file in 10 ms chunks and saves streamed output from the NIM to bnr_48k_output.wav file in the current folder.

python bnr.py --target 127.0.0.1:8001 --input ../assets/bnr_48k_input.wav --output bnr_48k_output.wav --streaming --sample-rate 48000

Only WAV files are supported as inputs.

For more details on modes, refer to Overview.

Note

When using --streaming mode, ensure that the selected --sample-rate (48000, 16000) aligns with the NIM_MODEL_PROFILE model type configuration to maintain compatibility. eg: if --sample-rate is 48000, then NIM_MODEL_PROFILE with model type v1-48k or v1-16k needs to be selected.

Note

Be aware that the first inference isn’t indicative of the model’s performance.

Usage for Preview API Request#

The Preview API provides a hosted version of BNR, which can be used for testing.

To use the hosted version of the API, use the following command:

python bnr.py --preview-mode \
    --ssl-mode TLS \
    --target grpc.nvcf.nvidia.com:443 \
    --function-id <function_id> \
    --api-key $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC \
    --input <input_file_path> \
    --output <output_file_path>

For further details, refer to the Try API documentation.

Command-Line Arguments#

To view the details of command-line arguments, run this command:

python bnr.py -h

  • --target <ip:port> - URI of NIM’s gRPC service. Use grpc.nvcf.nvidia.com:443 when hosted on NVCF. Default value is 127.0.0.1:8001.

  • --preview-mode - Flag to send request to preview NVCF server on Try API.

  • --ssl-mode - Set the SSL mode to TLS or MTLS. Defaults to no SSL. When running preview, TLS mode must be used with the default root certificate.

  • --ssl-key - The path to client’s PEM encoded private key. Only required for MTLS mode.

  • --ssl-cert - The path to client’s PEM encoded public certificate. Only required for MTLS mode.

  • --ssl-root-cert - The path to PEM encoded root certificate. Used only for TLS or MTLS modes.

  • --api-key <NGC_API_KEY> - NGC API key required for authentication. Utilized when using TRY API, ignored otherwise.

  • --function-id - NVCF function ID for the service. Utilized when using TRY API, ignored otherwise.

  • --input - The path to the input audio file. Default value is ../assets/bnr_48k_input.wav.

  • --output - The path for the output audio file. Default is ./bnr_48k_output.wav.

  • --streaming - Flag to enable grpc streaming mode.

  • --sample-rate - Sample rate of input audio file in Hz. It can be set to 48000 or 16000. Default value is 48000.

  • --intensity-ratio - Intensity ratio value between 0 and 1 to control denoising intensity. Default is 1.0 (maximum denoising).