Running Inference
Verify Server Health
Perform a health check on the gRPC endpoint.
-
Install
grpcurlfrom the fullstorydev/grpcurl repo.Example commands to run on Ubuntu:
-
Download the health checking proto:
-
Run the health check on
<server_ip>or localhost:If the service is ready, you get a response similar to the following:
To use grpcurl with an SSL enabled server, avoid using the --plaintext argument. Instead, use --cacert with a CA certificate, --key with a private key, or --cert with a certificate file. For more details, refer to grpcurl --help.
Running Inference via Script
1. Clone the Repository
Download the BNR Python client code by cloning the NVIDIA Maxine NIM Clients repository:
2. Install Dependencies
3. Run the Python Client
You can use the sample client script in the BNR GitHub repo to send a gRPC request to the hosted NIM server:
-
Go to the scripts directory.
-
Ensure that the server is launched in the correct mode.
- To use the client in Streaming mode, launch the NIM in Streaming mode.
- To use the client in Transactional mode, launch the NIM in Transactional mode.
-
Run the command to send the gRPC request.
Transactional mode: The following example command processes the packaged sample audio file and generates a bnr_48k_output.wav file in the current folder.
Streaming mode: The following example command streams the raw audio data from the sample audio file in 10 ms chunks and saves streamed output from the NIM to bnr_48k_output.wav file in the current folder.
Only WAV files are supported as inputs.
For more details on modes, refer to Overview.
When using --streaming mode, ensure that the selected --sample-rate (48000, 16000) aligns with the NIM_MODEL_PROFILE model type configuration to maintain compatibility.
eg: if --sample-rate is 48000, then NIM_MODEL_PROFILE with model type v1-48k or v1-16k needs to be selected.
Be aware that the first inference isn’t indicative of the model’s performance.
Usage for Preview API Request
The Preview API provides a hosted version of BNR, which can be used for testing.
To use the hosted version of the API, use the following command:
For further details, refer to the Try API documentation.
Command-Line Arguments
To view the details of command-line arguments, run this command:
--target <ip:port>- URI of NIM’s gRPC service. Usegrpc.nvcf.nvidia.com:443when hosted on NVCF. Default value is127.0.0.1:8001.--preview-mode- Flag to send request to preview NVCF server on Try API.--ssl-mode- Set the SSL mode toTLSorMTLS. Defaults to no SSL. When running preview, TLS mode must be used with the default root certificate.--ssl-key- The path to client’s PEM encoded private key. Only required for MTLS mode.--ssl-cert- The path to client’s PEM encoded public certificate. Only required for MTLS mode.--ssl-root-cert- The path to PEM encoded root certificate. Used only for TLS or MTLS modes.--api-key <NGC_API_KEY>- NGC API key required for authentication. Utilized when using TRY API, ignored otherwise.--function-id- NVCF function ID for the service. Utilized when using TRY API, ignored otherwise.--input- The path to the input audio file. Default value is../assets/bnr_48k_input.wav.--output- The path for the output audio file. Default is./bnr_48k_output.wav.--streaming- Flag to enable grpc streaming mode.--sample-rate- Sample rate of input audio file in Hz. It can be set to48000or16000. Default value is48000.--intensity-ratio- Intensity ratio value between 0 and 1 to control denoising intensity. Default is 1.0 (maximum denoising).