Running Inference#
Verify Server Health#
Perform a health check on the gRPC endpoint.
Install
grpcurl
from hereExample commands to run on Ubuntu:
wget https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_amd64.deb sudo dpkg -i grpcurl_1.9.1_linux_amd64.deb
Download the health checking proto:
wget https://raw.githubusercontent.com/grpc/grpc/master/src/proto/grpc/health/v1/health.proto
Run the health check on
<server_ip>
or localhost:grpcurl --plaintext --proto health.proto <server_ip>:8001 grpc.health.v1.Health/Check
If the service is ready, you will get a response similar to the following:
{ "status": "SERVING" }
Note
For using grpcurl with an SSL enabled server, avoid using --plaintext
argument, and use --cacert
with CA certificate, --key
with private key and --cert
with certificate files. Refer to grpcurl --help
for more details.
Running Inference via Script#
1. Clone the Repository#
Download the Studio Voice Python client code by cloning the NVIDIA Maxine NIM Clients Repository:
git clone https://github.com/NVIDIA-Maxine/nim-clients.git
# Go to the 'studio-voice' folder
cd nim-clients/studio-voice/
2. Install Dependencies#
sudo apt-get install python3-pip
pip install -r requirements.txt
3. Run the Python Client#
You can use the sample client script in the Studio Voice GitHub repo to send a gRPC request to the hosted NIM server:
Go to the scripts directory.
cd scripts
Run the command to send gRPC request.
python studio_voice.py --target <server_ip:port> --input <input_audio_file_path> --output <output_audio_file_path>
Transactional mode: The following example command processes the packaged sample audio file and generates a studio_voice_48k_output.wav
file in the current folder.
python studio_voice.py --target 127.0.0.1:8001 --input ../assets/studio_voice_48k_input.wav --output studio_voice_48k_output.wav
Streaming mode: The following example command processes the packaged sample audio file and generates a studio_voice_48k_output.wav
file in the current folder.
python studio_voice.py --target 127.0.0.1:8001 --input ../assets/studio_voice_48k_input.wav --output studio_voice_48k_output.wav --streaming --model-type 48k-ll
Note
When using --streaming
mode, ensure the selected --model-type
(48k-hq, 48k-ll, or 16k-hq) aligns with the NIM_MODEL_PROFILE
Model Type configuration to maintain compatibility .
Note
To use the client in Streaming mode, launch the NIM in Streaming mode. Similarly, to use the client in Transactional mode, launch the NIM in Transactional mode.
Note
Only WAV files are supported.
Note
Please note that the first inference isn’t indicative of the model’s performance.
Usage for Preview API Request#
python studio_voice.py --preview-mode \
--ssl-mode TLS \
--target grpc.nvcf.nvidia.com:443 \
--function-id <function_id> \
--api-key $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC \
--input <input_file_path> \
--output <output_file_path>
Command Line Arguments#
To view the details of command line arguments, run this command:
python studio_voice.py -h
--target <ip:port>
- URI of NIM’s gRPC service. Usegrpc.nvcf.nvidia.com:443
when hosted on NVCF. Default value is127.0.0.1:8001
.--preview-mode
- Flag to send request to preview NVCF server on Try API.--ssl-mode
- Set the SSL mode toTLS
orMTLS
. Defaults to no SSL. When running preview, TLS mode must be used with the default root certificate.--ssl-key
- The path to client’s PEM encoded private key. Only required for MTLS mode.--ssl-cert
- The path to client’s PEM encoded public certificate. Only required for MTLS mode.--ssl-root-cert
- The path to PEM encoded root certificate. Used only for TLS or MTLS modes.--api-key <NGC_API_KEY>
- NGC API key required for authentication. Utilized when using TRY API, ignored otherwise.--function-id
- NVCF function ID for the service. Utilized when using TRY API, ignored otherwise.--input
- The path to the input audio file. Default value is../assets/studio_voice_48k_input.wav
.--output
- The path for the output audio file. Default is./studio_voice_48k_output.wav
.--streaming
Flag to enable grpc streaming mode.--model-type
{48k-hq,48k-ll,16k-hq} Studio Voice model type, default is 48k-hq.