Running Inference
Verify Server Health
Perform a health check on the gRPC endpoint.
-
Install
grpcurlfrom hereExample commands to run on Ubuntu:
-
Download the health checking proto:
-
Run the health check on
<server_ip>or localhost:
If the service is ready, you will get a response similar to the following:
For using grpcurl with an SSL enabled server, avoid using --plaintext argument, and use --cacert with CA certificate, --key with private key and --cert with certificate files. Refer to grpcurl --help for more details.
Running Inference via Script
1. Clone the Repository
Download the Studio Voice Python client code by cloning the NVIDIA Maxine NIM Clients Repository:
2. Install Dependencies
3. Run the Python Client
You can use the sample client script in the Studio Voice GitHub repo to send a gRPC request to the hosted NIM server:
-
Go to the scripts directory.
-
Run the command to send gRPC request.
Transactional mode: The following example command processes the packaged sample audio file and generates a studio_voice_48k_output.wav file in the current folder.
Streaming mode: The following example command processes the packaged sample audio file and generates a studio_voice_48k_output.wav file in the current folder.
When using --streaming mode, ensure the selected --model-type (48k-hq, 48k-ll, or 16k-hq) aligns with the NIM_MODEL_PROFILE Model Type configuration to maintain compatibility .
To use the client in Streaming mode, launch the NIM in Streaming mode. Similarly, to use the client in Transactional mode, launch the NIM in Transactional mode.
Only WAV files are supported.
Please note that the first inference isn’t indicative of the model’s performance.
Usage for Preview API Request
Command Line Arguments
To view the details of command line arguments, run this command:
--target <ip:port>- URI of NIM’s gRPC service. Usegrpc.nvcf.nvidia.com:443when hosted on NVCF. Default value is127.0.0.1:8001.--preview-mode- Flag to send request to preview NVCF server on Try API.--ssl-mode- Set the SSL mode toTLSorMTLS. Defaults to no SSL. When running preview, TLS mode must be used with the default root certificate.--ssl-key- The path to client’s PEM encoded private key. Only required for MTLS mode.--ssl-cert- The path to client’s PEM encoded public certificate. Only required for MTLS mode.--ssl-root-cert- The path to PEM encoded root certificate. Used only for TLS or MTLS modes.--api-key <NGC_API_KEY>- NGC API key required for authentication. Utilized when using TRY API, ignored otherwise.--function-id- NVCF function ID for the service. Utilized when using TRY API, ignored otherwise.--input- The path to the input audio file. Default value is../assets/studio_voice_48k_input.wav.--output- The path for the output audio file. Default is./studio_voice_48k_output.wav.--streamingFlag to enable grpc streaming mode.--model-type{48k-hq,48k-ll,16k-hq}Studio Voice model type, default is 48k-hq.