Advanced Usage
This section provides a detailed breakdown of the inferencing script for more advanced users.
Studio Voice NIM uses gRPC endpoints. Import the compiled gRPC protos for invoking the NIM.
The NIM invocation uses bi-directional gRPC streaming. To generate the request data stream, define a Python generator function. This is also known as a Python iterator of form, a simple function that yields after a call. The yield returns a chunk to be streamed.
For NIM in streaming mode, the audio_stream_data in request should be PCM 32-bit float audio data with each data chunk of size 10ms for low latency model and 6 secs for high quality models.
Before invoking the NIM, define a function that handles the incoming stream and writes it to an output file.
For NIM in streaming mode, the output audio_stream_data in response is PCM 32-bit float audio data of same length as input or request.
Now that we have the request generator and output iterator setup, connect to the NIM and invoke it.
The input file path is stored in the variable input_filepath and the output file is written to the location specified in the variable output_filepath.
Wait for a message confirming that the function invocation has completed before checking the output file.
Fill in the correct host and port for your target in the code snippet below:
Compile the Protos (Optional)
The NVIDIA Maxine NIM Clients package comes with the pre-compiled protos. However, to compile the protos locally, install the required dependencies.
The compilation script enables developers to generate compiled protos that match their specific programming language (such as Python or C++) and version requirements, ensuring compatibility between client applications and compiled protos in their custom implementations.
Linux
To compile protos on Linux, run:
Windows
To compile protos on Windows, run:
Model Caching
When the container starts for the first time, it will download the required models from NGC. To avoid downloading the models on subsequent runs, you can cache them locally by using a cache directory:
docker
podman
Ensure the nim_model_profile is compatible with your GPU. For more information about nim_model_profile, refer to the NIM Model Profile Table.