Basic Inference
After the NIM is running, confirm that the gRPC service is healthy, then send a video with the sample client to produce a relit output video.
Prerequisites
Install the following on the machine that runs the client. The client runs on Linux and on Windows; the steps marked Windows give the Windows form, for PowerShell. On a machine other than the NIM host (for example, Windows or Linux arm64), replace localhost and 127.0.0.1 with the NIM host’s address in the health check and in --target.
-
Python 3.12 or 3.13, with
pipandvenv: thegrpcio1.67.1 that the client pins has no wheel for later Python versions. The client installs into a virtual environment in step 3 below. If your distribution’spython3is older, refer to Download Python. -
git,curlandwget. On Ubuntu:sudo apt-get update && sudo apt-get install -y git curl wget. On Windows, install Git for Windows;curl.exeandtarship with Windows 10 and later, and the Windows steps use them instead ofwget. -
FFmpeg, for the examples that prepare a file: the streaming and transactional examples in Input Modes, the custom HDR and background image examples in Advanced Usage, and the custom HDR cell of the sample notebook. FFmpeg runs on your machine to prepare a video or an image; it is not part of the NIM. Check with
ffmpeg -version. On Ubuntu, install it with the following command: -
Jupyter, only if you run the sample notebook. Refer to Running Inference via the Notebook.
Verify the Service and Install the Client
Complete the following steps before you send an inference request.
-
Perform a health check on the gRPC endpoint.
-
Install
grpcurlfrom github.com/fullstorydev/grpcurl/releases.Example commands to run on Ubuntu x86-64 (on arm64, use
grpcurl_1.9.1_linux_arm64.deb):On Windows x86-64, download the zip and unpack
grpcurl.exeinto the current folder: -
Download the health checking proto:
On Windows:
-
Run the health check from the directory that holds
health.proto:On Windows, run the
grpcurl.exeyou unpacked:If the service is ready, you get a response similar to the following:
-
To wait for the NIM from a script, poll the HTTP readiness endpoint on port 8000 instead. It returns
503while the NIM warms up and200when the NIM is ready:On Windows:
The health service reports
SERVINGonly after the NIM has warmed up all of its models. Until then, the gRPC port does not accept connections, so the health check fails to connect instead of returning a status. Retry it until it returnsSERVINGbefore sending inference requests. If the readiness poll prints nothing after 600 seconds, checkdocker logs relighting-nimfor the error.For using grpcurl with an SSL-enabled server, avoid using the
--plaintextargument, and use--cacertwith a CA certificate,--keywith a private key, or--certwith a certificate file. For more details, refer togrpcurl --help. -
-
Download the Relighting client code by cloning the gRPC client repository (NVIDIA AI for Media NIM Clients):
Run the install and proto steps from
nim-clients/relighting/. The client itself runs from thescriptsfolder, so the client commands on this page use paths relative tonim-clients/relighting/scripts/, and the sample assets are in../assets/. -
For the Python client, create a virtual environment and install the required dependencies. The virtual environment is required on distributions whose system Python is marked externally managed (for example, Ubuntu 24.04), where a plain
pip installfails witherror: externally-managed-environment.If
python3 --versionreports an older version (for example, 3.10 on Ubuntu 22.04), install Python 3.12 first from python.org, or create the environment with uv:uv venv --seed --python 3.12 .venvin place ofpython3 -m venv .venv.On Windows, install Python 3.12 from python.org, then create and activate the environment with the
pylauncher. PowerShell blocks the activation script by default; the second line allows it for the current window only:Keep the environment active for every
pythoncommand on this page.
Compile the Protos (Optional)
If you want to use the client code provided in the NIM Clients GitHub repository (NVIDIA AI for Media NIM Clients), you can skip this step.
The proto files are available in the relighting/protos folder. You can compile them to generate client interfaces in your preferred programming language. For more details, refer to Supported languages in the gRPC documentation.
The following example shows how to compile the protos for Python on Linux and Windows.
Python
The grpcio version needed for compilation can be referred from requirements.txt
To compile protos on Linux, run the following commands:
The script writes the Python stubs to interfaces/, replacing the ones that ship with the client. On failure it prints a line that starts with [Error] and exits with status 1.
To compile protos on Windows, run the following commands:
Running Inference via Python Script
You can use the sample client script in the NVIDIA AI for Media NIM Clients repo to send a gRPC request to the hosted NIM server:
-
Go to the Python scripts directory. From
nim-clients/relighting/: -
Run the command to send a gRPC request. All command-line parameters are optional; this example sets the most common ones. For the SSL options, refer to SSL Enablement.
The NIM selects streaming or transactional mode from the input: a streamable video (metadata at the start of the file) is processed in streaming mode, which returns output before the upload ends. The same command works for both. For more information, refer to Input Modes.
The client writes the output to
<output>.partwhile the video streams back and renames it to<output>only when the request succeeds. A failed or interrupted run leaves an existing<output>unchanged; a.partfile left behind after the client was killed can be deleted.
Usage for Preview API Request
To send a request to the preview NIM hosted on build.nvidia.com instead of a local NIM, use the --preview-mode flag with your NGC API key and the function ID:
--preview-mode- Flag to send the request to the preview NVCF-hosted NIM on build.nvidia.com.--api-key- NGC API key required for authentication (the example uses theNGC_API_KEYthat you exported in Getting Started). Utilized when using the preview API, ignored otherwise.--function-id- NVCF function ID for the service. Utilized when using the preview API, ignored otherwise.
To find the function ID, open the API page and sign in. The Send gRPC request example on that page shows the command with the function ID filled in as the value of --function-id. Replace {function_id} in the command above with that value.
To view details of command-line arguments, run the following command:
You get a response similar to the following:
Python 3.13 prints an option that has an alias in a shorter form. For example, it shows --video-bitrate, --bitrate VIDEO_BITRATE where Python 3.12 shows --video-bitrate VIDEO_BITRATE, --bitrate VIDEO_BITRATE. The options are the same.
To get more information about how to run the client with these parameters, refer to the NVIDIA AI for Media NIM Clients GitHub repository: NVIDIA AI for Media NIM Clients.
For more information about configuring advanced parameters such as bitrate, IDR interval, HDR settings, and background options, see Advanced Usage.
For command-line arguments that aren’t specified, the script uses the following default values.
Default Command-Line Arguments
Example Commands
-
Basic inference with default arguments:
-
Run inference on your own video with an HDR preset. Replace
{your_video_file}with the path to an MP4 file:
For a custom HDR environment map, a background image, and the output encoding options, refer to Advanced Usage.
Supported Formats
The NIM accepts MP4 input with H.264, H.265 (HEVC), or VP9 video. The examples on this page use H.264. The server also accepts WebM input, but the sample Python client sends only .mp4 files.
For the resolution and frame-rate rules, refer to Input Media.
The NIM copies the audio track of the input to the output without re-encoding it. If the input has no audio track, the output has no audio track.
Input Modes
Relighting provides two modes for processing input files: streaming and transactional.
Streaming Mode
Streaming mode is the recommended way to use Relighting. It allows inference to begin without receiving the whole video from the client. It processes video frames incrementally, and inference begins as soon as the first frame of information is available. The output frames are streamed back to the client immediately after inference.
The NIM automatically detects streamable videos and enables streaming mode. In this mode, the NIM starts returning output frames before the whole input is uploaded.
Use streaming mode for the following use cases:
- Streamable video inputs.
- Applications that benefit from receiving output as it is generated, without waiting for the entire file to be uploaded.
- Large video files that benefit from incremental processing and reduced disk I/O.
Streaming mode works with streamable videos in which metadata is positioned at the beginning of the file. The NIM automatically detects streamable videos and enables streaming mode. Videos that are not streamable can be easily converted to a streamable format.
To convert your video into a streamable video, use FFmpeg with the -movflags +faststart option. For example, from nim-clients/relighting/scripts/:
You can then specify the streamable video as input to the NIM by using the --video-input parameter:
Transactional Mode
In transactional mode, the entire video and audio files have to be received by the NIM before processing can begin. The NIM falls back to this mode when the input video is not streamable.
Transactional mode is suitable for the following use cases:
- Videos that are not optimized for streaming (such as non-streamable MP4 files in which metadata is located at the end of the file, requiring the entire file to be downloaded before playback can begin).
- Processing of small video and audio files for which streaming overhead is unnecessary.
- Applications that can wait for complete processing before receiving output.
To run Relighting in transactional mode, provide a non-streamable video as input. The sample video is already streamable; to create a non-streamable copy, remux it without +faststart (FFmpeg then writes the metadata at the end of the file):
To receive output while the upload is still running, convert your videos to a streamable format so that the NIM can use streaming mode.
Errors
The NIM reports a failed request with a gRPC status and a message in details. The sample client prints it as ERROR: gRPC error: StatusCode.{status} - {details} and exits 1. The client checks its own arguments before it connects and prints a problem there as Configuration error: {message}, also exiting 1. Those checks include the option ranges and an empty input file, so a sample-client user sees Configuration error for them; the matching rows below are what a direct gRPC caller receives. An unknown or malformed option, such as --blur, which earlier clients accepted, is rejected by argument parsing, which prints a usage: line and an error: line and exits 2.
Running Inference via the Notebook
The client folder also contains a Jupyter notebook, notebook/relighting_notebook.ipynb. It walks through the HDR, background, effect, and encoding options with the sample video.
To run it, install Jupyter into the virtual environment from step 3 and open the notebook from nim-clients/relighting/, with the virtual environment active:
Run the cells in order. A full run sends 20 requests, takes about 4 minutes on one RTX 6000 Ada GPU, and writes 20 MP4 files to notebook/. A request that fails stops its cell with the error. The notebook sends requests to localhost:8001. To use another server, change SERVICE_HOST and SERVICE_PORT in its configuration cell.