Skip to navigation

Basic Inference

View as MarkdownOpen in Claude

After the NIM is running, confirm that the gRPC service is healthy, then send a video with the sample client to produce a relit output video.

Prerequisites

Install the following on the machine that runs the client. The client runs on Linux and on Windows; the steps marked Windows give the Windows form, for PowerShell. On a machine other than the NIM host (for example, Windows or Linux arm64), replace localhost and 127.0.0.1 with the NIM host’s address in the health check and in --target.

  • Python 3.12 or 3.13, with pip and venv: the grpcio 1.67.1 that the client pins has no wheel for later Python versions. The client installs into a virtual environment in step 3 below. If your distribution’s python3 is older, refer to Download Python.

  • git, curl and wget. On Ubuntu: sudo apt-get update && sudo apt-get install -y git curl wget. On Windows, install Git for Windows; curl.exe and tar ship with Windows 10 and later, and the Windows steps use them instead of wget.

  • FFmpeg, for the examples that prepare a file: the streaming and transactional examples in Input Modes, the custom HDR and background image examples in Advanced Usage, and the custom HDR cell of the sample notebook. FFmpeg runs on your machine to prepare a video or an image; it is not part of the NIM. Check with ffmpeg -version. On Ubuntu, install it with the following command:

    sudo apt-get update && sudo apt-get install -y ffmpeg
  • Jupyter, only if you run the sample notebook. Refer to Running Inference via the Notebook.

Verify the Service and Install the Client

Complete the following steps before you send an inference request.

  1. Perform a health check on the gRPC endpoint.

    • Install grpcurl from github.com/fullstorydev/grpcurl/releases.

      Example commands to run on Ubuntu x86-64 (on arm64, use grpcurl_1.9.1_linux_arm64.deb):

      wget https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_amd64.deb
      sudo dpkg -i grpcurl_1.9.1_linux_amd64.deb

      On Windows x86-64, download the zip and unpack grpcurl.exe into the current folder:

      curl.exe -LO https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_windows_x86_64.zip
      tar -xf grpcurl_1.9.1_windows_x86_64.zip grpcurl.exe
    • Download the health checking proto:

      wget https://raw.githubusercontent.com/grpc/grpc-proto/813330824839bfdd3abc52f41807095c0de2ec19/grpc/health/v1/health.proto

      On Windows:

      curl.exe -LO https://raw.githubusercontent.com/grpc/grpc-proto/813330824839bfdd3abc52f41807095c0de2ec19/grpc/health/v1/health.proto
    • Run the health check from the directory that holds health.proto:

      grpcurl --plaintext --proto health.proto localhost:8001 grpc.health.v1.Health/Check

      On Windows, run the grpcurl.exe you unpacked:

      .\grpcurl.exe --plaintext --proto health.proto localhost:8001 grpc.health.v1.Health/Check

      If the service is ready, you get a response similar to the following:

      { "status": "SERVING" }
    • To wait for the NIM from a script, poll the HTTP readiness endpoint on port 8000 instead. It returns 503 while the NIM warms up and 200 when the NIM is ready:

      timeout 600 bash -c 'until curl -sf -o /dev/null localhost:8000/v1/health/ready; do sleep 2; done' && echo ready

      On Windows:

      $deadline = (Get-Date).AddSeconds(600)
      while ((Get-Date) -lt $deadline) {
      curl.exe -sf -o NUL http://localhost:8000/v1/health/ready
      if ($LASTEXITCODE -eq 0) { "ready"; break }
      Start-Sleep -Seconds 2
      }

    The health service reports SERVING only after the NIM has warmed up all of its models. Until then, the gRPC port does not accept connections, so the health check fails to connect instead of returning a status. Retry it until it returns SERVING before sending inference requests. If the readiness poll prints nothing after 600 seconds, check docker logs relighting-nim for the error.

    For using grpcurl with an SSL-enabled server, avoid using the --plaintext argument, and use --cacert with a CA certificate, --key with a private key, or --cert with a certificate file. For more details, refer to grpcurl --help.

  2. Download the Relighting client code by cloning the gRPC client repository (NVIDIA AI for Media NIM Clients):

    git clone https://github.com/NVIDIA-Maxine/nim-clients.git
    # Go to the 'relighting' folder
    cd nim-clients/relighting/

    Run the install and proto steps from nim-clients/relighting/. The client itself runs from the scripts folder, so the client commands on this page use paths relative to nim-clients/relighting/scripts/, and the sample assets are in ../assets/.

  3. For the Python client, create a virtual environment and install the required dependencies. The virtual environment is required on distributions whose system Python is marked externally managed (for example, Ubuntu 24.04), where a plain pip install fails with error: externally-managed-environment.

    # Install Python, pip, and venv on Linux
    sudo apt-get update && sudo apt-get install -y python3-pip python3-venv
    # python3 --version must report 3.12 or 3.13; otherwise run python3.12 -m venv .venv
    python3 -m venv .venv
    source .venv/bin/activate
    pip install -r requirements.txt

    If python3 --version reports an older version (for example, 3.10 on Ubuntu 22.04), install Python 3.12 first from python.org, or create the environment with uv: uv venv --seed --python 3.12 .venv in place of python3 -m venv .venv.

    On Windows, install Python 3.12 from python.org, then create and activate the environment with the py launcher. PowerShell blocks the activation script by default; the second line allows it for the current window only:

    py -3.12 -m venv .venv
    Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned -Force
    .venv\Scripts\Activate.ps1
    pip install -r requirements.txt

    Keep the environment active for every python command on this page.

Compile the Protos (Optional)

If you want to use the client code provided in the NIM Clients GitHub repository (NVIDIA AI for Media NIM Clients), you can skip this step.

The proto files are available in the relighting/protos folder. You can compile them to generate client interfaces in your preferred programming language. For more details, refer to Supported languages in the gRPC documentation.

The following example shows how to compile the protos for Python on Linux and Windows.

Python

The grpcio version needed for compilation can be referred from requirements.txt

To compile protos on Linux, run the following commands:

# From nim-clients/relighting/, go to the protos/linux/ folder
cd protos/linux/
chmod +x compile_protos.sh
./compile_protos.sh
# Return to nim-clients/relighting/
cd ../..

The script writes the Python stubs to interfaces/, replacing the ones that ship with the client. On failure it prints a line that starts with [Error] and exits with status 1.

To compile protos on Windows, run the following commands:

# From nim-clients\relighting\, go to the protos\windows\ folder
cd protos\windows
.\compile_protos.bat
# Return to nim-clients\relighting\
cd ..\..

Running Inference via Python Script

You can use the sample client script in the NVIDIA AI for Media NIM Clients repo to send a gRPC request to the hosted NIM server:

  1. Go to the Python scripts directory. From nim-clients/relighting/:

    cd scripts
  2. Run the command to send a gRPC request. All command-line parameters are optional; this example sets the most common ones. For the SSL options, refer to SSL Enablement.

    python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --output output.mp4 --hdri-id 0

    The NIM selects streaming or transactional mode from the input: a streamable video (metadata at the start of the file) is processed in streaming mode, which returns output before the upload ends. The same command works for both. For more information, refer to Input Modes.

    The client writes the output to <output>.part while the video streams back and renames it to <output> only when the request succeeds. A failed or interrupted run leaves an existing <output> unchanged; a .part file left behind after the client was killed can be deleted.

Usage for Preview API Request

To send a request to the preview NIM hosted on build.nvidia.com instead of a local NIM, use the --preview-mode flag with your NGC API key and the function ID:

python relighting.py --preview-mode \
--target grpc.nvcf.nvidia.com:443 \
--function-id {function_id} \
--api-key "$NGC_API_KEY" \
--video-input ../assets/sample_video.mp4 \
--output output.mp4
  • --preview-mode - Flag to send the request to the preview NVCF-hosted NIM on build.nvidia.com.
  • --api-key - NGC API key required for authentication (the example uses the NGC_API_KEY that you exported in Getting Started). Utilized when using the preview API, ignored otherwise.
  • --function-id - NVCF function ID for the service. Utilized when using the preview API, ignored otherwise.

To find the function ID, open the API page and sign in. The Send gRPC request example on that page shows the command with the function ID filled in as the value of --function-id. Replace {function_id} in the command above with that value.

To view details of command-line arguments, run the following command:

python relighting.py -h

You get a response similar to the following:

usage: relighting.py [-h] [--ssl-mode {DISABLED,MTLS,TLS}] [--ssl-key SSL_KEY] [--ssl-cert SSL_CERT] [--ssl-root-cert SSL_ROOT_CERT] [--target TARGET]
[--preview-mode] [--api-key API_KEY] [--function-id FUNCTION_ID] [--video-input VIDEO_INPUT] [--output OUTPUT]
[--video-bitrate VIDEO_BITRATE] [--idr-interval IDR_INTERVAL] [--lossless] [--custom-encoding-params CUSTOM_ENCODING_PARAMS]
[--hdr HDR_FILE] [--hdri-id {0,1,2,3,4}] [--pan PAN] [--vertical-fov VERTICAL_FOV] [--autorotate] [--rotation-rate ROTATION_RATE]
[--background-source {0,1,2}] [--background-image BACKGROUND_IMAGE] [--background-color BACKGROUND_COLOR]
[--foreground-gain FOREGROUND_GAIN] [--background-gain BACKGROUND_GAIN]
Run Relighting with all available features
options:
-h, --help show this help message and exit
--ssl-mode {DISABLED,MTLS,TLS} Flag to set SSL mode, default is DISABLED (default: DISABLED)
--ssl-key SSL_KEY The path to ssl private key. (default: ../ssl_key/ssl_key_client.pem)
--ssl-cert SSL_CERT The path to ssl certificate chain. (default: ../ssl_key/ssl_cert_client.pem)
--ssl-root-cert SSL_ROOT_CERT The path to ssl root certificate. (default: ../ssl_key/ssl_ca_cert.pem)
--target TARGET IP:port of gRPC service, when hosted locally. Use grpc.nvcf.nvidia.com:443 when hosted on NVCF.
(default: 127.0.0.1:8001)
--preview-mode Flag to send request to the preview NVCF-hosted NIM on build.nvidia.com. (default: False)
--api-key API_KEY NGC API key required for authentication, utilized when using TRY API ignored otherwise (default:
None)
--function-id FUNCTION_ID NVCF function ID for the service, utilized when using TRY API ignored otherwise (default: None)
Input/Output:
--video-input VIDEO_INPUT Path to the input video file. (default: /path/to/nim-clients/relighting/assets/sample_video.mp4)
--output OUTPUT Path for the output video file. (default: None)
--video-bitrate VIDEO_BITRATE, --bitrate VIDEO_BITRATE Output video bitrate in bps, at most 200000000 (200 Mbps). 0 = auto (server chooses from output
resolution: 10/15/25 Mbps). (default: 10000000)
--idr-interval IDR_INTERVAL IDR interval in frames, >= 1 (server auto ~2 s GOP if omitted in API). (default: 8)
--lossless Enable lossless video encoding. (default: False)
--custom-encoding-params CUSTOM_ENCODING_PARAMS Custom encoding parameters as JSON. (default: None)
HDR/Illumination:
--hdr HDR_FILE Custom .hdr file. (default: None)
--hdri-id {0,1,2,3,4} HDR preset: 0=Lounge, 1=Cobblestone Street Night, 2=Glasshouse Interior, 3=Little Paris Eiffel
Tower, 4=Wooden Studio. (default: 0)
--pan PAN Pan angle (degrees, -180 to 180). (default: -90.0)
--vertical-fov VERTICAL_FOV, --vfov VERTICAL_FOV Vertical FOV (degrees, 5.73-180). Applies only with --background-source 2 (HDR projection).
(default: 60.0)
Autorotate:
--autorotate Auto-rotate the HDR environment. (default: False)
--rotation-rate ROTATION_RATE Rotation speed (degrees/s, 0-360). Requires --autorotate. (default: 20.0)
Background:
--background-source {0,1,2} 0=source video, 1=custom image, 2=HDR projection. (default: 0)
--background-image BACKGROUND_IMAGE Custom background image (PNG, JPG, or HDR). Requires --background-source 1. (default: None)
--background-color BACKGROUND_COLOR Solid color as an integer, hex '0xRRGGBB' (e.g. 0x808080) or decimal. Requires --background-source
1; used only when no --background-image is given. (default: None)
Effect Parameters:
--foreground-gain FOREGROUND_GAIN Foreground relighting strength (0.0-2.0). (default: 1.0)
--background-gain BACKGROUND_GAIN Background relighting strength (0.0-2.0). Applies only with --background-source 2 (HDR projection).
(default: 1.0)

Python 3.13 prints an option that has an alias in a shorter form. For example, it shows --video-bitrate, --bitrate VIDEO_BITRATE where Python 3.12 shows --video-bitrate VIDEO_BITRATE, --bitrate VIDEO_BITRATE. The options are the same.

To get more information about how to run the client with these parameters, refer to the NVIDIA AI for Media NIM Clients GitHub repository: NVIDIA AI for Media NIM Clients.

For more information about configuring advanced parameters such as bitrate, IDR interval, HDR settings, and background options, see Advanced Usage.

For command-line arguments that aren’t specified, the script uses the following default values.

Default Command-Line Arguments

ArgumentDefault Value
target127.0.0.1:8001
preview-modeFalse
api-keyNone
function-idNone
video-inputnim-clients/relighting/assets/sample_video.mp4
outputNone (auto: <input_stem>-relighting_output.mp4 in the current directory)
video-bitrate10000000
idr-interval8
losslessFalse
custom-encoding-paramsNone
hdrNone
hdri-id0 (Lounge)
pan-90.0
vertical-fov60.0
autorotateFalse
rotation-rate20.0
background-source0 (source video)
background-imageNone
background-colorNone (not sent; the NIM uses 0x808080, gray)
foreground-gain1.0
background-gain1.0
ssl-modeDISABLED
ssl-key../ssl_key/ssl_key_client.pem
ssl-cert../ssl_key/ssl_cert_client.pem
ssl-root-cert../ssl_key/ssl_ca_cert.pem

Example Commands

  • Basic inference with default arguments:

    python relighting.py --target 127.0.0.1:8001
  • Run inference on your own video with an HDR preset. Replace {your_video_file} with the path to an MP4 file:

    python relighting.py --target 127.0.0.1:8001 --video-input {your_video_file} --hdri-id 3

For a custom HDR environment map, a background image, and the output encoding options, refer to Advanced Usage.

Supported Formats

The NIM accepts MP4 input with H.264, H.265 (HEVC), or VP9 video. The examples on this page use H.264. The server also accepts WebM input, but the sample Python client sends only .mp4 files.

For the resolution and frame-rate rules, refer to Input Media.

The NIM copies the audio track of the input to the output without re-encoding it. If the input has no audio track, the output has no audio track.

Input Modes

Relighting provides two modes for processing input files: streaming and transactional.

AspectStreaming ModeTransactional Mode
Data StorageOnly frames being processed are temporarily copied in memory.Entire video and audio files are temporarily copied on disk.
Processing StartNIM starts processing as soon as data chunk for first frame arrives.NIM waits to receive entire files before starting.
Processing TimingContinuous processing without waiting for complete input.Processing begins after all data is received.
Output DeliveryOutput frames are generated and returned immediately.Complete output video is returned to client after inference is finished for whole video.

Streaming Mode

Streaming mode is the recommended way to use Relighting. It allows inference to begin without receiving the whole video from the client. It processes video frames incrementally, and inference begins as soon as the first frame of information is available. The output frames are streamed back to the client immediately after inference.

The NIM automatically detects streamable videos and enables streaming mode. In this mode, the NIM starts returning output frames before the whole input is uploaded.

Use streaming mode for the following use cases:

  • Streamable video inputs.
  • Applications that benefit from receiving output as it is generated, without waiting for the entire file to be uploaded.
  • Large video files that benefit from incremental processing and reduced disk I/O.

Streaming mode works with streamable videos in which metadata is positioned at the beginning of the file. The NIM automatically detects streamable videos and enables streaming mode. Videos that are not streamable can be easily converted to a streamable format.

To convert your video into a streamable video, use FFmpeg with the -movflags +faststart option. For example, from nim-clients/relighting/scripts/:

ffmpeg -y -i ../assets/sample_video.mp4 -movflags +faststart -c copy ../assets/sample_video_streamable.mp4

You can then specify the streamable video as input to the NIM by using the --video-input parameter:

python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video_streamable.mp4

Transactional Mode

In transactional mode, the entire video and audio files have to be received by the NIM before processing can begin. The NIM falls back to this mode when the input video is not streamable.

Transactional mode is suitable for the following use cases:

  • Videos that are not optimized for streaming (such as non-streamable MP4 files in which metadata is located at the end of the file, requiring the entire file to be downloaded before playback can begin).
  • Processing of small video and audio files for which streaming overhead is unnecessary.
  • Applications that can wait for complete processing before receiving output.

To run Relighting in transactional mode, provide a non-streamable video as input. The sample video is already streamable; to create a non-streamable copy, remux it without +faststart (FFmpeg then writes the metadata at the end of the file):

ffmpeg -y -i ../assets/sample_video.mp4 -c copy ../assets/sample_video_transactional.mp4
python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video_transactional.mp4

To receive output while the upload is still running, convert your videos to a streamable format so that the NIM can use streaming mode.

Errors

The NIM reports a failed request with a gRPC status and a message in details. The sample client prints it as ERROR: gRPC error: StatusCode.{status} - {details} and exits 1. The client checks its own arguments before it connects and prints a problem there as Configuration error: {message}, also exiting 1. Those checks include the option ranges and an empty input file, so a sample-client user sees Configuration error for them; the matching rows below are what a direct gRPC caller receives. An unknown or malformed option, such as --blur, which earlier clients accepted, is rejected by argument parsing, which prints a usage: line and an error: line and exits 2.

gRPC statusCauseMessage (details)What to do
INVALID_ARGUMENTA configuration value is out of range, a custom encoding key is unknown, a custom encoding value is null, a list, or an object, or a custom encoding value conflicts with num-B-FramesInvalid RelightConfig: followed by one line per problem, for example - foreground_gain: must be in [0.0, 2.0], got 3.0 or - output_video_encoding.custom_encoding.custom['bitrate']: must be a number, string or booleanCorrect the named field.
INVALID_ARGUMENTThe first request does not carry a RelightConfigFirst request must include RelightConfigSend the configuration first.
INVALID_ARGUMENTThe input video is emptyNo video data received: the input video is empty.Send a video file that is not empty.
INVALID_ARGUMENTThe input is not a usable video: not MP4 or WebM, no video track, an unsupported codec, a chroma format or bit depth the GPU’s video decoder does not support, a resolution outside its range, or frame timestamps that repeat or go backwardsThe decoder’s reason, for example No decodable video stream found in the input. The file may be truncated or the video track may be missing.Convert the input to a supported format (refer to Input Media).
INVALID_ARGUMENTThe input video is less than 512 pixels wide or less than 288 pixels highThe input video is {width}x{height} pixels; Relighting needs at least 512 x 288 pixels (width x height).Send a video that is at least 512 x 288 pixels (refer to Input Media).
INVALID_ARGUMENTThe custom HDR environment map is not an .hdr file, or the background image is not an .hdr, PNG, or JPEG fileInvalid RelightConfig: followed by - HDRI image: unsupported format 'unknown', must be HDR (Radiance .hdr) or - Background image: unsupported format 'unknown', must be HDR, PNG, or JPEGSend a file of a supported format.
INVALID_ARGUMENTThe custom HDR environment map or background image is larger than 150 MBInvalid RelightConfig: followed by - HDRI image: must be at most 150000000 bytes or - Background image: must be at most 150000000 bytesSend a smaller image.
FAILED_PRECONDITIONThe custom HDR environment map or background image has a supported format but does not decode, for example a truncated fileHDRI image is corrupted or in an unsupported format or Background image is corrupted or in an unsupported formatSend a valid .hdr, PNG, or JPEG file.
FAILED_PRECONDITIONThe custom HDR environment map or background image is wider than 8192 pixels or taller than 4096 pixelsHDRI image: the image is {width}x{height} pixels, larger than 8192 x 4096, or the same with Background image:Resize the image to 8192 x 4096 pixels or less.
FAILED_PRECONDITIONThe custom HDR environment map is not twice as wide as it is tall (a map that is also larger than 8192 x 4096 pixels gets the size error in the row above first)HDRI image: the image is {width}x{height} pixels; an HDR environment map must be equirectangular, twice as wide as it is tallUse an equirectangular map, for example 4096 x 2048 pixels.
RESOURCE_EXHAUSTEDEvery request slot stayed busy for 3 secondsServer is at capacity. Please retry shortly (active request limit reached).Retry after the running request finishes (refer to Multiple Concurrent Inputs).
RESOURCE_EXHAUSTEDThe GPU ran out of memory while it decoded the inputThe GPU ran out of memory while decoding the input video. Retry with fewer concurrent requests or a lower input resolution.Retry with fewer concurrent requests or a lower input resolution.
INTERNALAn unexpected server errorException iterating requests: {type}: {message}Check the server log with docker logs relighting-nim.
UNAVAILABLEA model failed to load or warm upRelighting service is not ready: a model failed to load or warm up (see server logs)Check the server log with docker logs relighting-nim and restart the NIM.
UNAVAILABLEThe NIM stopped while the request ran, for example after docker stopThe server is shutting down. Retry the request., or Socket closed or Cancelling all calls from the gRPC client library when the container exits firstRetry the request when the NIM is running again.
UNAVAILABLEThe NIM is not running, or its gRPC port does not accept connections yet; or --ssl-mode does not match the server’s NIM_SSL_MODERaised by the gRPC client libraryWait until the health check returns SERVING. If the NIM is running, set --ssl-mode to match NIM_SSL_MODE; refer to SSL Enablement.
CANCELLEDThe client opened the stream and sent no request for 30 secondsCANCELLEDSend the configuration as soon as the stream opens.

Running Inference via the Notebook

The client folder also contains a Jupyter notebook, notebook/relighting_notebook.ipynb. It walks through the HDR, background, effect, and encoding options with the sample video.

To run it, install Jupyter into the virtual environment from step 3 and open the notebook from nim-clients/relighting/, with the virtual environment active:

cd .. # from scripts/ back to nim-clients/relighting/
pip install notebook
jupyter notebook notebook/relighting_notebook.ipynb

Run the cells in order. A full run sends 20 requests, takes about 4 minutes on one RTX 6000 Ada GPU, and writes 20 MP4 files to notebook/. A request that fails stops its cell with the error. The notebook sends requests to localhost:8001. To use another server, change SERVICE_HOST and SERVICE_PORT in its configuration cell.