Skip to navigation

Support Matrix

View as MarkdownOpen in Claude

Models

Relighting uses the following models.

Model NameModel IDPublisher
RelightingrelightingNVIDIA

Optimized Configurations

Relighting has optimized configurations for the following server GPUs.

Server GPUPrecision
A10G, A40FP16
L4, L40SFP16
RTX PRO 6000 Blackwell Server Edition, RTX PRO 4500 Blackwell Server EditionFP16

Other Architectures (Consumer and Desktop GPUs)

Relighting also runs on the following consumer and desktop GPUs.

Consumer GPUPrecision
RTX A6000FP16
RTX 6000 Ada Generation, RTX 4090FP16
RTX PRO 6000 Blackwell Workstation Edition, RTX 5090FP16

Relighting is compatible with professional, datacenter, and consumer GPUs that have Tensor cores and are based on the NVIDIA Blackwell, Ada, or Ampere architecture. The GPU must have NVENC. The NIM encodes its output on the GPU’s video encoder, so a GPU without NVENC, such as the A100, is not supported. The GPU needs more than 17 GB of memory, because the NIM uses about 17 GB after its models are loaded. Refer to GPU Memory for the figures.

Relighting requires NVENC/NVDEC hardware for video encoding/decoding. For details about supported GPUs and H264 YUV formats, refer to the Video Encode and Decode Support Matrix.

GPU Memory

The NIM uses about 17 GB of GPU memory once its models are loaded and it has served its first request. Each request adds about 0.3 GB for 1080p input and about 0.7 GB for 4K input, the same with eight concurrent requests. On a GPU with 24 GB of memory, two concurrent 4K requests complete with every frame: in NVIDIA internal benchmark runs on the 4K clip of Performance Results, each request ran at 14.7 fps on the A10G and 11.6 fps on the L4. More than two concurrent 4K requests were not benchmarked on these GPUs (refer to Multiple Concurrent Inputs).

On the RTX 6000 Ada Generation GPU used for these measurements, eight concurrent requests peaked at about 19.4 GB in total for 1080p input and 22.6 GB for 4K input. The figures come from the sample video (1080p) and a 4K H.264 clip, on a freshly started NIM, as the peak nvidia-smi memory use during the requests minus the use with the NIM idle after its first request, sampled every 100 ms, in GB (1 GB = 10^9 bytes). They vary slightly by GPU architecture and by input clip.

Host Resource Requirements

Approximate figures for planning, measured with one request at a time on a host with an RTX 6000 Ada Generation GPU and 12 CPU threads, as docker stats reports the container’s memory use (MEM USAGE) and CPU. docker stats shows memory in GiB. The following figures are approximate and are for planning (1 GB = 10^9 bytes). Concurrent requests use more host memory and CPU.

ResourceRequirement
Container imageAbout 5.8 GB to download, 9.3 GB on disk
Model artifactsAbout 0.1 GB, downloaded on first start into the cache directory. Mount a persistent cache to avoid downloading them on every launch; refer to Model Caching.
Host memoryAbout 1.1 to 1.3 GB after the NIM starts. It grows during the first requests and levels off within about 40 requests, at about 1.9 to 2.0 GB with the NIM idle and up to about 2.1 GB during a 1080p or 4K request.
CPUAbout 1.3 CPU cores during a request
DiskSpace for the image, the model cache, and your input and output media
Shared memory--shm-size=8GB, as in every launch command

Input Media

Relighting accepts the following input media.

ContainerMP4. The NIM also accepts WebM; the sample client sends only .mp4 files.
Video codecH.264, H.265 (HEVC), VP9
ResolutionAt least 512 pixels wide and 288 pixels high, the minimum of the AI Green Screen segmentation that Relighting uses, and within the range that the GPU’s video decoder (NVDEC) supports for the codec. A smaller video is rejected with INVALID_ARGUMENT. A video outside the NVDEC range is also rejected with INVALID_ARGUMENT, and the message gives the supported minimum and maximum. If the resolution changes during the video, the output keeps the resolution of the first frame: later frames are scaled to it, and stretched if their aspect ratio differs. Re-encode such a video at one resolution to avoid this.
Pixel formatYUV 4:2:0, 8-bit or 10-bit, as NVDEC supports for the codec. The output is 8-bit YUV 4:2:0. A chroma format or bit depth that NVDEC does not decode, such as H.264 4:4:4, is rejected with INVALID_ARGUMENT.
Frame rateConstant or variable. The output keeps the frame timestamps of the input, so each frame must be later than the one before. A video whose timestamps repeat or go backwards, such as two files joined with ffmpeg -c copy, is rejected with INVALID_ARGUMENT.
TracksOne video track and at most one audio track. Other tracks, such as metadata or subtitles, are ignored. To keep only the first video track and the first audio track of an input, use for example ffmpeg -i input.mp4 -map 0:v:0 -map '0:a:0?' -c copy output.mp4.
AudioCopied to the output without re-encoding.
File sizeNo maximum. A streamable video is processed as it arrives. A video that is not streamable is stored in full in the container before processing starts, and so is its output: the container needs free disk space for both (refer to Input Modes).

Software

The Relighting NIM requires the following host software, NVIDIA platform components, and disk and memory resources.

NVIDIA Driver and Prerequisites

NVIDIA driver requirements and other prerequisites for Relighting. The NIM runs on Linux x86-64 (amd64) hosts.

PrerequisiteVersionDownload and install steps
NVIDIA Graphic Drivers for Linux570.26+https://www.nvidia.com/en-us/drivers/unix/
DockerlatestUbuntu, CentOS, and Debian: https://docs.docker.com/engine/install/; Rocky Linux: https://docs.rockylinux.org/gemstones/containers/docker/
NVIDIA Container ToolkitlatestInstallation and configuration instructions: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
acl package, provides setfacl (SSL and mTLS only)AnyYour distribution’s package manager, for example sudo apt install acl. Needed only by the SSL Enablement recipe.

Relighting uses the following NVIDIA software platforms:

ComponentsVersion
CUDA12.8.1
cuDNN9.8.0.87
TRT10.9.0.34
Triton Inference Serverv2.56.0
DeepStream8.0