Advanced Usage#

Model Caching#

When the container launches for the first time, it downloads the required models from NGC. To avoid downloading the models on subsequent runs, you can cache them locally by using a cache directory:

# Create the cache directory on the host machine.
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
chmod 777 $LOCAL_NIM_CACHE

# Choose manifest profile ID based on target architecture.
export MANIFEST_PROFILE_ID=<enter_valid_manifest_profile_id>

# Run the container with the cache directory mounted in the appropriate location.
docker run -it --rm --name=relighting-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=$MANIFEST_PROFILE_ID \
  -e NIM_HTTP_API_PORT=8000 \
  -p 8000:8000 \
  -p 8001:8001 \
  -p 9002:9002 \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  nvcr.io/nim/nvidia/ai4m-relighting-nim:1.1.0

For more information about MANIFEST_PROFILE_ID, refer to Model Manifest Profiles.

SSL enablement#

Relighting NIM provides an SSL mode to ensure secure communication between clients and the server by encrypting data in transit. To enable SSL, you must provide the path to the SSL certificate and key files in the container. The following example shows how to do this:

export NGC_API_KEY=<add-your-api-key>
SSL_CERT=path/to/ssl_key

docker run -it --rm --name=relighting-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -v $SSL_CERT:/opt/nim/crt/:ro \
  -e NGC_API_KEY=$NGC_API_KEY \
  -p 8000:8000 \
  -p 8001:8001\
  -e NIM_SSL_MODE="mtls" \
  -e NIM_SSL_CERT_PATH="/opt/nim/crt/ssl_cert_server.pem" \
  -e NIM_SSL_KEY_PATH="/opt/nim/crt/ssl_key_server.pem" \
  nvcr.io/nim/nvidia/ai4m-relighting-nim:1.1.0

NIM_SSL_MODE can be set to mtls, tls, or disabled. If set to mtls, the container uses mutual TLS authentication. If set to tls, the container uses TLS authentication. For more information, refer to Environment Variables.

Be sure to verify the permissions of the SSL certificate and key files on the host machine. The container cannot access the files if they are not readable by the user running the container.

NIM Service Configuration Parameters via Client#

Use the following options to configure each request to the Relighting NIM.

HDR/Illumination Options#

hdri_preset_id: Specifies a built-in HDR preset to create one of several lighting moods. Mutually exclusive with hdri_image_provided.
- 0 (Lounge): Lounge environment.
- 1 (Cobblestone Street Night): Cobblestone street at night.
- 2 (Glasshouse Interior): Glasshouse interior.
- 3 (Little Paris Eiffel Tower): Little Paris Eiffel Tower.
- 4 (Wooden Studio): Wooden studio environment.
The following example uses the sample Python client with the Little Paris Eiffel Tower HDR preset:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --hdri-id 3
```
hdri_image_provided: When set to true, indicates that a custom HDR environment map is sent inline as ImageData. Custom HDR images take priority over preset IDs.

The following example uses the sample Python client with a custom HDR file:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --hdr /path/to/custom.hdr
```
angle_pan_radians: Pan angle for rotating the HDR environment (in radians via API, degrees via CLI). Default is -90 degrees.
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --pan -45
```
angle_v_fov_radians: Vertical field of view for HDR projection (in radians via API, degrees via CLI). Default is 60 degrees.
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --vfov 90
```

Autorotate Options#

autorotate: Enable automatic rotation of the HDR environment during video playback for dynamic lighting effects.
rotation_rate: Speed of rotation in degrees per second (default: 20). Only applies when autorotate is enabled.

The following example enables autorotate with a custom rotation speed:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --autorotate --rotation-rate 45
```

Background Source Options#

background_source: Selects which background source to use. Combine with blur_strength for blurred variants.
- 0 (UNSPECIFIED): Use source video frame as background (default).
- 1 (FROM_IMAGE): Use custom background image uploaded via ImageData.
- 2 (FROM_HDR): Use HDR environment map as background.
The following example uses the sample Python client with blurred source background:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --background-source 0 --blur 0.7
```
background_image_type: When using background_source = 1 (FROM_IMAGE), a custom background image is sent inline as ImageData with IMAGE_TYPE_BACKGROUND. Supports PNG, JPG, and HDR formats.
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --background-source 1 --background-image /path/to/background.png
```
background_color: Solid background color specified as ‘gray’ or hex value like ‘0x808080’. Used when background_source is FROM_IMAGE and no image is uploaded.

Effect Parameters#

foreground_gain: Controls the strength of relighting applied to the foreground subject (0.0–2.0, default: 1.0). Higher values produce more intense relighting effects.
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --foreground-gain 1.5
```

background_gain: Controls the strength of relighting applied to the background (0.0–2.0, default: 1.0).

python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --background-gain 0.8

blur_strength: Controls background blur intensity (0.0–1.0, default: 0.0). Can be combined with any background_source to produce blurred variants.
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --blur 0.5
```
specular: Controls specular highlight intensity (0.0–2.0, default: 0.0). Higher values produce more prominent highlights on reflective surfaces.
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --specular 1.5
```

Encoding Options#

To configure the encoding quality of the output video, use the output_video_encoding parameter. The encoding uses a VideoEncoding message with three mutually exclusive modes: lossy, lossless, or custom.

Bitrate control (lossy mode): Target bitrate is specified in megabits per second (bitrate_mbps). Use 0 for auto, which selects the server default 10 Mbps (fixed; not tied to output resolution). Higher values improve quality and increase output size; lower values save bandwidth at the cost of quality.
- In the configuration message, set output_video_encoding.lossy.bitrate_mbps to the desired Mbps or 0 for auto (10 Mbps).
- The sample Python CLI uses --bitrate in bps (rounded to Mbps for the proto); default matches 10 Mbps (10,000,000 bps). Example explicit bitrate:
```
# 20,000,000 bps = 20 Mbps
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --bitrate 20000000
```
IDR interval control: idr_interval is the spacing between IDR frames in frames (not seconds). The sample CLI defaults to 8 frames and always sends a value to the server. Values must be ≥ 1 (0 is invalid). Sample CLI override:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --idr-interval 10
```
Lossless mode: Encodes the output video without compression. To enable lossless mode via the sample client, use the --lossless flag:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --lossless
```
Custom encoding parameters: Provides fine-grained control for expert users via JSON configuration. These parameters configure properties of the DeepStream H264 encoder.

To specify custom encoding parameters, use the --custom-encoding-params option:
```
python relighting.py --target 127.0.0.1:8001 --video-input /path/to/video.mp4 --custom-encoding-params '{"bitrate":5000000,"idr_interval":16,"maxbitrate":6000000}'
```
Note

Custom encoding parameters override standard bitrate and IDR interval settings. Use with caution because incorrect values can affect output quality or encoding stability.

The following example shows how to set the configuration parameters while sending an inference request from a Python client to the Relighting NIM:

import relighting_pb2
from nvidia.ai4m.video.v1 import video_pb2

config = relighting_pb2.RelightConfig()

# Set HDR preset or custom HDR image
config.hdri_preset_id = 3  # Little Paris Eiffel Tower preset
# OR config.hdri_image_provided = True  # Send custom HDR inline via ImageData

# Set pan and vfov angles (in radians)
config.angle_pan_radians = -1.5708  # -90 degrees
config.angle_v_fov_radians = 1.0472  # 60 degrees

# Set background source (0=source video, 1=custom image, 2=HDR)
config.background_source = 0  # BACKGROUND_UNSPECIFIED (source video)

# Set gain/effect parameters
config.foreground_gain = 1.0
config.background_gain = 1.0
config.blur_strength = 0.5  # Combine with any background_source for blur
config.specular = 0.0

# Set output video encoding (lossy): 10 Mbps with 60-frame IDR interval
config.output_video_encoding.CopyFrom(
    video_pb2.VideoEncoding(
        lossy=video_pb2.LossyEncoding(bitrate_mbps=10, idr_interval=60)
    )
)

yield relighting_pb2.RelightRequest(config=config)

For more information, refer to the AI for Media Relighting NIM clients GitHub repository: NVIDIA-Maxine/nim-clients.