> This page is for NVIDIA Relighting NIM, version 1.2.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.2.0 (Latest) (default): https://docs.nvidia.com/nim/maxine/relighting/1.2.0/llms.txt
> - 1.1.0: https://docs.nvidia.com/nim/maxine/relighting/1.1.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/maxine/llms.txt.

# Advanced Usage

## Model Caching

When the container launches for the first time, it downloads the required models from NGC. To avoid downloading the models on subsequent runs, you can cache them locally by using a cache directory:

```bash
# Create the cache directory on the host machine.
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
chmod a+rwX "$LOCAL_NIM_CACHE" 2>/dev/null || true

# Run the container with the cache directory mounted in the appropriate location.
```

```bash
docker rm -f relighting-nim 2>/dev/null
docker run -d --name=relighting-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NV_AI4M_MAX_CONCURRENCY_PER_GPU=1 \
  -e NIM_HTTP_API_PORT=8000 \
  -e NIM_GRPC_API_PORT=8001 \
  -p 8000:8000 \
  -p 8001:8001 \
  -p 9002:9002 \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  nvcr.io/nim/nvidia/relighting:1.2.0

docker logs -f relighting-nim
```


The container runs as uid 1000, so the model folders it writes into the cache belong to uid 1000. If your user has a different uid, `rm` cannot delete them; remove the cached Relighting models with the container instead:

```bash
# The cache directory from the command above, set again so this also works in a new shell.
export LOCAL_NIM_CACHE=~/.cache/nim
```

```bash
docker run --rm -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  nvcr.io/nim/nvidia/relighting:1.2.0 \
  bash -c 'rm -rf /opt/nim/.cache/ngc/hub/models--*--relighting'
```


Afterwards, `ls "$LOCAL_NIM_CACHE/ngc/hub"` lists no `models--*--relighting` folder.

## SSL Enablement

NVIDIA AI for Media Relighting provides an SSL mode to ensure secure communication between clients and the server by encrypting data in transit.
To enable SSL, you must provide the path to the SSL certificate and key files in the container. The following example shows how to do this:

The container expects a CA certificate, a server certificate signed by it, and the matching server key. `mtls` mode also needs a client certificate signed by the same CA. To generate a self-signed set for local testing, in an empty directory:

```bash
mkdir -p certs && cd certs

# 1. CA
openssl req -x509 -newkey rsa:4096 -nodes -days 365 \
  -keyout ssl_ca_key.pem -out ssl_ca_cert.pem \
  -subj "/CN=relighting-test-ca"

# 2. Server key + CSR
openssl req -newkey rsa:4096 -nodes \
  -keyout ssl_key_server.pem -out server.csr \
  -subj "/CN=localhost"

# 3. Server certificate, signed by the CA. The client verifies the server's
#    hostname against the certificate, so the SAN is required, not optional.
printf "subjectAltName=DNS:localhost,IP:127.0.0.1\n" > san.cnf
openssl x509 -req -in server.csr -days 365 \
  -CA ssl_ca_cert.pem -CAkey ssl_ca_key.pem -CAcreateserial \
  -extfile san.cnf \
  -out ssl_cert_server.pem

# 4. Client key + CSR (mtls only -- the server rejects a connection with no
#    client certificate)
openssl req -newkey rsa:4096 -nodes \
  -keyout ssl_key_client.pem -out client.csr \
  -subj "/CN=relighting-client"

# 5. Client certificate, signed by the same CA
openssl x509 -req -in client.csr -days 365 \
  -CA ssl_ca_cert.pem -CAkey ssl_ca_key.pem -CAcreateserial \
  -out ssl_cert_client.pem

# openssl writes ssl_key_server.pem at mode 600 (owner-only). The server runs
# inside the container as uid 1000 (user nvs), not as the uid that ran openssl,
# so grant that uid, and only that uid, read access; otherwise the server
# refuses to start:
#   ERROR:inference:SSL configuration is invalid; refusing to start.
#   nimlib.exceptions.SSLConfigurationError: Error verifying SSL files: [Errno 13] Permission denied
# Nothing to do when you are uid 1000. setfacl comes from the acl package.
[ "$(id -u)" = 1000 ] || setfacl -m u:1000:r ssl_key_server.pem

cd ..
```

<Note>
This block is POSIX shell, so it runs under `sh`/`dash` as well as bash and zsh. It leaves a few scratch files next to the certificates -- `san.cnf`, the two CSRs (`server.csr`, `client.csr`), and the CA serial (`ssl_ca_cert.srl`).
</Note>

<Warning>
`SSL_CERT` must be an **absolute** path. Docker reads a relative `-v` source as the name of a volume rather than a directory, so a relative value silently creates an empty named volume, mounts it over `/opt/nim/crt`, and the server then exits reporting that it cannot find a certificate you just generated.
</Warning>

```bash
# Run from the directory where you ran the recipe above, so that $PWD/certs is the directory it created.
export SSL_CERT="$PWD/certs"
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
chmod a+rwX "$LOCAL_NIM_CACHE" 2>/dev/null || true
```

```bash
docker rm -f relighting-nim 2>/dev/null
docker run -d --name=relighting-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -v "$SSL_CERT:/opt/nim/crt/:ro" \
  -e NGC_API_KEY=$NGC_API_KEY \
  -p 8000:8000 \
  -p 8001:8001 \
  -p 9002:9002 \
  -e NIM_SSL_MODE="mtls" \
  -e NIM_SSL_CA_CERTS_PATH="/opt/nim/crt/ssl_ca_cert.pem" \
  -e NIM_SSL_CERT_PATH="/opt/nim/crt/ssl_cert_server.pem" \
  -e NIM_SSL_KEY_PATH="/opt/nim/crt/ssl_key_server.pem" \
  nvcr.io/nim/nvidia/relighting:1.2.0

docker logs -f relighting-nim
```


`NIM_SSL_MODE` can be set to `mtls`, `tls`, or `disabled`. If set to `mtls`, the container uses mutual TLS authentication. If set to `tls`, the container uses TLS authentication.
For more information, refer to [Environment Variables](/nim/maxine/relighting/getting-started#environment-variables).

Verify the permissions of the SSL certificate and key files on the host machine. The container runs as uid 1000 and cannot start if the files are not readable by that user, for example a key that `openssl` wrote with mode `600`. Grant read access to uid 1000 only, with `setfacl -m u:1000:r ssl_key_server.pem`, rather than making the private key world-readable. The server certificate must carry a Subject Alternative Name for the hostname that clients connect to.

<Note>
`NIM_SSL_MODE` secures the gRPC port (8001) only. The HTTP endpoints on port 8000 stay plaintext.
</Note>

To connect the sample Python client to an SSL-enabled server, set `--ssl-mode` to match `NIM_SSL_MODE`. Pass `--ssl-root-cert` for `TLS` or `MTLS`, and additionally `--ssl-cert` and `--ssl-key` for `MTLS`. Run the client from its `scripts` folder, with the virtual environment from [Basic Inference](/nim/maxine/relighting/basic-inference#verify-the-service-and-install-the-client) active. The launch commands above export `SSL_CERT`, so the first line below keeps that value in the same shell. In a new shell, first run `export SSL_CERT="{absolute path of the certs directory the recipe created}"`; otherwise the first line stops with a message naming `SSL_CERT`:

```bash
export SSL_CERT="${SSL_CERT:?set SSL_CERT to the absolute path of the certs directory the recipe created}" && \
python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 \
  --ssl-mode MTLS \
  --ssl-root-cert "$SSL_CERT/ssl_ca_cert.pem" \
  --ssl-cert "$SSL_CERT/ssl_cert_client.pem" \
  --ssl-key "$SSL_CERT/ssl_key_client.pem"
```

A client whose `--ssl-mode` does not match the server fails to connect with `UNAVAILABLE`, for example a plaintext client against a `tls` or `mtls` server, or `--ssl-mode TLS` against an `mtls` server.

## Multiple Concurrent Inputs

By default, the NIM processes one request per GPU (`NV_AI4M_MAX_CONCURRENCY_PER_GPU=1`). To process several requests at once, set `NV_AI4M_MAX_CONCURRENCY_PER_GPU` to an integer greater than 1 on the server container. The total number of concurrent requests the server accepts is the number of GPUs multiplied by `NV_AI4M_MAX_CONCURRENCY_PER_GPU`.

For example, to process two requests at once on one GPU, launch the NIM with the command in [Launching the NIM Container](/nim/maxine/relighting/getting-started#launching-the-nim-container), with `-e NV_AI4M_MAX_CONCURRENCY_PER_GPU=2` in place of `-e NV_AI4M_MAX_CONCURRENCY_PER_GPU=1`. Then send two requests at the same time from `nim-clients/relighting/scripts/`:

```bash
python relighting.py --target 127.0.0.1:8001 --output output_1.mp4 & p1=$!
python relighting.py --target 127.0.0.1:8001 --output output_2.mp4 & p2=$!
wait $p1; echo "request 1 exit $?"
wait $p2; echo "request 2 exit $?"
```

A request that arrives when every slot is busy waits up to 3 seconds for a slot. If no slot frees in that time, the request fails with `RESOURCE_EXHAUSTED` and the message `Server is at capacity. Please retry shortly (active request limit reached).` Retry the request after the running request finishes, or raise `NV_AI4M_MAX_CONCURRENCY_PER_GPU` and restart the container.

Each concurrent request uses its own GPU memory, and the amount grows with the input resolution. If the GPU runs out of memory while it decodes a request's frames (for example, more concurrent 4K requests than the GPU memory can hold), that request fails with `RESOURCE_EXHAUSTED`. The request ends with this error, never with a shorter video reported as success. Retry with fewer concurrent requests or a lower input resolution, or lower `NV_AI4M_MAX_CONCURRENCY_PER_GPU`.

Requests with `background_source = 2` (HDR projection) that run at the same time on one GPU must have the same video resolution. When two such requests of different resolutions overlap, the background of each comes out at the wrong size. Send them one after another, or keep `NV_AI4M_MAX_CONCURRENCY_PER_GPU=1`.

## NIM Service Configuration Parameters via Client

Use the following options to configure each request to Relighting. Run the examples in this section from `nim-clients/relighting/scripts/` with the virtual environment from [Basic Inference](/nim/maxine/relighting/basic-inference#verify-the-service-and-install-the-client) active.

### HDR/Illumination Options

- `hdri_preset_id`: Specifies a built-in HDR preset to create one of several lighting moods. Mutually exclusive with `hdri_image_provided`.
  - `0` (Lounge): Lounge environment.
  - `1` (Cobblestone Street Night): Cobblestone street at night.
  - `2` (Glasshouse Interior): Glasshouse interior.
  - `3` (Little Paris Eiffel Tower): Little Paris Eiffel Tower.
  - `4` (Wooden Studio): Wooden studio environment.

  The following example uses the sample Python client with the Little Paris Eiffel Tower HDR preset:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --hdri-id 3
  ```

- `hdri_image_provided`: When set to `true`, indicates that a custom HDR environment map is sent inline as `ImageData`. Set either this field or `hdri_preset_id`, not both. If the configuration sets `hdri_preset_id`, the NIM ignores an HDR `ImageData` and logs a warning. The image must be an equirectangular map, twice as wide as it is tall (for example 4096 x 2048 pixels), and can be up to 150 MB and 8192 x 4096 pixels. The NIM checks the size first, so an image that breaks both rules gets the size error.

  The following example uses the sample Python client with a custom HDR file. The shipped `sample_lounge.hdr` is the Lounge preset, so the first command mirrors it into a different environment map:

  ```bash
  ffmpeg -y -i ../assets/sample_lounge.hdr -vf hflip ../assets/custom.hdr
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --hdr ../assets/custom.hdr
  ```

- `angle_pan_radians`: Pan angle for rotating the HDR environment, in radians (range -π to π). Default is -1.5708 (-90 degrees).

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --pan -45  # degrees
  ```

- `angle_v_fov_radians`: Vertical field of view of the HDR environment map when it is used as the background (`background_source = 2`), in radians (range 0.1 to π). Default is 1.0472 (60 degrees).

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --background-source 2 --vfov 90  # degrees
  ```

### Autorotate Options

- `autorotate`: Enable automatic rotation of the HDR environment during video playback for dynamic lighting effects.

- `rotation_rate`: Speed of rotation in radians per second (range 0 to 2π). Default is 0.349 (20 degrees per second). Only applies when autorotate is enabled.

  The following example enables autorotate with a custom rotation speed:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --autorotate --rotation-rate 45  # degrees per second
  ```

### Background Source Options

- `background_source`: Selects which background source to use.
  - `0` (UNSPECIFIED): Use source video frame as background (default).
  - `1` (FROM_IMAGE): Use custom background image uploaded via `ImageData`.
  - `2` (FROM_HDR): Use HDR environment map as background.

  The following example uses the sample Python client with the HDR environment map as the background:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --background-source 2
  ```

- Custom background image: with `background_source = 1` (FROM_IMAGE), send the image inline as `ImageData` with `IMAGE_TYPE_BACKGROUND`. PNG, JPEG, and HDR are supported. The server detects the format from the image bytes, so `background_image_type` is not used. The NIM decodes the background image and resizes it to the video once per request, so a large image or an `.hdr` file adds time to each request. For the shortest requests, use a PNG or JPEG background at the output resolution. The image can be up to 150 MB and 8192 x 4096 pixels.

  ```bash
  ffmpeg -y -i ../assets/sample_video.mp4 -frames:v 1 ../assets/background.png
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --background-source 1 --background-image ../assets/background.png
  ```

- `background_color`: Solid background color as an integer in `0xRRGGBB` form. If the request does not set it, the NIM uses `0x808080` (gray); the sample client sets it only when you pass `--background-color`. Used when `background_source` is `FROM_IMAGE` and no image is uploaded. The following example composites the subject over solid blue:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --background-source 1 --background-color 0x0000FF
  ```

### Effect Parameters

- `foreground_gain`: Controls the strength of relighting applied to the foreground subject (0.0–2.0, default: 1.0). Higher values produce more intense relighting effects.

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --foreground-gain 1.5
  ```

- `background_gain`: Controls the strength of relighting applied to the background (0.0–2.0, default: 1.0). Applies only when `background_source` is `2` (FROM_HDR).

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --background-source 2 --background-gain 0.8
  ```

### Encoding Options

To configure the encoding quality of the output video, use the `output_video_encoding` parameter. The encoding uses a `VideoEncoding` message with three mutually exclusive modes: lossy, lossless, or custom.

If the configuration message has no `output_video_encoding`, the NIM uses lossy mode with the auto bitrate and an IDR interval of about 2 seconds of video: the input frame rate multiplied by 2, in frames (60 frames for 30 fps input). If a `lossy` message does not set `bitrate_mbps` or `idr_interval`, the NIM uses the same default for that field. In lossy mode, the sample client always sends both fields.

The sample client sends one mode for each request. If you use `--lossless` and `--custom-encoding-params` together, the client stops with a configuration error. With `--lossless` or `--custom-encoding-params`, the client ignores `--bitrate` and `--idr-interval` and prints a warning.

- **Bitrate control (lossy mode):** Target bitrate is specified in megabits per second (`bitrate_mbps`). Use `0` for auto, which selects a bitrate from the output resolution: 10 Mbps up to 720p (1280×720 pixels), 15 Mbps up to 1080p (1920×1080 pixels), and 25 Mbps above 1080p. Higher values improve quality and increase output size; lower values save bandwidth at the cost of quality.
  - In the configuration message, set `output_video_encoding.lossy.bitrate_mbps` to the desired Mbps (1 to 200) or `0` for auto.

  - The sample Python CLI uses `--bitrate` in bps (rounded to Mbps for the proto); default matches 10 Mbps (10,000,000 bps). Example explicit bitrate:
    ```bash
    # 20,000,000 bps = 20 Mbps
    python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --bitrate 20000000
    ```

- **IDR interval control:** `idr_interval` is the spacing between IDR frames in frames (not seconds). The sample CLI defaults to 8 frames and always sends a value to the server. Values must be ≥ 1 (`0` is invalid).
  Sample CLI override:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --idr-interval 10
  ```

- **Lossless mode:** Encodes the output video with lossless H.264 compression. Use lossless mode when the output is an intermediate file for more work, such as color grading, compositing, editing, or a quality comparison against a reference. To deliver video to viewers, use lossy mode.

  Before you use lossless mode, consider these points:

  - Output files are several times larger than lossy output, so each request transfers and stores more data.
  - The NIM converts the frames to 8-bit YUV 4:2:0 before it encodes them, as in lossy mode. The encoding adds no loss, but the color information is subsampled to 4:2:0. Lossless mode does not give 4:4:4 or RGB output.
  - The output uses the H.264 High 4:4:4 Predictive profile, which lossless H.264 requires. Some players and devices decode only the Baseline, Main, and High profiles. Make sure that your player or tool decodes this profile.
  - Bitrate and IDR interval are lossy-mode settings and do not apply in lossless mode.

  To enable lossless mode with the sample client, use the `--lossless` flag:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --lossless
  ```

- **Custom encoding parameters:** Provides fine-grained control for expert users via JSON configuration. These parameters configure properties of the [DeepStream H264 encoder](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvvideo4linux2.html#encoder).

  To specify custom encoding parameters, use the `--custom-encoding-params` option:

  ```bash
  python relighting.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --custom-encoding-params '{"bitrate":5000000,"idrinterval":16,"maxbitrate":6000000}'
  ```

  The NIM accepts these encoder (`nvv4l2h264enc`) properties: `bitrate`, `maxbitrate`, `iframeinterval`, `idrinterval`, `vbvbufsize`, `vbvinit`, `SliceIntraRefreshInterval`, `slice-header-spacing`, `preset-id`, `aq`, `cq`, `num-B-Frames`, `num-Ref-Frames`, `control-rate`, `tuning-info-id`, `profile`, `temporalaq`, `insert-vui`, `insert-sps-pps`, `insert-aud`, `disable-cabac`, and `extended-colorformat`. Any other key, such as `idr_interval` or `qp-range`, or a value outside the property's range makes the request fail with `INVALID_ARGUMENT`.

  B-frames (`num-B-Frames` 1 or 2) need `profile` `2` (Main), `4` (High), or `7` (High 4:4:4), and `idrinterval` and `iframeinterval` values that are multiples of `num-B-Frames` + 1. If the request does not set `profile` or `idrinterval`, the NIM uses `profile` `4` and the smallest such `idrinterval` of 8 or more. Another profile or interval makes the request fail with `INVALID_ARGUMENT`.

  <Note>
  With custom encoding parameters, the `bitrate` and `idrinterval` keys in the JSON set the bitrate and IDR interval; if the JSON has no `bitrate` key, the encoder uses 20 Mbps. `bitrate` and `maxbitrate` are in bits per second (`5000000` is 5 Mbps), as `--bitrate` is; the lossy `bitrate_mbps` field of the configuration message is in Mbps. Use with caution because incorrect values can affect output quality or encoding stability.
  </Note>

The following example builds the configuration message of a request with the Python client's generated stubs. It sends nothing and prints nothing; to send it, use `generate_requests()` in `scripts/relighting.py`:

```python
# Run from nim-clients/relighting/scripts/ with the virtual environment active.
import sys

sys.path.append("../interfaces")  # Python stubs shipped with the client

from nvidia.ai4m.relighting.v1 import relighting_pb2
from nvidia.ai4m.video.v1 import video_pb2

config = relighting_pb2.RelightConfig()

# Set HDR preset or custom HDR image
config.hdri_preset_id = 3  # Little Paris Eiffel Tower preset
# OR config.hdri_image_provided = True  # Send custom HDR inline via ImageData

# Set background source (0=source video, 1=custom image, 2=HDR)
config.background_source = 2  # BACKGROUND_SOURCE_FROM_HDR (HDR projection)

# Set pan and vertical FOV angles (in radians).
# The vertical FOV applies only with background_source = 2.
config.angle_pan_radians = -1.5708  # -90 degrees
config.angle_v_fov_radians = 1.0472  # 60 degrees

# Set gain parameters. background_gain applies only with background_source = 2.
config.foreground_gain = 1.0
config.background_gain = 1.0

# Set output video encoding (lossy): 10 Mbps with 60-frame IDR interval
config.output_video_encoding.CopyFrom(
    video_pb2.VideoEncoding(
        lossy=video_pb2.LossyEncoding(bitrate_mbps=10, idr_interval=60)
    )
)

# The configuration is the first message of the Relight request stream,
# followed by any image data and then the video chunks.
request = relighting_pb2.RelightRequest(config=config)
```

The sample client builds the complete request stream in `generate_requests()` in `scripts/relighting.py`.

For more information, refer to the NIM Clients GitHub repository: [NVIDIA AI for Media NIM Clients](https://github.com/NVIDIA-Maxine/nim-clients).