> This page is for version 1.0.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/harmonizer/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/harmonizer/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/harmonizer/_mcp/server.

# Harmonizer NIM Documentation

> Understand Harmonizer image enhancement, processing lanes, and APIs, and find deployment and integration guides.

[Harmonizer NIM](https://catalog.ngc.nvidia.com/orgs/nim/nvidia/containers/harmonizer)
serves the NVIDIA Harmonizer model (`nvidia/harmonizer`), an online
generative enhancement
model for RGB frames rendered from neural-reconstructed scenes. It is designed
to reduce novel-view reconstruction artifacts and improve foreground and background
appearance, lighting, and cast-shadow consistency when dynamic objects are
inserted into reconstructed scenes. Temporal mode uses previously enhanced
frames to improve frame-to-frame consistency. A request contains one or more
rendered frames; `return_policy` controls which enhanced frames are returned.

The NIM exposes a custom image-to-image API over both HTTP and gRPC. The active
protocol is selected at startup. The logical request and response contract is
equivalent across transports, but the wire representations differ: HTTP uses
base64-encoded image payloads and string parameter values, while gRPC uses
binary `bytes` and protobuf enums.

## Skip Ahead

#### [Support Matrix](/nim/harmonizer/support-matrix)

Check supported GPUs, precision, and deployment profiles.

#### [Get Started](/nim/harmonizer/get-started)

Choose a Docker or Kubernetes deployment and run your first request.

#### [API Examples](/nim/harmonizer/api-reference)

Integrate HTTP or gRPC inference into your application.

#### [Troubleshooting](/nim/harmonizer/troubleshooting)

Resolve startup, model-cache, and inference problems.

## Use Cases

Choose the processing pattern that matches your rendered frames.

#### [Enhance Independent Frames](/nim/harmonizer/sampling-params)

Use the non-temporal lane to process each image independently.

#### [Enhance Frame Sequences](/nim/harmonizer/sampling-params)

Use temporal context to improve consistency across a sequence of frames.

## Architecture

Harmonizer is a single-step, temporally conditioned image diffusion model built
on the NVIDIA Cosmos backbone (`nvidia/Cosmos-Predict2-0.6B-Text2Image`).
The public research release is open under Apache-2.0. Source code is available
in the [/NVIDIA/harmonizer GitHub repo](https://github.com/NVIDIA/harmonizer),
and checkpoints are available on the
[NVIDIA Harmonizer Hugging Face model page](https://huggingface.co/nvidia/Harmonizer).

The NIM serves a single release-approved checkpoint
(`diffusion_harmonizer.pkl`) through two processing modes over the same model:

* A **non-temporal image lane** runs the model single-frame (`V=1`) for
  independent image-to-image enhancement. This is equivalent to the research
  repository's `--nontemporal` mode and is the lane wired into render-loop
  integrations.
* A **temporal context lane** stacks previously generated output frames as
  autoregressive context (`V=5`: the current frame plus four previous
  generated outputs) for higher temporal consistency. The lane falls back to
  `V=1` for the first frames of a sequence until enough history exists.

The lane is selected per request, not at startup. Regardless of whether HTTP or
gRPC was selected as the inference transport, a single running NIM serves both
processing patterns without a restart. Concurrent same-shape requests are
coalesced into GPU batches by an internal queue, so the same deployment
dynamically serves low-latency per-frame traffic and high-throughput batch
traffic. The effective batch size adapts to concurrency.

## Inputs and Outputs

The NIM runs inference at `1024x576` in bfloat16 (BF16) precision.
Inputs can use any positive image resolution. The runtime derives the dimensions
from decoded PNG/JPEG data or from the raw tensor `shape`, resizes the image to
the model resolution for inference, and resizes the output back to the
corresponding input resolution. This resize is a direct stretch: it does not
preserve aspect ratio, letterbox, or crop. Use 16:9 inputs to avoid geometric
distortion during inference. Outputs are PNG, JPEG, or raw `uint8` tensors,
selected per request. For transport and codec performance guidance, refer to
the [Support Matrix](/nim/harmonizer/support-matrix).

## APIs

The container exposes the following inference surface:

* `POST /v1/infer` for the native Harmonizer request and
  response schema over HTTP.
* `harmonizer.v1.Harmonizer/Process` for the same payload over binary gRPC.

For request examples and the embedded OpenAPI schema, refer to the
[API Reference](/nim/harmonizer/api-reference).