Quickstart for Harmonizer NIM

Launch Harmonizer NIM and process a sample image. For system and GPU requirements, refer to Harmonizer NIM Prerequisites.

Prerequisites

Before starting, complete these requirements:

• Prepare a supported NVIDIA GPU and install the Docker runtime and NVIDIA Container Toolkit listed in Harmonizer NIM Prerequisites.
• Create an NGC account that can access the Harmonizer NIM container and model assets.
• Install Python 3 and ensure that pip and venv are available on the client host.

Quickstart Steps

Complete the following steps to launch the NIM and verify an inference response. Choose one inference transport when starting the container.

1. Generate an NGC API key.

   An NGC API key is required to access NGC resources. Generate the key on the NGC Setup page.

   When you create the key, select NGC Catalog in the Services Included dropdown. Select additional services if you reuse the key for other purposes.

   

2. Export the API key into the current shell.

   Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.

   Store the key in a file that is readable only by your user account. Load it into the current terminal without placing the key itself in your shell history:

   export NGC_API_KEY_FILE=/path/to/ngc_api_key
   chmod 600 "$NGC_API_KEY_FILE"
   export NGC_API_KEY="$(cat "$NGC_API_KEY_FILE")"

   Do not add the key value to ~/.bashrc, ~/.zshrc, or another shell startup file. The exported value applies only to the current shell session.

3. Log in to the NVIDIA Container Registry.

   To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry using the following command:

   echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

   Use $oauthtoken as the username and NGC_API_KEY as the password. The special $oauthtoken username indicates that you authenticate with an API key instead of a username and password.

4. Launch the Harmonizer NIM container with HTTP or gRPC inference.

   Set the container image and persistent cache location:

   export IMG_NAME=nvcr.io/nim/nvidia/harmonizer:1.0.0
   export LOCAL_NIM_CACHE=~/.cache/nim
   mkdir -p "$LOCAL_NIM_CACHE"
   chmod -R a+rwX "$LOCAL_NIM_CACHE"

   The launch command keeps the current terminal attached to the container. Keep it running, and open a second terminal on the same host for readiness checks and client requests.

   Select one inference transport when you start the NIM:

   HTTP (default)

   gRPC

   docker run -it --rm --name harmonizer \
      --runtime=nvidia \
      --gpus all \
      --shm-size=8g \
      --ulimit memlock=-1 \
      --ulimit stack=67108864 \
      -e NGC_API_KEY \
      -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
      -p 8000:8000 \
      $IMG_NAME

   The HTTP and gRPC inference transports are mutually exclusive startup choices. The HTTP health and readiness routes remain available in either mode:

   Startup Selection	Inference Endpoint	Health Endpoints
   HTTP (default)	POST http://localhost:8000/v1/infer	GET http://localhost:8000/v1/health/live and GET http://localhost:8000/v1/health/ready
   gRPC	harmonizer.v1.Harmonizer/Process on localhost:50051	GET http://localhost:8000/v1/health/live and GET http://localhost:8000/v1/health/ready

   Both transports support both processing modes. Select non-temporal image mode with context_length=0 or temporal mode with context_length=4 on each request; switching processing modes does not require restarting the NIM.

   The container runs as user ID 1000, so the cache directory mounted at /opt/nim/.cache must be writable by UID 1000. The setup command above grants the required access for a local cache. NFS-backed home directories can still reject access because of server-side identity mapping or ACLs; in that case, use a local-disk cache or run the container with the host user’s UID. Refer to Environment Variables and Configuration for cache permission options.

   Docker Launch Flag Reference

   Flag	Description
   -it	Runs the container interactively with a terminal attached.
   --rm	Deletes the container after it exits.
   --name harmonizer	Gives the container a stable name for local inspection.
   --runtime=nvidia	Makes NVIDIA drivers available inside the container.
   --gpus all	Exposes GPUs inside the container. Harmonizer uses one GPU per container.
   --shm-size=8g	Sets the shared-memory segment to the recommended 8 GiB launch value.
   --ulimit memlock=-1	Allows locked memory for GPU-serving runtime components.
   --ulimit stack=67108864	Raises the process stack limit.
   -e NGC_API_KEY	Provides the container with the token necessary to download models and resources from NGC. Refer to Export the API Key in the Quickstart.
   -v "$LOCAL_NIM_CACHE:/opt/nim/.cache"	Mounts a persistent model and compiled-kernel cache.
   -e NIM_INFERENCE_PROTOCOL=grpc	Selects the gRPC inference transport instead of the default HTTP transport.
   -p 8000:8000	Exposes HTTP inference in HTTP mode and health endpoints in both modes.
   -p 50051:50051	Exposes gRPC inference in gRPC mode.

5. Check that the NIM is ready.

   Check readiness:

   curl http://localhost:8000/v1/health/ready

   Readiness is reported after the model loads and startup warmup completes. A cold start typically takes about 2 to 9 minutes, depending on the GPU. Persisting /opt/nim/.cache lets later starts reuse the compiled-kernel cache. For per-GPU estimates and startup tuning, refer to Environment Variables and Configuration.

6. Download the sample frame and run an HTTP or gRPC request.

   The examples use an input frame from the NVIDIA Harmonizer model-publication repository. The API accepts encoded image data rather than an image URL, so download the sample to the current directory first:

   curl -L \
     https://raw.githubusercontent.com/NVIDIA/harmonizer/d1234ff7afb00b6084967dc31b18b526c9b9e7e0/examples/0.png \
     -o input.png

   The first examples send this image to the non-temporal image lane (context_length=0) and write the enhanced image back to disk.

   HTTP Request

   This example requires a NIM started with the default HTTP launch option above. Use HTTP for a simple JSON interface for testing and debugging. For latency-sensitive production traffic, prefer the gRPC path below.

   In the client terminal, create a Python virtual environment and install the HTTP client dependency:

   python3 -m venv .venv
   source .venv/bin/activate
   python -m pip install requests

   Save the following example as http_client.py in the directory containing input.png:

   import base64
   from pathlib import Path
   import requests
   def image_data(path: str) -> str:
       return base64.b64encode(Path(path).read_bytes()).decode("ascii")
   payload = {
       "request_id": "demo-0001",
       "images": [
           {
               "id": "frame-000000",
               "data": image_data("input.png"),
           }
       ],
       "parameters": {
           "output_format": "png",
           "context_length": 0,
           "return_policy": "all",
       },
   }
   response = requests.post(
       "http://localhost:8000/v1/infer",
       json=payload,
       timeout=300,
   )
   response.raise_for_status()
   result = response.json()
   output = result["outputs"][0]["image"]
   Path("output.png").write_bytes(base64.b64decode(output["data"]))
   print(result["request_id"], output["width"], output["height"])

   Run the HTTP client:

   python http_client.py

   Optional: Process a Temporal Sequence

   For a compact demonstration of the temporal API, repeat the downloaded sample five times and set context_length=4. The first four entries establish context, and return_policy=exclude_context returns only the fifth result. Repeating a frame demonstrates the request mechanics; use consecutive frames from a real sequence to evaluate temporal behavior.

   Save the following example as temporal_client.py in the same directory as input.png:

   import base64
   from pathlib import Path
   import requests
   def image_data(path: str) -> str:
       return base64.b64encode(Path(path).read_bytes()).decode("ascii")
   sample = image_data("input.png")
   payload = {
       "request_id": "temporal-demo-0001",
       "images": [
           {"id": f"frame-{i:06d}", "data": sample}
           for i in range(5)
       ],
       "parameters": {
           "output_format": "png",
           "context_length": 4,
           "return_policy": "exclude_context",
       },
   }
   response = requests.post(
       "http://localhost:8000/v1/infer",
       json=payload,
       timeout=300,
   )
   response.raise_for_status()
   result = response.json()
   output = result["outputs"][0]["image"]
   Path("temporal-output.png").write_bytes(base64.b64decode(output["data"]))
   print(result["request_id"], output["width"], output["height"])

   Run the temporal client in the virtual environment used for the HTTP example:

   python temporal_client.py

   gRPC Request

   This example requires a NIM started with the gRPC launch option above. If the current container is using the default HTTP transport, stop it and relaunch it with the gRPC command:

   docker stop harmonizer

   The persistent cache mounted at /opt/nim/.cache is reused after the restart, although model loading and warmup still run. Refer to startup time and compile-cache guidance for representative times.

   Use the gRPC API when your client can send compressed image bytes directly. It avoids the HTTP base64 and JSON adapter path. The service is harmonizer.v1.Harmonizer and the method is Process.

   Install the Python gRPC runtime and code generator:

   python -m pip install grpcio grpcio-tools

   Download the canonical schema for this documentation version: harmonizer.proto. Save it in the directory where you will generate the client modules.

   Generate the Python message and service modules:

   python -m grpc_tools.protoc \
     -I. \
     --python_out=. \
     --grpc_python_out=. \
     harmonizer.proto

   Save the following client as grpc_client.py in the same directory as the generated modules and input.png:

   from pathlib import Path
   import grpc
   import harmonizer_pb2
   import harmonizer_pb2_grpc
   request = harmonizer_pb2.ProcessRequest(
       request_id="demo-0001",
       parameters=harmonizer_pb2.ProcessParameters(
           output_format=harmonizer_pb2.OUTPUT_FORMAT_PNG,
           context_length=0,
           return_policy=harmonizer_pb2.RETURN_POLICY_ALL,
       ),
   )
   frame = request.images.add()
   frame.id = "frame-000000"
   frame.data = Path("input.png").read_bytes()
   options = [
       ("grpc.max_receive_message_length", 256 * 1024 * 1024),
       ("grpc.max_send_message_length", 256 * 1024 * 1024),
   ]
   with grpc.insecure_channel("localhost:50051", options=options) as channel:
       stub = harmonizer_pb2_grpc.HarmonizerStub(channel)
       response = stub.Process(request, timeout=300)
   Path("output.png").write_bytes(response.outputs[0].image.data)
   print(response.request_id, response.outputs[0].image.width, response.outputs[0].image.height)

   Run the client:

   python grpc_client.py
   The gRPC server does not enable server reflection. Generic clients such as grpcurl must be given harmonizer.proto explicitly instead of using reflection to discover the service.

   The gRPC server accepts messages up to 256 MiB (268,435,456 bytes) so a full temporal clip can be sent in a single request. Set the client grpc.max_receive_message_length and grpc.max_send_message_length channel options accordingly.

Next Steps

Explore the request contract, API, and supported GPUs: