Use the NuRec Fixer Model#

Fixer is a post-trained model derived from the Sana model, trained to perform artifact removal in Neural Reconstruction tasks. Sana can generate high-resolution, photorealistic images from detailed text prompts, including scenes, objects, environments, and abstract compositions.

Fixer is meant to assist developers of Autonomous Vehicles in their efforts to enhance and improve Neural Reconstruction pipelines. The model takes an image as an input and outputs a fixed image.

References

  • Model Card++ is available on NGC

  • Sana GitHub Repo

  • Sana Project Page

  • Sana Paper titled as “SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers”

Prerequisites#

Hardware Support#

Model Precision

GPU Memory (GB)

# of GPUs

FP32

>= 24 GB

1

BF16

>= 12 GB

1

Software Support#

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

For more information on enumerating multi-GPU systems, please see the NVIDIA Container Toolkit’s GPU Enumeration Docs.

NGC#

You must have an authenticated NGC (NVIDIA GPU Cloud) account with access to the model checkpoint. Use the following procedure to log in to NGC, and set the NGC_API_KEY environment variable.

  1. Create an account on NGC.

  2. Generate an API Key. The following steps require your NGC API key.

  3. Authenticate local Docker with NGC by running the following code. For more details, see the NGC authentication documentation.

docker login nvcr.io
Username: $oauthtoken
Password: <NGC API key>

  1. Set the NGC_API_KEY environment variable in your shell.

export NGC_API_KEY=<NGC API key>

  1. Install and Configure NGC CLI

You must install NGC CLI on the host system which will be used to deploy CVDS and start the ingestion pipeline. Follow this guide to download and install NGC CLI - https://docs.ngc.nvidia.com/cli/cmd.html.

Set the NGC configuration file:

ngc config set

You will be asked to enter information about the following properties:

  • Enter API key []. Choices: [<VALID_APIKEY>, ‘no-apikey’]:

  • Enter CLI output format type []. Choices: [‘ascii’, ‘csv’, ‘json’]:

  • Enter org []. Choices: [<org1>, <org2>]:

  • Enter team []. Choices: [<team1>, <team2>]:

  • Enter ace []. Choices: [‘no-ace’]:

Finally, a NGC config will be saved with the following output on the terminal:

Validating configuration...
Successfully validated configuration.
Saving configuration...
Successfully saved NGC configuration to /path/to/home/.ngc/config

Quickstart Guide#

Choose the model#

Multiple versions of the model are available for download.

Model Precision

Model name

Model Tag

Model Size

FP32

nurec-fixer

0.1_fp32

17 GB

BF16

nurec-fixer

0.1_bf16

8.5 GB

Model Requirements#

Input Format

RGB

Input Resolution

1024 x 576

Input Precision

FP32, BF16

Output Format

RGB

Output Resolution

1024 x 576

Output Precision

FP32

Download the model#

Run the following command to download the model:

ngc registry model download-version "nvidia/nre/<model name>:<model tag>" --org <org>

Download PyTorch container#

Run the following command to download the docker container:

docker pull nvcr.io/nvidia/pytorch:24.10-py3

Launch the docker#

Run the following command to launch the docker container:

docker run --rm --runtime=nvidia --gpus all --shm-size 2g \
    -v /path/to/working/directory:/path/to/working/directory:rw \
    -it nvcr.io/nvidia/pytorch:24.10-py3

Note

Remember to change the “/path/to/working/directory” to the correct path on your machine.

Check the model quality#

Inference script “infer.py” is provided along with the model weights to run the model inference on a set of input images.

The inference script takes a directory of PNG images, location of the model weight and generates PNG images as output.

Run the following command to run the inference script inside the docker container:

python3 infer.py --image_path /path/to/images/ --output_path /path/to/output/ --model_path /path/to/cosmos_difix.pt
On this page