Use the NuRec Render Command

To render views from a trained NuRec model, use the render command. The render command takes a USDZ file as the model input and doesn’t require the original input clip. You can use it instead of rendering through the NuRrec gRPC API.

The render command supports rendering from a training trajectory, or from a trajectory that is at a fixed rotation/translation away from each training pose. It does not currently support user-fed trajectory and intrinsics. It is limited to the intrinsics of a training camera, and you must pass the camera name when you run the command.

For example, to render from the original training views, you would use the following command (with the proper USDZ file path and output directories set):

docker run --shm-size=64g -it --rm --gpus all \
    --network host \
    --volume /path/to/output/folder:/workdir/output \
    nvcr.io/nvidia/nre/nre-ga:latest \
    -- render \
    --artifact-path "${{USDZ_PATH}}" \
    --output-dir "${{RENDER_OUTPUT_DIR}}" \
    --frame-step 1 \
    --image-scale 0.25 \
    --camera-id camera_front_wide_120fov \
    --camera-id camera_front_tele_30fov \
    --image-format png \
    --frame-naming frame-end-timestamp \
    --replicate-training-views

The --replicate-training-views option ensures that the trained per-frame ISP parameters are applied, otherwise the pixel intensities and colors may differ from those in the training images. Use --no-replicate-training-views if you are planning to apply transformations, generating novel views, using the --rolling-shutter-duration override, or passing the --enable-editing-actors flag.

Apply fixed transformations in the vehicle space (see the conventions) to every pose of the rig trajectory by appending the following options.

  --rig-rotation-offset <yaw> <-roll> <pitch>  --rig-translation-offset <tx> <ty> <tz>

The --enable-editing-actors flag allows modifications to dynamic actor poses and only renders the dynamic actors present within the current frame’s start and end timestamp. The --demo-actor-transform flag applies a precomputed transformation (+1m Z-lift) to discriminate the editable (dynamic) actors by lifting them from the road surface in the rendered images. If you’re passing the --demo-actor-transform flag, you must also pass the --enable-editing-actors flag.

For more details on available options for the render command, pass the --help option.

To render NuRec models through the NuRec gRPC service, see Use the NuRec gRPC API Server.