The NVIDIA® Static Camera Calibration tutorial describes how to perform intrinsic and extrinsic calibration for a vehicle's cameras using the DriveWorks Static Calibration Tools.
The step-by-step tutorial covers the following:
The following cameras and targets are required before proceeding in this tutorial.
The following cameras are required for calibration:
This tutorial requires the use of targets to calibrate the vehicle's cameras.
The targets used in this tutorial can be found in the following folder:
data/tools/calibration/aprilTargets
An example of the target used in this tutorial:
The targets must be printed as follows:
data/tools/calibration/aprilTargets
.data/tools/calibration/aprilTargets
.For each target, measure the length of the horizontal bar in millimeters (mm) and note it in the corner, as demonstrated in the above image.
Depending on the number and location of the cameras on the vehicle, different scene setups may be used.
In this tutorial, a setup with 12 cameras mounted on a car roof is used as an example.
Ensure the following constraints when placing targets during scene setup:
In this tutorial, a set of 8 large and 4 small targets are used as an example.
The following types of data must be captured for static camera calibration:
To capture data for intrinsic camera calibration:
format=h264,output-format=yuv
as camera sensor properties. The calibration tool will automatically extract all valid frames from the recorded video.
Refer to 4.2 Intrinsic Camera Calibration.
To capture data for extrinsic camera calibration, use one of the following methods to capture one image from each car camera:
Use the DriveWorks Recorder Tool to simultaneously record video from all cameras and ffmpeg to extract one frame from the video.
ffmpeg -i \<folder\>/camera_X.h264 -t 1 -f image2 camera_X.png
Use the DriveWorks Sample sample_camera_multiple_gmsl
to capture a frame.
./sample_camera_multiple_gmsl --selector-mask=111111111111
Once you have followed one of the two methods described above, rename and copy the files to an appropriate location as indicated by the DriveWorks Calibration Tools.
In addition to capturing frames from the vehicle's cameras, a set of images also needs to be captured with the external camera.
These images help construct a calibration graph as they introduce a link between targets not observable by the car cameras.
These targets can include ground targets, wheel targets, and between targets of individual cameras.
It is recommended you capture at least 20 images with the external camera, so that all possible target pairs are recorded.
Ensure these images simultaneously observe both the ground targets, and the wheel targets.
The following demonstrates the types of images to capture with the external camera:
The Static Calibration Tool suite consists of multiple command line tools. This tutorial covers the tools required for static calibration of the vehicle's cameras. There are additional calibration tools for auxiliary support and additional sensor. These are not covered in this tutorial. All tools used in this section are documented in Camera Calibration Tools.
The calibration tools expect a specific directory structure as well as a predefined collection of files:
targets.json is a target database shipped with DriveWorks. It contains all the large targets that can be used in the scene.
Modify this file and change the barLength
fields for all targets in the scene to the length (in meters) which you measured in 1.2.1 Target Prints.
special-targets.json: The targets attached to the wheels and the targets that lie flat on the floor must also be declared in this file.
The structure of this file is as follows:
Ensure that the targets which are on the ground and which are attached to the wheels are correctly identified in this file.
This section requires the usage of the DriveWorks Intrinsics Constraints Tool, to extract intrinsics constraints used during calibration for each individual camera.
The input for this tool is the video recorded in 3.1 Capturing Data for Intrinsic Camera Calibration.
This tool will then export all required constraints and output them in a .json
file to be placed in the /intrinsics/
subfolder in the directory structure.
Run the tool by executing:
./calibration-intrinsics-constraints --use-checkerboard=11x8 --input-video=<video_path>/camera-0.h264 --output=<calib_data_path>/intrinsics/camera-0.json
Run the tool by executing:
./calibration-intrinsics-constraints --targetDB=<calib_data_path>/targets.json --input-video=<video_path>/camera-0.h264 --output=<calib_data_path>/intrinsics/camera-0.json
The tool will open a window playing back the input video and indicate with a red or green border if a new intrinsic constraint has been collected (i.e. target or checkerboard has been found).
This section requires the usage of the DriveWorks Graph Calibration Tool, to construct a graph-based representation of the data obtained, which is then further optimized.
Running this tool without parameters assumes you have the file structure specified in 4.1 Directory Structure.
The tool determines the names of the cameras from the extrinsics
folder. It then searches the intrinsic
folder for .json
files corresponding with the intrinsics constraints.
The images captured with the external camera will also be used for intrinsic calibration for the external camera, and for extrinsics constraints.
The Calibration Tool will then use these images to construct a graph representation which is later optimized.
Ensure the following before proceeding:
\<calib_data_path\>/extrinsics
folder. The image file .json
file names generated in 4.2 Intrinsic Camera Calibration.\<calib_data_path\>/external
folder.Run the tool by executing the following:
./calibration-graph-cli --dir=<calib_data_path>
The tool outputs progress to the console. Take note of warnings in yellow and errors in red as they indicate possible problems with the calibration.
The following outputs are placed in the target folder:
calibrated-graph.json
is the main output for this tool. This contains all constraints, the calibrated camera models, the camera poses, the target poses, etc.The output is generated for all car cameras, (validation-intrinsics-<camera>.jpg
) and the external cameras.
x = 0
and y = 0
are orange.The output is generated for all car cameras, and each external image (validation-<camera>.jpg
).
Validation images can be used to double check if the computed calibration is valid, i.e., if the green re-projections match the targets.
In the following image, a green mask clearly overlays the targets, and the corners are well aligned. This indicates a valid computed calibration.
In the following image, a green mask is misaligned with the target. This indicates the calibration results might not be correct, or that there is a problem with the calibration target.
The Graph Calibration Tool also provides a calibrated-graph.json
file as output. This file contains intermediate
calibration results which cannot yet be consumed by the NVIDIA® DriveWorks SDK. These results can be converted using the Calibrated Graph Tool into a valid DriveWorks SDK rig.json
file representing the full calibrated camera rig configuration, and can be directly consumed by the DriveWorks modules, e.g., Rig Configuration.
To perform the conversion, run the tool by executing the following:
./calibration-graph-to-rig --graph=<calib_data_path>/calibrated-graph.json
The tool generates a rig.json
file in the current folder. This file lists all cameras used during calibration with their final intrinsic and extrinsic calibration data.
If an existing rig.json
is passed as input, all camera entries are modified with the new calibration results.
The Static Calibration Tool suite supports calibration for intrinsic parameters only, if extrinsic calibration is not desired or required.
The procedure is similar to the full calibration workflow, with several modifications:
intrinsics
folder and targets.json
file..json
files for each camera.calibrated-graph.json file
in the input directory, and intrinsic validation images as demonstrated in the section. Verify that these validation images meet the requirements described above.rig.json
file with the result. Refer to 4.4 Generating DriveWorks `rig.json` File for more information.