Given a horizontal stereo rig, the right camera's extrinsic pose calibration with respect to the left camera is represented by both orientation and position parameters.
Orientation parameters are defined as roll, pitch and yaw, while position parameters are defined as x, y, z. The orientation parameters identify the rotation that aligns the right camera orientation to the left camera orientation. The position parameters identify the translation between the right camera position and the left camera position. All six parameters are crucial in tasks such as stereo rectification and disparity estimation that rely on the relative pose of the right camera with respect to the left camera.
NVIDIA® DriveWorks uses visually matched features to calibrate the right camera's orientation and position in a self-calibration setting. While orientation is calibrated fully, position is only calibrated up to norm, i.e., up to the relative translation between the two cameras specified in the rig file. All six parameters are calibrated simultaneously by estimating the transformation that best satisfies the epipolar constraint on the feature matches.
The features obtained from the left image are matched to those obtained from the right image. Using the well-known epipolar constraint, the relative transformation between the left and the right camera is estimated. This relative transformation provides roll, pitch, yaw, as well as x, y, z components of the right camera with respect to the left camera.
The instantaneous calibration for an image pair is accumulated in histograms for the calibration's roll, pitch, yaw, x, y, z components to estimate a final calibration in a robust way.
Several hours of data are used to produce a reference calibration value for cross-validation. Then, short periods of data are evaluated for whether they can recover the same values. For example, the graph below shows precision/recall curves of stereo self-calibration. Precision indicates that an accepted calibration is within a fixed precision threshold from the reference calibration, and recall indicates the ratio of accepted calibrations in the given amount of time.
The following code snippet shows the general structure of a program that performs stereo self-calibration
This workflow is demonstrated in the following sample: Stereo Calibration Sample