BundleAdjustment#
Functions#
- NVCVStatus pvaBundleAdjustmentCreate(NVCVOperatorHandle *handle, NVCVTensorRequirements const *const rigPoses, NVCVTensorRequirements const *const points3d, NVCVTensorRequirements const *const measurements, NVCVTensorRequirements const *const cameraBitmask, NVCVTensorRequirements const *const rig2cam, NVCVTensorRequirements const *const updatedRigPoses, NVCVTensorRequirements const *const updatedPoints3d, NVCVTensorRequirements const *const finalCost, NVCVTensorRequirements const *const iterationsUsed, PvaBundleAdjustmentConfig const *const config)
Constructs an instance of the Bundle Adjustment operator.
- NVCVStatus pvaBundleAdjustmentSubmit(NVCVOperatorHandle handle, cupvaStream_t stream, NVCVTensorHandle rigPoses, NVCVTensorHandle points3d, NVCVTensorHandle measurements, NVCVTensorHandle cameraBitmask, NVCVTensorHandle rig2cam, NVCVTensorHandle updatedRigPoses, NVCVTensorHandle updatedPoints3d, NVCVTensorHandle finalCost, NVCVTensorHandle iterationsUsed, PvaBundleAdjustmentParams const *params)
Submits the Bundle Adjustment operator to a cuPVA stream.
- NVCVStatus pvaBundleAdjustmentSubmit(NVCVOperatorHandle handle, cudaStream_t stream, NVCVTensorHandle rigPoses, NVCVTensorHandle points3d, NVCVTensorHandle measurements, NVCVTensorHandle cameraBitmask, NVCVTensorHandle rig2cam, NVCVTensorHandle updatedRigPoses, NVCVTensorHandle updatedPoints3d, NVCVTensorHandle finalCost, NVCVTensorHandle iterationsUsed, PvaBundleAdjustmentParams const *params)
Submits the Bundle Adjustment operator to a CUDA stream.
Functions#
- NVCVStatus pvaBundleAdjustmentCreate(
- NVCVOperatorHandle *handle,
- NVCVTensorRequirements const *const rigPoses,
- NVCVTensorRequirements const *const points3d,
- NVCVTensorRequirements const *const measurements,
- NVCVTensorRequirements const *const cameraBitmask,
- NVCVTensorRequirements const *const rig2cam,
- NVCVTensorRequirements const *const updatedRigPoses,
- NVCVTensorRequirements const *const updatedPoints3d,
- NVCVTensorRequirements const *const finalCost,
- NVCVTensorRequirements const *const iterationsUsed,
- PvaBundleAdjustmentConfig const *const config,
Constructs an instance of the Bundle Adjustment operator.
Bundle adjustment jointly refines a set of rig poses and a set of 3D track (point) positions by minimizing the sum of squared reprojection residuals across every camera that observes each track. The operator runs a full Levenberg-Marquardt (LM) solver on-device: it iteratively builds the reduced (Schur-complement) camera normal-equations system, solves it via Cholesky decomposition, applies the resulting pose/point update, and adapts the LM damping factor (lambda) based on whether the reprojection cost decreased, until convergence or
maxIterations(see PvaBundleAdjustmentParams) is reached.Rig poses tensor (
rigPoses/updatedRigPoses): Data Layout: [N, 16] Extent N: [kMaxCameraCount] Data Type: [32bit Float] Each row is a row-major 4x4 SE(3) world-to-rig pose matrix. A world-frame point is taken into a camera frame asrig2cam* rigPose, i.e. the rig pose maps world -> rig andrig2camthen maps rig -> camera.3D track positions tensor (
points3d/updatedPoints3d): Data Layout: [N, 4] Extent N: [kMaxTrackCount] Data Type: [32bit Float] Each row is a homogeneous 3D point [x, y, z, 1].Measurements tensor (
measurements): Data Layout: [N, C, 2] Extent N: [kMaxTrackCount] Extent C: [kMaxCameraCount] Data Type: [32bit Float] Per-track, per-camera 2D measurement [u, v] in normalized image-plane coordinates: pixel coordinates with the principal point subtracted and divided by the focal length. No camera intrinsics or lens distortion are applied, so measurements must already be undistorted.Camera visibility bitmask tensor (
cameraBitmask): Data Layout: [N] Extent N: [kMaxTrackCount] Data Type: [32bit Signed] One int32 bitmask per track: bit i (for i in [0, cameraCount)) set means the track is observed by camera i; bit 31 set means the track is active (its 3D point is optimized), and bit 31 clear means the track is fixed (its point is excluded from the point update, though its observations still contribute to the cost and to the camera Hessian/gradient).Rig-to-camera extrinsics tensor (
rig2cam): Data Layout: [16] Data Type: [32bit Float] Row-major 4x4 SE(3) transform from the rig frame to the camera frame.Final cost tensor (
finalCost): Data Layout: [1] Data Type: [32bit Float] Robust (Cauchy) cost of the returned state, which is the last accepted LM state.Iterations used tensor (
iterationsUsed): Data Layout: [1] Data Type: [32bit Signed] Number of LM iterations executed, accepted or rejected, in [1, maxIterations].Outputs:
updatedRigPoses:cameras outside the active window keep their input pose; rows fromcameraCounton are zero.updatedPoints3d:only the firsttrackCountrows are written; fixed tracks keep their input [x, y, z], and the homogeneous coordinate is 1.
Limitations:
trackCount(see PvaBundleAdjustmentParams, set at Submit time) must be an exact multiple ofkTrackCountTile(128). Remainder/partial tiles are not supported.cameraCount(set at Submit time) must be at most 21: each track’s visibility is packed into one 32-bitcameraBitmaskentry and bit 31 is reserved for the per-track active/fixed flag.This operator requires PVA hardware generation 3 (e.g. Thor) and is not supported on PVA GEN2 (Orin): its on-chip working set needs a 4th VMEM bank that GEN2 hardware does not provide. Attempting to create or run this operator on a GEN2 platform fails.
All tensors must be densely packed. The DMA transfers use fixed, tightly packed row pitches, and the strides are not validated: a padded tensor gives wrong results, not an error.
One camera model applies to all rig poses: a single shared
rig2cam, no intrinsics and no lens distortion.
- Parameters:
handle – [out] Where the operator instance handle will be written to.
Must not be NULL.
rigPoses – [in] Tensor requirements for the input rig poses tensor (world-to-rig).
Must not be NULL.
points3d – [in] Tensor requirements for the input 3D track positions tensor.
Must not be NULL.
measurements – [in] Tensor requirements for the input per-track, per-camera 2D measurements tensor.
Must not be NULL.
cameraBitmask – [in] Tensor requirements for the input per-track camera-visibility bitmask tensor.
Must not be NULL.
rig2cam – [in] Tensor requirements for the input rig-to-camera extrinsics tensor.
Must not be NULL.
updatedRigPoses – [in] Tensor requirements for the output refined rig poses tensor.
Must not be NULL.
updatedPoints3d – [in] Tensor requirements for the output refined 3D track positions tensor.
Must not be NULL.
finalCost – [in] Tensor requirements for the output final reprojection cost tensor.
Must not be NULL.
iterationsUsed – [in] Tensor requirements for the output number-of-LM-iterations tensor.
Must not be NULL.
config – [in] Create-time configuration (active-camera window). See PvaBundleAdjustmentConfig.
Must not be NULL.
- Return values:
NVCV_ERROR_INVALID_ARGUMENT – Handle, a tensor requirements pointer, or config is null, or a tensor/config field is outside its valid range.
NVCV_ERROR_OUT_OF_MEMORY – Not enough memory to create the operator.
NVCV_SUCCESS – Operation executed successfully.
- NVCVStatus pvaBundleAdjustmentSubmit(
- NVCVOperatorHandle handle,
- cupvaStream_t stream,
- NVCVTensorHandle rigPoses,
- NVCVTensorHandle points3d,
- NVCVTensorHandle measurements,
- NVCVTensorHandle cameraBitmask,
- NVCVTensorHandle rig2cam,
- NVCVTensorHandle updatedRigPoses,
- NVCVTensorHandle updatedPoints3d,
- NVCVTensorHandle finalCost,
- NVCVTensorHandle iterationsUsed,
- PvaBundleAdjustmentParams const *params,
Submits the Bundle Adjustment operator to a cuPVA stream.
- Parameters:
handle – [in] Handle to the operator.
Must not be NULL.
stream – [in] Handle to a valid cuPVA stream.
rigPoses – [in] Input rig poses tensor handle (world-to-rig).
points3d – [in] Input 3D track positions tensor handle.
measurements – [in] Input per-track, per-camera 2D measurements tensor handle.
cameraBitmask – [in] Input per-track camera-visibility bitmask tensor handle.
rig2cam – [in] Input rig-to-camera extrinsics tensor handle.
updatedRigPoses – [out] Output refined rig poses tensor handle.
updatedPoints3d – [out] Output refined 3D track positions tensor handle.
finalCost – [out] Output final reprojection cost tensor handle.
iterationsUsed – [out] Output number-of-LM-iterations tensor handle.
params – [in] Submit-time Levenberg-Marquardt tuning parameters. See PvaBundleAdjustmentParams.
Must not be NULL.
- Return values:
NVCV_ERROR_INVALID_ARGUMENT – Some parameter is outside valid range.
NVCV_ERROR_INTERNAL – Internal error in the operator, invalid types passed in.
NVCV_SUCCESS – Operation executed successfully.
- NVCVStatus pvaBundleAdjustmentSubmit(
- NVCVOperatorHandle handle,
- cudaStream_t stream,
- NVCVTensorHandle rigPoses,
- NVCVTensorHandle points3d,
- NVCVTensorHandle measurements,
- NVCVTensorHandle cameraBitmask,
- NVCVTensorHandle rig2cam,
- NVCVTensorHandle updatedRigPoses,
- NVCVTensorHandle updatedPoints3d,
- NVCVTensorHandle finalCost,
- NVCVTensorHandle iterationsUsed,
- PvaBundleAdjustmentParams const *params,
Submits the Bundle Adjustment operator to a CUDA stream.
Note
CUDA stream support requirements:
PVA SDK 2.7.0 or later
Jetpack 7 or later
DriveOS 7 or later
x86 Emulator is not supported
- Parameters:
handle – [in] Handle to the operator.
stream – [in] Handle to a valid CUDA stream.
rigPoses – [in] Input rig poses tensor handle (world-to-rig).
points3d – [in] Input 3D track positions tensor handle.
measurements – [in] Input per-track, per-camera 2D measurements tensor handle.
cameraBitmask – [in] Input per-track camera-visibility bitmask tensor handle.
rig2cam – [in] Input rig-to-camera extrinsics tensor handle.
updatedRigPoses – [out] Output refined rig poses tensor handle.
updatedPoints3d – [out] Output refined 3D track positions tensor handle.
finalCost – [out] Output final reprojection cost tensor handle.
iterationsUsed – [out] Output number-of-LM-iterations tensor handle.
params – [in] Submit-time Levenberg-Marquardt tuning parameters. See PvaBundleAdjustmentParams.
Must not be NULL.
- Return values:
NVCV_ERROR_INVALID_ARGUMENT – Some parameter is outside valid range.
NVCV_ERROR_INTERNAL – Internal error in the operator, invalid types passed in.
NVCV_SUCCESS – Operation executed successfully.