This section describes Camera, Grid/Matrix/Image, and Robot coordinate frames.

Values on a grid \(G\) are stored such that the grid cell stores the value at the coordinate \(i \times d\ \)where \(d\) is the spacing between sample points. This means \(G\left\lbrack i \right\rbrack = f\left( i d \right)\). If \(y = f\left( x \right)\) shall be retrieved in “nearest” interpolation mode, \(x\) is rounded to the nearest index \(i\) and thus \(y = G\left\lbrack i \right\rbrack\). In “linear” interpolation mode \(x\) is mapped to the lower index \(i_0\) and thus \(y = \left( 1 - p \right) G\left\lbrack i_0 \right\rbrack + p G\left\lbrack i_0 + 1 \right\rbrack\).

Coordinates for matrices are defined in the order (row, column). This means the first coordinate indicates the row and the second coordinate the column. It does not necessarily mean that the storage order is row- or column-major. Eigen uses row-major storage by default.

Images are seen as matrices. Pixel coordinates for images are defined in the same (row, column) order which is used for matrices. To avoid confusion Isaac normally does not use x or y to refer to pixel coordinates, but use row or column instead. Images are always stored in column-major storage order. The pixel with coordinate (0,0) is in top left corner.

The vector order is: - 0: row - 1: column

A 3D point (x,y,z) in the camera frame is represented as depicted in the diagram below. The x-axis points to the right, the y-axis points downwards, and the z-axis points out of the camera into the world. If a 3D point in camera coordinates is projected onto an image the coordinates are switched to image coordinates which follow the (row, column) order as described above.

The vector order is: - 0: right - 1: down - 2: forward

The robot coordinate frame for a mobile base is chosen such that the x-axis points forward, the y-axis points to the left and the z-axis points upwards. The center of the coordinate frame is generally placed on the ground and in the center of rotation of the robot. For Carter the robot coordinate frame is centered between the two main wheels.

The vector order is:

0: forward

1: left

2: up