# ORB¶

This gem provides a feature detector and descriptor extractor.

Features are used in applications such as:

• 3D reconstruction using structure-from-motion
• Visual odometry (motion tracking) & SLAM
• Content-based image retrieval
• Image alignment & panorama stitching

“ORB” stands for “Oriented FAST and rotated BRIEF”. This shows that ORB is based on FAST, a feature detector, and BRIEF, a binary descriptor.

The original publication by Rublee, et al., titled “ORB: An efficient alternative to SIFT or SURF”, can be found here: http://www.willowgarage.com/sites/default/files/orb_final.pdf

ORB has the following key qualities compared to other feature types:

• Resistant to image noise
• Rotation invariant
• Multi-scale

The Isaac SDK ORB gem follows the original publication closely. In addition to that, it also:

• Implements most of the pipeline in CUDA for higher performance by running on the GPU
• Adds a spatial regularization (“grid filtering”) step, which fixes a fundamental flaw of other implementations (e.g., OpenCV’s) by distributing keypoints over the image more evenly. This has significant benefits for applications such as motion tracking.

The algorithm steps are as follows:

1. Downsample input image to different scale levels
2. Extract FAST features on all levels
3. Apply grid filtering
4. Extract feature orientation
5. Extract descriptors

## Types Provided by the Gem¶

### Keypoints¶

A Keypoint has the following properties:

• x, y: Keypoint coordinates in the image it was extracted from. Note: Since there are multiple scale levels, use the utility function GetCoordsAtTopLevel to get the coordinates at the topmost (initial) image level.
• response: “Strength” of a feature. A higher score means a higher local contrast.
• angle: Orientation angle (in radians)
• scale: Radius of the support region (in pixels)
• level: Scale level from which the feature was extracted. Starts at zero.

The Keypoints type is a vector of keypoints.

### Descriptors¶

A Descriptor has the following properties:

• id: An ID that allows connecting a descriptor back to the keypoint from which it originated
• elements: The binary descriptor data, packed into a small buffer of elements.

The Descriptors type is a vector of descriptors.

## How to Use the Gem (Interface)¶

The gem provides a single function, ExtractOrbFeatures, which is used to extract ORB features and descriptors from an image.

As an input parameter, an image has to be passed.

As an output parameter, Keypoints and Descriptors objects have to be passed.

These are the additional parameters that can be set:

• Max. features (default 1500): The desired number of best features to retain.
• FAST threshold (default 30): A threshold for the minimum local contrast a pixel has to exhibit in order to be considered a feature
• Grid number of cells linear (default 8): How many bins to use for spatial regularization in horizontal / vertical direction. The total number of bins is the square of the parameter. A higher number means spreading features more evenly. Setting to 1 disables the regularization.
• Downsampling factor (default 0.7): The size of the subsequent scale level, as a ratio of the current level. Values between 0.5 (inclusive) and 1.0 (exclusive) are allowed.
• Max. levels (default 4): How many scale levels to use.

## Building the package¶

Run the following command to build the gem, the codelet, and the sample application:



### Embedded Jetson Device¶

1. Deploy //packages/orb/apps/orb_demo:orb_demo-pkg to the robot as explained in Deploying and Running on Jetson.
2. SSH into the device with the following command:
$ssh nvidia@ROBOTIP  1. Run the demo application on the device with the following command, after changing to the deployed directory: $ cd orb_demo-pkg/
\$ ./packages/orb/apps/orb_demo/orb_demo