The NVIDIA® DriveWorks SDK includes the following modules.
| Module | Description |
|---|---|
| Core | Provides features for most applications, including context handles, system/platform information, and memory management. |
| Module | Description |
|---|---|
| Sensors | Accesses sensors by utilizing a hardware abstraction layer. |
| Module | Description |
|---|---|
| Rig Configuration | Reads and enumerates pre-calibrated car and sensor properties. |
| VehicleIO | Actuates the vehicle and provides information regarding its current state. |
| Module | Description |
|---|---|
| Image | Performs image abstractions, streamers, and format conversions. |
| Image Transformation | Provides scaling utilities. |
| Camera Color Correction | Adjusts the image color captured from cameras to match a reference camera's statistical properties. |
| Connected Components | Applies unique labels for connected components in a grey scale input image. |
| Rectifier | Projects images acquired from an input camera model to an output camera model. |
| Features | Provides feature detection utilities. |
| Filtering | Provides a Gaussian pyramid image representation. |
| Tracking | Detects and tracks feature points or templates between frames from an input camera. |
| Stereo | Provides stereo rectification for images acquired from a calibrated stereo camera. It also estimates the disparity between two rectified images on a confidence map. |
| Structure from Motion (SFM) | Reconstructs the 3D structure of a scene given a moving camera rig. |
| Dense Optical Flow | Estimates motion vectors between frames. |
| Module | Description |
|---|---|
| Point Cloud Processing | Provides algorithms for low level point cloud processing. |
| Module | Description |
|---|---|
| Data Conditioner | Provides image pre-processing functionalities to modify the network input format. |
| DNN | Runs inference on pre-trained deep neural networks. |
| Clusterer | Implements a DBSCAN data clustering algorithm to group bounding boxes with similar properties. |
| Module | Description |
|---|---|
| Intrinsic Camera Models | Initializes camera models to provide pixel-to-ray and ray-to-pixel transformations. |
| Self-Calibration | Correction of nominal calibration based on up-to-date sensor readings. It supports cameras, IMUs, Radars, Lidars, and Steering. |
| Egomotion | Tracking and prediction of a vehicle's pose, based on a motion model. |
| Module | Description |
|---|---|
| IPC | Provides inter-process communication between various platforms. |
| Module | Description |
|---|---|
| Renderer | View renderer for primitives and projections. |
The Perception modules provide detection and classification capabilities for obstacles, paths, and wait conditions. They utilize Deep Neural Network (DNN) solutions
to enable situational awareness for the ego-vehicle's surroundings. These surroundings include obstacles, paths, intersections, traffic lights, traffic signs, and camera obstruction.
| Module | Description |
|---|---|
| Object Detector | Pipeline creation for objects detection from camera images. Network Used: DriveNet, which detects cars, trucks, traffic signs, traffic lights, bicycles and pedestrians. |
| Freespace Perception | Detects drive-able collision-free space. Network Used: OpenRoadNet, which detects drive-able space on the road without collision. |
| Module | Description |
|---|---|
| Landmark Perception | Detects lane markings and poles from camera images. Network Used: MapNet : Detects multiple landmark types. |
| Path Perception | Detects drive-able paths from camera images. Network Used: PathNet : Detects the ego-path, and left and right adjacent paths when present. |
| Pilotnet Detector | Predicts trajectories for lane keeping, splits and lane changes from camera images. Network Used: PilotNet : Predicts the drive-able path for various maneuvers in world coordinates. |
| Module | Description |
|---|---|
| Wait Conditions Detection and Classification | Detects and classifies traffic lights, signs, and intersections from camera images. Networks Used: LightNet : Classifies traffic lights and their different signals. SignNet : Classifies different traffic signs including speed limit, stop signs, and more. WaitNet : Detects traffic signs, lights, and intersections. |
| Module | Description |
|---|---|
| Camera-based Headlight Controller | Camera based headlight/Low-Beam controller module. |
| ClearSightNet | Detects camera obstruction and blindness. |
The Mapping modules support mapping use cases including accessing HD maps and localizing the vehicle to a map.
| Module | Description |
|---|---|
| Maps | Access to HD maps. |
| Maps Renderer | Provides rendering for dwMaps. |
| Localization | Localization to HD maps using camera data. |
The Planning modules support the planning and control of the vehicle's motion.
| Module | Description |
|---|---|
| Lane Planner | Computes a path from a starting location to a target location. |
| Safety Force Field | Safety Force Field computation for obstacle avoidance. |