Isaac SDK Developer Guide¶
The Isaac SDK is the main software toolkit for NVIDIA robotics, and is comprised of the following:
- Isaac Robot Engine: A framework which allows you to easily write modular applications and deploy them on your robots.
- Isaac GEMs: A collection of robotics algorithms from planning to perception, most of them GPU-accelerated.
- Applications: Various example applications from basic samples which show specific features to applications that facilitate complicated robotics use cases.
The following diagram shows the Isaac SDK ecosystem:
The Isaac SDK comes with a C++ and Python toolchain based on the Bazel build system for building and
deploying applications. You can build and run applications with a command as simple as
bazel run //apps/samples/realsense_camera. All external dependencies are pulled automatically
to your system and all necessary files are compiled. The Setup section of this document
explains the few steps necessary for getting started.
Isaac Robot Engine¶
Isaac SDK includes the Isaac Robot Engine, a feature-rich framework for building modular robotics applications. With Isaac you can build an application out of small components which pass messages between each other and which can be configured to your custom use case. The Understanding Codelets section of this document gives a brief introduction on how to use components.
Many useful robotics libraries already exist, and Isaac integrates many of these, such as CUDA, NPP, OpenCV, ROS, PCL (wip), Eigen, and others.
Isaac is optimized for NVIDIA hardware like Jetson Xavier, Jetson TX/2, Jetson Nano, and workstation GPUs. You can deploy a cross-compiled application optimized for your platform with a single short command line script.
Robotics often combines many different disciplines including low-level hardware drivers, safe planning algorithms, fast and accurate computer vision, and high-level artificial intelligence. All of these disciplines have different challenges and success often require years of expertise.
The Isaac SDK comes with a collection of high-performance algorithms to solve challenging robotics problems. These algorithms are known as GEMs. Most importantly, Isaac provides integration of deep neural networks like stereo depth estimation and object detection.
Isaac includes GEMs for support of important hardware components and robotic peripherals.
The Isaac SDK prioritizes navigation functionality in the current release (2019.1), but other domains are planned to be added or enhanced in future releases. See the GEMs section of this document for further information.
Various sample applications are provided which highlight features of the Robot Engine or focus on the functionality of a particular GEM. These sample applications are good starting points for learning Isaac.
The Isaac SDK is meant for development of applications for complicated use cases like a delivery robot. The Carter application gives you a starting point for building your own delivery robot. Carter can drive to a goal location, patrol in a building, or bring you popcorn. The navigation stack of Carter is based on a Lidar.
To start on a smaller scale, see the Kaya application. Isaac includes instructions on how to build your own robot and bring it to life with artificial intelligence.
To ask and answer questions in the Isaac SDK developer community, visit the Isaac SDK Forum.
Table of Contents¶
- Getting Started
- How to Get Started With Jetson Nano
- Developing Codelets in C++
- Developing Codelets in Python
- Frequently Asked Questions
- Introduction to Isaac Robotics Engine
- Understanding Codelets
- Record and Replay
- Machine Learning Workflow
- Coordinate Frames
- The Isaac Scheduler
- Navigation Stack
- Object Detection Pipeline
- Stereo Depth DNN
- Stereo Visual Odometry
- Isaac Warp Services
- Map Editor
- Sensors and Other Hardware
- ROS Bridge Overview and Workflow
- NVIDIA Carter
- NVIDIA Kaya
- Wire the BMI160 IMU to the Jetson Nano or Xavier