These notes outline new features and improvements in Isaac SDK version 2020.1
The Bazel version used by Isaac SDK has been updated from 0.19.1 to 2.0.0.
Update Instructions: Use the
engine/build/scripts/install_dependencies.shscript in Isaac SDK to upgrade Bazel to version 2.0.0. To perform a manual update, see the GitHub page for the Bazel installer.
The performance of the Bazel build process has improved.
The Python toolchain now uses Python3 by default. Use
--python_version=PY2to switch to Python2.
External workspace references are now handled correctly.
Unity Editor 2019.3.0f6
IsaacSim now uses Unity Editor version 2019.3.0f6 and the Vulkan graphics API. Follow the steps below to update Unity Editor.
After you update the Unity Editor, you will no longer be able to use previous versions of Unity Editor with Isaac SDK.
If you don’t have the Unity Hub, download it as described on the Isaac SDK Setup page.
In the Unity Hub, select Installs on the left, then select Add.
Locate and install Unity 2019.3.0f6.
The first time you open a Unity project after updating the Unity Editor, you will be prompted to upgrade the project to Asset Database Version 2. Click Yes to upgrade. The project will then re-import its assets before opening.
Virtual Factory of the Future (FOF)
IsaacSim Unity3D now features a complete sample scene around a virtual Factory of the Future.
The high-definition render pipeline (HDRP) was used to create realistic visuals close to the real factory.
The scene includes various new 3D models for the factory of the future: carts, shelves, etc. These assets can be moved around the scene as needed to reproduce different setups.
The scene is provided as a binary release in this release.
Along with this simulation environment, Isaac SDK offers a full-stack sample app for cart delivery, which has been tested and fine-tuned for the virtual Factory of the Future.
This is a complete app that combines cart detection, pose estimation, localization, object avoidance, a planner, control, etc.
The robot autonomously picks up a cart and delivers it to a chosen waypoint using a behavior tree package available at
Improved 3D Object Pose Estimation
Isaac SDK features a refined 3D object pose estimation pipeline using a CNN autoencoder architecture.
The pose estimation DNN outputs an object segmentation mask.
This application performs pose estimation without a codebook.
RL Docking Policy
Isaac SDK has a new docking policy for cart pickup that uses reinforcement learning.
The policy is trained in a Gym-like training environment with 9 robots per simulator.
There is a sample application for inference in simulation.
Isaac SDK provides a new Python API for developing applications.
You can now define your own costmaps to customize the behavior of the robot path planner with the following areas:
Restricted areas: Where the robot is not allowed
Penalty areas: Where the robot should normally not go
Highway lanes: To guide the robot through large maps with left/right traffic rules
3D Object Pose Estimation for boxes
The object pose estimation pipeline now supports pose estimation for boxes. Currently, this feature is supported in simulation only.
Detection of small (non-foldable) boxes from different sides is possible, as well as dirty surfaces (glue, stickers, etc.).
Strong light reflections, translucency, and differently typed and colored boxes are not accounted for yet.
Motion Planning for Manipulation
This release contains an arm motion planner based on the LQR algorithm, new messages for arm motion planning tasks, and other utility components.
Capabilities for multi-joint motion planning are included in this release, including an interface to a simulation environment for testing and development. However, obstacle avoidance is not yet available for 6 DoF environments.
The navigation stack now supports a second and optional flatscan data source for localization and obstacle avoidance. In particular, data from two different LIDARs can be
processed by the Particle Filter Localization algorithm for more stable localization.
combined into a single Local Map for safer obstacle avoidance.
The navigation stack can be easily extended to support more LIDARs if needed. The NavSim subgraph has also been extended to accept and connect a second optional LIDAR data source from simulation.
Object Detection from Simulation in Docker
Isaac SDK now provides a full pipeline for training object detection with inference.
Improved Visual Odometry
Visual Intertial Odometry now has additional visual diagnostics.
The functionality of Isaac nodes has been revised.
stop()functions have been moved from the component class to the codelet class.
Codelets within the same node no longer tick in parallel: If a codelet is ticking, it will prevent other codelets in the node from ticking. To run codelets in parallel, you must now place them in separate nodes.
The Isaac engine now generates a log warning when a codelet tick is being blocked for a long period of time.
Image and Tensor Type Changes
The Image type now contains a tensor, with a shape of
(X, X, N), which you can use to
perform tensor operations on the image.
The SampleCloud type is now also based on the Tensor type, with a shape of
The Tensor type now allows fixed and dynamic dimensions, similar to Eigen.
The Tensor type now allows basic slicing on the last and first dimensions.
The Tensor type is now compatible with Eigen functions.
Map View Fix
The Map View window in Websight now renders correctly.
The TensorListProto message type has been removed from Isaac SDK.
Tensors are now passed as individual message channels. The channel name describes the tensor data; channel names are stored in a dictionary of tensors.
This change improves the reusability of codelets that process tensors.
ML codelets now work with an arbitrary number of input/output channels.