Omniverse IsaacSim

Omniverse IsaacSim is built on the NVIDIA Omniverse simulation platform. It leverages the NVIDIA Omniverse Kit and has been enhanced with robotics-specific extensions. It provides access to NVIDIA PhysX and RTX ray-tracing technologies for high performance, photo-realistic robotic simulations.

Isaac SDK integrates with Omniverse IsaacSim through the Robot Engine Extension. Sensor data and drive commands are exchanged though TCP. Isaac SDK provides navigation and manipulation sample applications that work with Omniverse IsaacSim.

Download Omniverse Isaac Sim from Isaac Sim Downloads and follow instructions there to set up the simulator.

Note

For the samples below, the full Omniverse Isaac Sim 2020.1 release is required; the preview release does not support Isaac SDK. If you see the preview release on the above page, the full release is not available yet. Please check back after Jun 6, 2020.

The physics simulation setting can be access in Physcics > Settings. The default physics settings of 60 timesteps per second and maximum 2 steps per frame provide sufficient accuracy for differential base simulation when the simulation framerate is higher than 30Hz. If the framerate drops below 30Hz, increase the max num steps to 3. This setting will persist after restarting Omniverse Kit.

Note

Increasing the maximum number of physics steps per frame will further decrease the framerate.

kit-phyx-max-step.png

Camera simulation in Omniverse IsaacSim currently only publishes the viewport camera. To provide valid first-person robot camera simulation data to Isaac SDK, switch the viewport camera to the robot camera and change the viewport resolution to 1280x720 to match the expected camera resolution for most Isaac SDK perception modules. This setting will not persist after restarting kit.

kit-viewport.png

Warehouse Navigation with Carter

Launch Omniverse Kit and open the omni:/Isaac/Samples/Isaac_SDK/Scenario/carter_warehouse_with_forklift.usd stage. Start the Robot Engine Bridge and press Play.

Run the Isaac SDK application with the Carter navigation stack:

Copy
Copied!
            

bob@desktop:~/isaac$ bazel run apps/navsim/navsim_navigate -- --map_json apps/assets/maps/virtual_test_warehouse_1.json

Once connection is established between Isaac SDK and Omniverse Kit, open Isaac Sight at http://localhost:3000. In the Application Configuration panel on the right, search for “goal_behavior” and change the “desired_behavior” in “SelectorBehavior” from pose to random. Carter will start navigating to randomly generated goal positions.

kit-carter-test-warehouse.png


Cart Delivery

Launch Omniverse Kit and open the omni:/Isaac/Samples/Isaac_SDK/Scenario/str_dolly_warehouse_with_forklifts.usd stage. Start the Robot Engine Bridge.

Note

Before you continue, follow the steps in Simulator Settings to complete these setting changes:

1. Increase “Max num steps” for physics to 3: Once the perception model in the cart_delivery application is running, GPU usage will increase and the Kit framerate will drop. Increasing the “Max num steps” for physics is necessary to keep the differential base simulation accurate.

2. Change the viewport camera to first person and “Render Resolution” to 1280x720: The sample application needs to detect the dolly pose from the robot camera before driving underneath, so Kit needs to publish images from the first-person camera at the expected resolution for the perception models.

Press Play in Omniverse kit. Run the Isaac SDK application with the cart-delivery stack:

Copy
Copied!
            

bob@desktop:~/isaac$ bazel run packages/cart_delivery/apps:cart_delivery -- --map_json apps/assets/maps/virtual_test_warehouse_1.json --pose2_planner ""

Once connection is established between Isaac SDK and Omniverse Kit, open Isaac Sight at http://localhost:3000. The robot should detect the dolly and move it to a predefined drop-off waypoint.

kit-str-test-warehouse.png


See Manipulation Sample Applications to learn how to use Isaac SDK manipulation sample applications with Omniverse Isaac Sim.

© Copyright 2018-2020, NVIDIA Corporation. Last updated on Feb 1, 2023.