One of the two main goals of the NVIDIA robotics project is realistic simulation of robots in their environments. The main advantage of simulation is that robotics algorithms and systems can be developed, tested and evaluated without actual robotic hardware. This accelerates development and improves scenario coverage considerably.

With Isaac Sim you can do the following:

  • Simulate robot dynamics to test control algorithms

  • Simulate robot sensors to produce photorealistic camera, depth and segmentation images, LIDAR, IMU

  • Simulate different environments and scenarios to test algorithms under different conditions and circumstances

  • Simulate agents, human actors around the robots that provide interesting dynamic environments

  • Randomize the domain to create huge and diverse training sets with randomly varying objects and environment properties

  • Generate procedures that result in various scenarios for training and evaluation

  • Simulate specific modules of the robotic software stack to target testing and evaluation at specific areas

With Isaac Sim’s high-performance 3D renderer and realistic physics engine, you can test your robot, its interaction with the world, and its sensors, the way you do with actual hardware. All data flows use the same standard messages as used for the real robot, so you can use them interchangeably throughout your application. For example, your simulated camera publishes exactly the same message as your actual camera hardware. The simulator also provides ground truth data with rewind and replay functionality.

Isaac Sim is based on Unreal Engine 4.20.3. The main components of the simulator are the world model, robot model, dynamic characters, virtual sensors, virtual actuators, and ground truth information for deep learning applications. Isaac Sim communicates with the Isaac SDK through Isaac Bridge. The following image depicts Isaac Sim.