Quick Start

This section describes how to quickly get started simulating the Carter and Kaya robots in Isaac Sim.

Carter

The procedures in this section walk you through running the carter_sim sample application included in Isaac SDK. The application demonstrates Carter, a two wheeled robot, navigating in Isaac Sim, providing simulated sensor data displayed in the Sight web application.

Note

Follow the procedures in the Isaac SDK Developer Guide to set up Isaac SDK before using the procedures in this section.

Setup Paths

In Isaac SDK, edit apps/carter_sim/bridge_config/carter_full.json so that the paths to carter_full_graph.json and carter_full_config.json are correct full paths to those files:

{
 "graphs": ["FULL_PATH_TO/apps/carter_sim/bridge_config/carter_full_graph.json"],
 "configs": ["FULL_PATH_TO/apps/carter_sim/bridge_config/carter_full_config.json"]
}

Start the carter_sim Application

Start the carter_sim application with the following command:

bazel run apps/carter_sim:carter_sim -- --config="apps/assets/maps/carter_warehouse_p.config.json" --graph="apps/assets/maps/carter_warehouse_p.graph.json"

Once the carter_sim application has started, navigate to localhost:3000 in a web browser to start the Sight application. Before Isaac Sim is started, the Sight interface should be similar to the following:

_images/Quickstart_sight_no_sim.png

Start Isaac Sim

Note

The first time you run Isaac Sim, shaders are compiled, slowing the startup process. Waiting for the Compiling Shaders popup window to finish will fix this. Also, objects or characters might appear in black. Subsequent runs will fix this.

  1. Start the Isaac Sim application with the following command:

    ./Engine/Binaries/Linux/UE4Editor IsaacSimProject CarterWarehouse_P -vulkan -isaac_sim_config_json="<FULL_PATH>/apps/carter_sim/bridge_config/carter_full.json"
    

    where <FULL_PATH> completes the full path to the JSON file.

  2. When Isaac Sim finishes loading, click Play. After the robot spawns and localizes, it navigates around the warehouse environment, similar to the following image:

    _images/Quickstart_isaac_sim.png
  3. At this point, the Sight interface resembles the following image:

    _images/Quickstart_sight_disabled.png
  4. On the left side, in the Channels section, make sure the carter_sim checkbox is checked. Once checked, Sight shows the latest data received from Isaac Sim.

    _images/Quickstart_sight_enabled.png

Note

For information on the supported environments see Supported Environments for Navigation. Information on running carter_sim with joystick control or in mapping mode can be found in the Isaac SDK documentation for the carter_sim application.

Kaya

The procedures in this section walk you through simulating three sample applications included for the Kaya robot in the Isaac SDK.

Note

Follow the procedures in the Isaac SDK Developer Guide to set up Isaac SDK before using the procedures in this section.

In order to access the Kaya sim apps, download Kaya Sim App for Isaac SDK and extract it into the SDK folder:

tar -xvf kayasim.tar.xz -C <ISAAC_SDK_ROOT_PATH>

Setup Paths

In Isaac SDK, edit apps/kaya_sim/bridge_config/kaya_full.json so that the paths to kaya_full_graph.json and kaya_full_config.json are correct full paths:

{
 "graphs": ["FULL_PATH_TO/apps/kaya_sim/bridge_config/kaya_full_graph.json"],
 "configs": ["FULL_PATH_TO/apps/kaya_sim/bridge_config/kaya_full_config.json"]
}

Start Isaac Sim

Note

The first time you run Isaac Sim, shaders are compiled, slowing the startup process. Waiting for the Compiling Shaders popup window to finish will fix this. Also, objects or characters might appear in black. Subsequent runs will fix this.

  1. Start the Isaac Sim application with the following command:

./Engine/Binaries/Linux/UE4Editor IsaacSimProject Kaya_Table_P -vulkan -isaac_sim_config_json="FULL_PATH_TO/apps/kaya_sim/bridge_config/kaya_full.json"

where <FULL_PATH> completes the full path to the JSON file.

  1. When Isaac Sim finishes loading, click Play.

Using the Joystick Application in Simulation

This application demonstrates joystick control of Kaya in Isaac Sim.

Start the Joystick application with the following command:

bazel run apps/kaya_sim:joystick

To navigate via joystick, press and hold the left shoulder button to enter manual control mode, and then and use the left thumbstick to move the robot forward/backward and the right thumbstick to turn left/right.

Using the Object Detection Application in Simulation

This application demonstrates Kaya detecting objects in Isaac Sim and also navigating around via joystick control.

Start the object_detection application with the following command:

bazel run apps/kaya_sim:object_detection

Once the object_detection application has started, navigate to localhost:3000 in a web browser. The Sight interface is similar to the following, with static images:

_images/Kaya_1.png

On the left side, in the Channels section, make sure the object_detection checkbox is checked. Once checked, Sight shows the latest data received from Isaac Sim, including a bounding box around the object detected by Kaya and the type of object (compost/metal/paper) placed on the table.

_images/Kaya_2.png

You can also navigate via joystick using the controls mentioned earlier to detect other objects on the table.

Using the Follow Me Application in Simulation

This application demonstrates Kaya detecting AprilTags in Isaac Sim and moving towards them.

Start the follow_me application with the following command:

bazel run apps/kaya_sim:follow_me

Once the follow_me application has started, navigate to localhost:3000 in a web browser.

On the left side, in the Channels section, make sure the follow_me checkbox is checked. Once checked, Sight shows the latest data received from Isaac Sim.

_images/Kaya_3.png

To navigate via joystick, press and hold the right shoulder button for Kaya to navigate and follow the detected AprilTag.