Carter Sim Application

The Carter Sim application connects to Isaac Sim via TCP and instantiates the robot with multiple sensors in the simulator.

The following sensors are simulated:

  • Stereo RGB cameras
  • Depth images
  • Ground truth segmentation, class instances
  • Ground truth bounding boxes
  • 3D LIDAR
  • Robot odometry

After installing the Isaac SDK and Isaac Sim, follow the steps in this section to control the simulated robot for navigation and mapping using the SDK application located at apps/carter/carter_sim.

Random Walk

Instantiate the robot with its sensors with the following command:

bazel run apps/carter/carter_sim:carter_sim -- --config="apps/assets/maps/carter_warehouse_p.config.json" --graph="apps/assets/maps/carter_warehouse_p.graph.json"

The simulated Carter robot starts moving after it localizes itself and selects a random point every 30 seconds. It automatically navigates to the target location once it has been selected, avoiding obstacles on the way.

Supported Environments for Navigation

The maps supported with Carter Sim are listed below. For each map, the carter_sim command-line and associated Isaac Sim command-line are provided.

carter_warehouse_p:

bazel run apps/carter/carter_sim:carter_sim -- --config="apps/assets/maps/carter_warehouse_p.config.json" --graph="apps/assets/maps/carter_warehouse_p.graph.json"
./Engine/Binaries/Linux/UE4Editor IsaacSimProject CarterWarehouse_P -vulkan -isaac_sim_config_json="ABSOLUTE_PATH_TO/apps/carter/carter_sim/bridge_config/carter_full.json"

carter_office:

bazel run apps/carter/carter_sim:carter_sim -- --config="apps/assets/maps/carter_office.config.json" --graph="apps/assets/maps/carter_office.graph.json"
./Engine/Binaries/Linux/UE4Editor IsaacSimProject Carter_Office -vulkan -isaac_sim_config_json="ABSOLUTE_PATH_TO/apps/carter/carter_sim/bridge_config/carter_full.json"

hospital:

bazel run apps/carter/carter_sim:carter_sim -- --config="apps/assets/maps/hospital.config.json" --graph="apps/assets/maps/hospital.graph.json"
./Engine/Binaries/Linux/UE4Editor IsaacSimProject Hospital -vulkan -isaac_sim_config_json="ABSOLUTE_PATH_TO/apps/carter/carter_sim/bridge_config/carter_full.json"

Each environment has a custom spawn pose that must be changed in carter_full_config.json. The default pose is specified for the warehouse map:

{
  "name": "carter_1",
  "reference": "Blueprint'/Game/Carter/VehicleRigs/Pawn_Carter_FirstPerson.Pawn_Carter_FirstPerson'",
  "pose": [1, 0, 0, 0, -11.01, 63.47, 0.92]
}

See Environments for details on the Spawn location for Carter in each environment.

Mapping

In order to create a map with Isaac Sim, run the carter_sim_mapping application from the SDK and start Isaac Sim with the environment you would like to map.

To map the warehouse for example, do the following:

  • Start the IsaacSim:

    ./Engine/Binaries/Linux/UE4Editor IsaacSimProject CarterWarehouse_P -vulkan -isaac_sim_config_json="ABSOLUTE_PATH_TO/apps/carter/carter_sim/bridge_config/carter_full.json"
    
  • Run the carter_sim_mapping app

    bazel run apps/carter/carter_sim:carter_sim_mapping
    
  • Use the joystick to control the robot and drive it around the map until complete

The settings for Cartographer are specified in apps/carter/carter_sim/carter.lua.

Press CTRL-C to exit the carter_sim application. This causes the mapping node to write the final merged map to disk, then stop. The map image is saved in the following file:

/tmp/submap_merged.png

To use this map in your applications, change the PNG file specified in carter_sim_config.json to the new map.

Maps can be cleaned up to remove noise and/or errors and make corrections. Crop the map to make it easier to work with and improve performance.