Running the Navigation Stack in Isaac Sim

This section will guide you through the process of installing the Navigation Stack on your machine and running a simple demo with the Carter platform in Isaac Sim.

Isaac SDK and Isaac Sim provide prebuilt Docker images that contain the entire navigation stack. Thus, you only need to have Docker installed on your system.

Ensure you have access to the nvcr.io NVIDIA container registry. You can log in to nvcr.io as follows:

            

            
docker login nvcr.io
        

Local and cloud versions of Isaac Sim can be found here.

Follow the official Isaac Sim documentation here to install the desired version(s) and learn how to use Isaac Sim.

This guide will focus on running the Isaac 2.0 Navigation Stack in Isaac Sim.

Isaac SDK provides Docker containers that are set up for specific environments. Use the following commands to pull and run the Docker container with the navigation stack for an environment.

            

            
docker login nvcr.io
docker run --entrypoint bash -it --gpus all --rm --network=host --privileged \
  -v /dev:/dev \
  -v /sys:/sys \
  nvcr.io/nvstaging/isaac-amr/isaac_sim_single_robot_demo_<environment>:master-aarch64
# In the docker container:
/app/apps/amr/navigation_stack/isaac_sim_single_robot_demo_<environment>
        

Make sure to replace <environment> with the environment where the robot is located. The following are possible values, corresponding to the associated USD scenes in Isaac Sim:

• environment_full_warehouse

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/scenes/warehouse/GXF/large_warehouse_carter_v2_3.usd

• environment_voyager_l1_cafeteria

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/GXF/Robots/voyager_cafeteria_gxf_demo.usd

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/cuvslam/voyager_cafe_carter_2_3_gxf.usd

• environment_walled_room

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/GXF/Robots/carter_v2_gxf_stereocamera.usd

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/GXF/Robots/Carter_v2_GXF.usd

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/gxf_bridge/test_walled_room_carter_v2_3.usd

  • isaac-dev.ov.nvidia.com/Projects/isaac_amr_envoy/gxf_bridge/test_walled_room.usd

When the navigation stack finishes initializing, it will begin looking for a TCP connection (by default, on localhost).

1. Open Isaac Sim, then open a USD scene corresponding to the environment selected for the navigation stack.

2. Enable the GXF Bridge extension by selecting Window > Extension, then searching for “GXF Bridge” and clicking the toggle switch so it is green. This will cause a tab labeled “GXF Bridge” to appear at the bottom of the window, in the same group as “Content”.

3. If a Carter robot is not already in the scene, add one by selecting File > Add Payload. Navigate to /Isaac/Samples/GXF/ and select the appropriate robot, then click Add Payload. You can then adjust the robot’s initial pose in the scene.

4. Select the GXF Bridge tab, and confirm the Graph Path is set to the tcp_server.yaml file. You can modify the graph here to anything on your local system; by default, this file will launch a TCP server application designed to connect with the navigation stack.

5. Click “Create Application” to start the graph in Isaac Sim.

6. Once the Navigation Stack has finished initializing and has started looking for a TCP connection, click the Play button in Isaac Sim. The server and client will connect, and the navigation stack will begin.

7. Now that the Navigation Stack is running, you can visualize it by opening <Isaac Sim workstation address>:3000 in your browser. You should see the robot visualized on top of a 2D occupancy map.

   Additionally, you should see a visualized circle with the subscript robot/goal. This is the marker that represents the goal position of the robot. If the goal position is not on top of the robot, you should also see a visualized path starting at the robot and ending at the goal.

   You can update the robot goal position by clicking and dragging the circle representing the goal position.

   Note

   If the simulation is paused (see below), Isaac Sight will also pause.

8. At any point, you can pause the simulation by clicking the Pause button in Isaac Sim. This will also pause the Navigation Stack . You can then click the Play button to resume simulation and navigation.

9. At any point, you can stop the simulation by clicking the Stop button in Isaac Sim. This will reset the robot to its initial position in the scene. At this point, the Navigation Stack will end fatally if you click Play in Isaac Sim, as the simulation clock has been reset while the Navigation Stack clock has merely paused.

   To restart the simulation, click Destroy Application in the GXF Bridge tab, close the Navigation Stack app via Ctrl-C, then follow steps 1-7.

Pull and run the Isaac Sim 2023.1.0 docker container:

            

            
docker login nvcr.io
docker run --entrypoint bash -it --gpus all --rm --network=host --privileged \
-v /dev:/dev \
-v /sys:/sys \
nvcr.io/nvidia/isaac-sim:2023.1.0
# In the docker container:
./python.sh standalone_examples/testing/omni.isaac.gxf_bridge/run_scene.py -h
        

The run_scene.py script will then show command-line interface options. You must specify, at minimum, the path on the Nucleus server to a scene matching the environment that the navigation stack is using (e.g. /Projects/isaac_amr_envoy/scenes/warehouse/GXF/large_warehouse_carter_v2_3.usd).

If the scene includes an ActionGraph that will generate the TCP server application, you do not need to specify additional YAML files on the command line. Otherwise, after specifying the path, specify the YAML server graph that ships with Isaac Sim: exts/omni.isaac.gxf_bridge/data/config/tcp_server.yaml.

Common variations may look like the following:

            

            
./python.sh standalone_examples/testing/omni.isaac.gxf_bridge/run_scene.py \
 # Use the large warehouse scene that ships with Carter 2.3
/Projects/isaac_amr_envoy/scenes/warehouse/GXF/large_warehouse.usd \
# Use the GXF server graph that ships with Isaac Sim
exts/omni.isaac.gxf_bridge/data/config/tcp_server.yaml \
# Set the frame rate to 60 Hz
-r 60 \
# Run for 60000 frames (1000 seconds of simulated time)
-n 60000
        

            

            
./python.sh standalone_examples/testing/omni.isaac.gxf_bridge/run_scene.py \
# Use the large warehouse scene that ships with Carter 2.3
/Projects/isaac_amr_envoy/scenes/warehouse/GXF/large_warehouse.usd \
# Use the GXF server graph that ships with Isaac Sim
exts/omni.isaac.gxf_bridge/data/config/tcp_server.yaml \
# Run the scene indefinitely at 60 Hz (default rate), eg. for a random-walk test
--run_indefinitely \
# Place the robot in the scene at (5, 2) in world coordinates,
# with initial yaw 90 degrees
--robot carter_v2_3
--initial_pos 5.0 2.0
--initial_yaw 90
        

Now that the Navigation Stack is running, you can visualize it by opening <Isaac Sim workstation address>:3000 in your browser. You should see the robot visualized on top of a 2D occupancy map.

Additionally, you should see a visualized circle with the subscript robot/goal. This is the marker that represents the goal position of the robot. If the goal position is not on top of the robot, you should also see a visualized path starting at the robot and ending at the goal.

You can update the robot goal position by clicking and dragging the circle representing the goal position.

When running Isaac Sim from the Docker container, you cannot manually pause or stop the simulation. If you wish to cancel the simulation, you can stop the container. To restart the simulation, close the Navigation Stack app via Ctrl-C, then restart the Navigation Stack and Isaac Sim docker containers.