NVIDIA Carter

Carter is a robot developed as a platform to demonstrate the capabilities of the Isaac SDK. It is based on a differential drive and uses lidar and a camera to perceive the world.

This document walks you through hardware assembly and software setup for Carter.

The following components are required to build the Carter hardware platform:

Devices

Sensors

Power/Control

Hardware – Custom Parts

Hardware – Standard Parts

Cables and Wiring

The following tools are required to build the Carter hardware platform:

• Screwdriver set

• Socket wrench set

• Metric hex driver set

• Wire cutters

• Wire terminal crimping tool (for Anderson Powerpole or equivalent connectors)

• Multimeter

You will need a Linux host machine to deploy applications to Carter. System requirements for this host machine are described in the System Requirements section of the SDK Manager documentation.

It takes, on average, three business days for first-time assembly of a Carter robot. This time period does not include receiving shipped parts or machining custom parts.

Fig. 4Carter - front view

Fig. 5Carter - rear view

The following diagram shows the wiring for Carter. It may be useful to reference this diagram during assembly.

Fig. 6Carter wiring diagram

Follow these steps to assemble Carter:

1. Assemble T Brackets into Legs

Carter has two legs that connect to the Segway RMP. Each leg consists of three T brackets and five M8x30 screws.

For each leg, secure three T brackets together using two screws on the inner side and three screws on the outer side, arranged as shown in the image below.

The two legs should be mirrored after assembly.

Fig. 7Left leg - inner view (L) and outer view (R)

Fig. 8Right leg - inner view (L) and outer view (R)

2. Attach Legs to RMP

Secure each leg to the Segway RMP platform using two M8x40 screws in the inner holes of the bottom bracket. The image below shows the right leg.

Fig. 9Carter right leg

3. Attach Terminal Blocks and Wiring to Xavier Plate

A small mount plate will serve as the mount plate for the Jetson AGX Xavier board.

Attach the two power distribution blocks to the top half of the plate using sticky-back hook-and-loop strips. These blocks break out power from the 12V RMP battery to other devices in the system. The RMP wiring harness uses 18-gauge wires.

Use a crimping tool to crimp the Anderson spade connectors to ten 18-gauge wires (five red and five black), and use these connectors to connect the wires to the terminal blocks. Use a positive and negative wire to wire each of the following connectors:

• The 2.5mm barrel plug for Xavier

• The blade contact for the Power1+ and Power1- wires on the RMP CONNECTOR I Breakout Harness, which supplies 12V from the RMP battery

• The 2.1mm barrel plug for the GS108 or GS105 Ethernet switch

• The 2.1mm barrel plug for the VLP-16 lidar

• The 5mm pitch terminal block plug for the Startech USB hub

Screw the WiFi antennas into the two holes on the top flange of the plate; use a single washer to secure each antenna. Feed the wiring through the two nearby large holes on the face of the plate. This will allow the wires to reach the Xavier board on the other side of the plate.

Attach the Ethernet cable to the plate. One end will be attached to the Xavier board and the other to the Ethernet switch. The connections will be made in a later step.

The image below shows the various connections from the power distribution blocks, the Ethernet cable, and WiFi antennas on the back of the Xavier plate.

Fig. 10Carter power breakout

4. Prepare RMP CONNECTOR I Harness

Cut the factory crimps off of the Power1+ (red) and Power1- (Return, green) wires on the RMP CONNECTOR I harness. Add a red blade-housing connector to these wires for Power1 wiring to the power distribution blocks on the Xavier plate.

Fig. 11Modified RMP CONNECTOR I harness wiring

5. Prepare the Xavier board

Install the WiFi card in the J505 M.2 slot on the bottom of the Xavier board.

Verify that the jumper on the J514 Voltage Select shorts pins 1 and 2 (denoted by the small green box in the diagram below).

Position the Xavier plate near the Xavier board and attach the antenna cables to the WiFi card.

Fig. 12Location of the J505 M.2 slot on the Xavier board

6. Attach the Xavier Board to the Xavier Plate

Remove the four screws shown below from the Xavier board.

Fig. 13Location of screws on the Xavier board

Leave the Xavier board upside down. Align the side of the Xavier board with the power jack and Ethernet and USB connectors, shown at the top of the image above, with the top edge of the Xavier Plate. With the Xavier Plate facing downward, lift the plate, place the M3 washers on the Xavier mounting holes, and insert the M3x35 screws.

Lower the plate onto the Xavier board. Ensure you are aligning the screws with the holes in the feet and avoiding the antenna wires. Secure the M3 screws to mount the Xavier board to the plate.

The photo below shows the face of the Xavier plate with the Xavier board mounted. Note the USB, Ethernet, and power connectors on the top edge and the unused PCIe connector on the left side.

Fig. 14Xavier plate with Xavier board mounted

7. Attach Lidar and RealSense Camera to Mount and Plate

The other small mount plate carries the VLP-16 lidar. The 3D-printed lidar mount has two small holes on the top for the alignment dowels and a through hole in the center for securing the lidar with the 1/4”-20x1” bolt. The default orientation of the lidar is pointing away from the caster (i.e. the small third wheel) of the RMP, with the USB cable on the back pointed toward the caster.

Secure the lidar to the lidar mount with the 1/4”-20x1” bolt. Then, secure the lidar mount to the small mount plate with two M8x30 screws and M8 locknuts.

Use two M3x6 screws to secure the RealSense camera to its mount. This mount attaches to the front portion of the lidar mount with a M6x20 screw and nut. The mount angle should be set to 0 degrees by default, and can be adjusted in 15 degree steps. Plug the Type C end of a USB Type C to Type A cable into the RealSense camera.

Secure the lidar wiring and components to the bottom of the lidar plate.

Fig. 15Lidar wiring and components secured to lidar plate

8. Attach Lidar Plate to Carter

Use four M8x12 or M8x20 screws in the corners to secure the lidar plate to the middle T bracket, as shown in the photo below. Connect one end of an Ethernet cable to the lidar. The connections to the Ethernet switch and power will be made in a later step.

Fig. 16Lidar plate attached to Carter

9. Connect ZED Camera to Camera Plate

Attach the ZED camera to the stereo camera mount using a 1/4”-20x1/2” mount bolt. Attach the stereo camera mount to the center of the stereo camera plate mount using an M6x20 screw. There is a small notch at the rear edge of the part that indicates the center and helps with alignment.

Use double-stick tape to attach the stereo camera plate mount to the large mounting plate (camera plate). The mount angle should be set to 0 degrees (horizontal) by default and can be adjusted in 15-degree steps.

Secure the camera plate to the top T bracket with four M8x12 or M8x20 screws in the corners.

Fig. 17Camera plate attached to the top of Carter

10. Attach E-Stop button

Use double-stick tape to attach the E-Stop mount bracket to the side of the RMP E-Stop Button housing that contains the cable. Route the cable through the cut-out in the mount bracket.

Use a M5x20 screw and M5 locknut, along with double-sided tape, to secure the top right of the E-Stop Button housing to the lidar plate.

Connect the E-Stop cable to CONNECTOR II on the RMP Centralized Control Unit (CCU). Secure the cable so that it doesn’t interfere with the wheels.

Fig. 18E-Stop button attached to the lidar plate

11. Attach USB Hub to RMP Battery

Use hook and loop fasteners to attach the USB hub to the RMP battery cover. The USB hub should be located to the left of the E-Stop. This location will make it easier to connect devices for development and debugging in the future.

The photo below shows the hub after all connections are made, but you don’t need to connect anything in this step.

Fig. 19USB hub with all connections made

12. Attach Ethernet Switch to Front of RMP CCU

Use hook and loop fasteners on the bottom of the Ethernet switch to attach it vertically to the front right of the CCU, with the Ethernet ports facing to the left. Leave enough room for the barrel plug to connect on the right side.

The photo below shows the Ethernet switch after all connections are made, but you don’t need to connect anything in this step.

Fig. 20Ethernet switch with all connections made

13. Make Connections on the Xavier Plate

Route the cabling from the RMP harness below the CCU to the Xavier plate. The Ethernet cable and power1+/power1- wires (12V source) are the only connections that you need to make.

Now is a good to time to check voltages: Power on the RMP and use a multimeter to verify that the polarity and voltage is correct on all power connectors. Power off the RMP before continuing.

Move the Xavier plate close to the front of Carter and make these connections from the back of the Xavier plate:

1. power1+/power1- wires (with spade connector) from power distribution block to power1+/power1- wires (with blade-housing connector) from RMP CONNECTOR I harness

2. 2.1mm barrel plug from power distribution block to Ethernet switch

3. 2.1mm barrel plug from power distribution block to lidar

4. Power terminal from power distribution block to the USB hub

5. USB cable from USB hub (Type B end), routed toward the Xavier Plate

14. Set up IMU

Add a 6-pin socket or header to the VIN, 3V3, GND, SCL, and SDA signals. Solder the 6-pin socket or header to the BMI160 board. See the BMI160 wiring guide for more details.

Secure the BMI160 board to the IMU mount with two M2x6 screws, with the socket oriented as shown below.

Fig. 21BMI160 board attached to IMU mount

Depending on the resolution of the 3D printer, the mounting holes may need to be drilled to allow the screws to fit. The two cutouts in the mount next to the board allow you to use a zip tie to secure the mount to the RMP battery cover.

Use jumper wires or create a cable to connect the 3V3, GND, SCL, and SDA pins of the BMI 160 to the Xavier J30 40-pin connector. If you are using a 40-pin socket for the Xavier side, take care to note the location of pin 1.

Fig. 22BMI 160 with example harness

Fig. 23BMI 160 wiring to Xavier 40-pin socket

The bottom of the 3D-printed mount has two tabs that insert into two vent holes on top of the RMP battery cover. It should be installed in the two center holes with the connector side closer to the edge of the cover and secured with a zip tie.

Route the Xavier side of the IMU cable to the bottom of the RMP platform. It will curl under the Xavier plate to make its connection.

15. Make Final Connections

Make the following connections:

1. USB cable (Type A end) to Xavier

2. Ethernet cable from the back of the Xavier plate to Xavier

3. 2.5mm barrel plug to Xavier jack

4. Ethernet from lidar to Ethernet switch

5. Ethernet from RMP CONNECTOR I harness to Ethernet switch

6. One end of the Ethernet cable on the Xavier Plate to the Ethernet switch

7. ZED Camera USB to USB Hub

8. RealSense camera USB to USB Hub

16. Attach Xavier Plate and IMU

Use four M8x12 or M8x20 screws to secure the Xavier plate to the lowest bracket of each leg. Attach the IMU 40-pin connector to the Xavier board. Pin 1 of the connector is on the right when facing the Xavier board.

Fig. 24Xavier plate with IMU 40-pin connector connected

17. Secure Cables

Use zip ties to secure the USB cables for the RealSense and ZED cameras–and any other loose cables as needed.

After you assemble Carter, go through the following steps to configure the software on it.

1. Configure the Jetson AGX Xavier

1. Install the Jetson operating system on the Jetson AGX Xavier as described in the SDK Manager documentation.

2. Obtain the IP address of the robot as described in the Getting Started With Jetson Nano guide.

3. Follow the Setup guide to install Isaac SDK, along with all of its dependencies, on the Xavier.

4. Follow the steps in the Deploying and Running on Jetson section to register your SSH key with the Xavier.

2. Configure the VLP-16 Lidar

Out of the box, the VLP-16 lidar is configured with an IP address of 192.168.1.201. You will need to change this address:

1. Set the Jetson AGX Network to 192.168.1.5.

2. Use a web browser to navigate to http://192.168.1.201. This should open the configuration page for the VLP-16 lidar.

   1. Change the Host (Destination) IP address from “255.255.255.255” to “192.168.0.5” (i.e. the IP address of the Xavier) and click the Set button. Note that each section has its own Set button.

   2. Change the IP address of the Network (Sensor) (i.e. the Lidar itself) to “192.168.0.201” and click the Set button.

   3. Click the Save Configuration button.

3. Change the Jetson AGX Network back to 192.168.0.5.

4. Ping the VLP-16 lidar from the Xavier to confirm the new configuration:

   Copy

   Copied!

               
   
               
   $ ping 192.168.0.201
           

   If the ping fails, refer to Appendix J - Network Configuration in the VLP-16 User Manual for help.

3. Configure the RMP 210

Out of the box, the RMP 210 has a default IP address of 192.168.0.40. Ping the RMP 210 from the Xavier to confirm these devices can see each other:

            

            
$ ping 192.168.0.40
        

4. Configure the Xavier

1. Connect the Jetson AGX Xavier to a display and connect a keyboard and mouse to the USB hub.

2. Log in to the Xavier.

3. Connect to a strong WiFi network.

4. Install dependencies on the Xavier:

   1. Install “ssh” on the Xavier:

      Copy

      Copied!

                  
      
                  
      sudo apt-get install ssh
              

   2. Install “jstest-gtk” on the Xavier. This is required to configure and calibrate the joystick controller.

      Copy

      Copied!

                  
      
                  
      sudo apt-get install jstest-gtk
              

5. Set the controller to pairing mode (refer to the controller manual for instructions). From Bluetooth Settings, pair and connect the joystick.

6. Set up the RealSense camera:

   1. Configure the RealSense camera codelet as described on the RealSense Camera page.

   2. From the host machine, deploy the sample application to verify that the RealSense camera is working:

      Copy

      Copied!

                  
      
                  
      ./engine/build/deploy.sh --remote_user <username_on_robot> -p //apps/samples/realsense_camera:realsense_camera-pkg -d jetpack44 -h <robot_ip>
              

      Where <username_on_robot> is your username on the Xavier and <robot_ip> is the IP address of the Xavier.

   3. Log in to the Xavier from the host machine using SSH and run the sample application as follows:

      Copy

      Copied!

                  
      
                  
      cd deploy/user_name/realsense_camera
      ./apps/samples/realsense_camera/realsense_camera
              

   4. On the host machine, navigate to http://<robot_ip>:3000/. Check the “realsense camera” channel to verify that the camera is running.

7. Configure the ZED camera as described on the ZED Camera page.

8. On the Xavier, edit the sensor configuration files for Carter to accurately reflect the position of the sensors. The following are file locations and sample configurations for the different sensors:

   Tip

   See this FAQ for a description of 3D pose values.

   1. Lidar: isaac/apps/carter/robots/

      Copy

      Copied!

                  
      
                  
      "2d_carter.carter_hardware.vlp16_initializer": {
         "lidar_initializer": {
           "lhs_frame": "robot",
           "rhs_frame": "lidar",
           "pose": [1.0, 0.0, 0.0, 0.0, 0.022, 0.0, 0.625]
         }
       }
              

   2. RealSense camera: isaac/apps/samples/realsense_camera

      Copy

      Copied!

                  
      
                  
      "2d_carter.carter_hardware.camera": {
         "realsense_pose": {
           "lhs_frame": "robot",
           "rhs_frame": "camera",
           "pose": [0.5, -0.5, 0.5, -0.5, 0.150, 0, 0.573]
         }
              

      Tip

      Refer to the RealSense Camera document for more information on using the RealSense camera with Isaac applications.

   3. ZED camera: isaac/apps/samples/follow_me

      Copy

      Copied!

                  
      
                  
      "camera": {
           "zed_left_camera_initializer": {
             "lhs_frame": "robot",
             "rhs_frame": "left_camera",
             "pose": [0.5, -0.5, 0.5, -0.5, 0.098, 0.06, 0.763]
           }
              

      Tip

      Refer to the ZED Camera document for more information on using the ZED camera with Isaac applications.

Follow the steps below to create a map and configure it for the Carter application:

1. Set up the Isaac SDK on your host machine as described on the Setup page.

2. From the Isaac SDK repository on your host machine, deploy either the GMapping Application or the Cartographer application to Carter.

3. Use the Map Editor to create waypoints and restricted areas on your map.

4. Create a config and graph file for the map. Refer to the sample files in the Isaac SDK.

This section describes how to run four different sample applications on Carter.

Application 1: Random

This application instructs Carter to travel from one waypoint to another using randomly chosen waypoints.

1. From the host machine, deploy the Carter application to the robot:

   Copy

   Copied!

               
   
               
   ./engine/build/deploy.sh -p //apps/carter:carter-pkg -d jetpack44 -h <robot_ip> --remote_user <username_on_robot> -s
           

2. SSH into the robot (using the Xavier IP).

3. Go to the /deploy/<host_username>/carter-pkg directory.

4. Run the application:

   Copy

   Copied!

               
   
               
   ./apps/carter/carter -m <map_name>
           

5. To create a map of large environments, you need to tune the map to create a dense graph. This will speed up path planning for the robot. Refer to the Global Planner section of the Tuning the Navigation Stack document for more details.

6. Go to the Isaac Websight visualization tool (https://<robot_ip>:3000) to visualize the application.

7. Switch to a different goal behavior in Websight:

   1. Go to the “goals.goal_behavior” node.

   2. Change the “desired_behavior” from “Random” to “WayPoint”, “Pose”, “Route” etc.

   3. Go to “goals.pose_as_goal”, “goals.waypoint_as_goal”, or “goals.patrol” and click Submit to activate the application with that configuration.

Application 2: Waypoint

This application moves Carter to a given goal using the isaac.navigation.MapWaypointAsGoal component.

To use the Waypoint application, add the sample Flatsim delivery application to your Carter application. This sample application is located at isaac/apps/carter/carter_delivery/carter_delivery_flatsim.app.json.

Application 3: Pose

This application moves Carter to a given goal in Websight.

The “pose_as_goal” config is part of the “goal_generators.subgraph.json” file by default. Follow these steps to enable this application:

1. Open Websight.

2. Under “goals.goal_behavior”, change the “desired_behavior” to “pose_as_goal”.

3. Add the “pose_as_goal” marker to navigate the robot using the pose.

Fig. 25Changing the “desired_behavior” to “pose_as_goal”

Refer to the Additional Notes section of the Interactive Markers document for more details.

Application 4: Patrol

This application instructs Carter to patrol along a pre-defined route.

First, modify the config for “patrol” mode in isaac/packages/navigation/apps/goal_generators.subgraph.json: Add a “route” and list the waypoints on the patrol route.

            

            
"patrol": {
      "isaac.navigation.Patrol": {
         "tick_period": "250ms",
         "route": [
            "home",
            "kitchen",
            "office"
         ]
      }
   }
        

After adding the route to the sample JSON, run the Carter application and open Websight. Under the “goals.goal_behavior” configuration, change the “desired_behavior” to “patrol”.

Fig. 26Changing the “desired_behavior” to “patrol”

RMP 210 does not move

Ensure the RMP 210 has the correct IP address (192.168.0.40) and you are able to ping that address from the Xavier. If the RMP 210 cannot connect to its host, it will power down automatically after a timeout.

VLP-16 lidar does not connect Ensure the lidar has been programmed with the correct IP addresses: 192.168.0.201 for itself and 192.168.0.5 for the host/Xavier. Ping the VLP-16 from the Xavier to ensure the connection is working.

Isaac application cannot access the IMU

Check the continuity of the cable from the Xavier to the pins on the IMU board.

USB hub is not working

Check the power- and ground-wire mapping and continuity to the terminal.

RealSense camera not working

Ensure that the StarTech USB hub is externally powered and that you are using properly rated USB 3.1 cables. Double-check that any issues can be reproduced with the RealSense camera connected directly to the Xavier using its original USB cable.

Wires are coming loose

Use ferrules to attach wires to the barrel plugs and USB hub power terminal. They make for a more secure connection than bare wire.