NVIDIA Carter

The NVIDIA Carter robot has been developed as a platform to demonstrate the capabilities of the NVIDIA Isaac SDK. It is based on the Segway RMPLite 220 differential drive and uses Lidar and a camera to perceive the world.

This document walks you through hardware assembly and software setup for Carter version 2.0.

The following components are required to build the Carter v2.0 hardware platform. The total cost for all parts is approximately 10,000 USD, the majority of which consists of the Lidar (~4,000), wheel base (~3,000), and sheet metal (<2,000).

Custom Parts

Building a Carter robot requires custom sheet-metal and 3D-printed parts. This OnShape project contains CAD files for required custom parts. Refer to the CAD Export Instructions section below for further details.

Sheet-metal parts:

3D-printed parts:

Devices

* This part can be excluded if Lidar is the only Ethernet device required.

Power

Comsumables

Fasteners

Signaling

The following tools are required to build the Carter hardware platform:

• Hex wrenches (various sizes for M2-M5 bolts)
• Socket wrench (M4 – 7mm, M5 – 8mm)
• Heat gun (for wiring heat shrink)
• Soldering iron
• Crimp tools
  • Powerpole
  • Ferrule
  • Fork spade crimps
  • Molex SL, Micro-Fit 3.0
• Wire strippers and cutters
• Multimeter
• Heat set insert tool (soldering iron can be used)

You will need a Linux host machine to deploy applications to Carter. System requirements for this host machine are described in the System Requirements section of the SDK Manager documentation.

• Time requirement: It takes, on average, five business days for first-time assembly of a Carter robot. This time period does not include receiving shipped parts or machining custom parts.
• Skill requirements: The user should be comfortable fabricating cable assemblies with crimp contacts, soldering, using a multimeter, and working with low-voltage DC electronics. Basic mechanical competence is required for the use of hand tools.
• Lidar guard: This sheet-metal component is optional and provides some protection for the VLP-16.
• Side supports: These sheet-metal components are optional and provide additional structural integrity for accessories or payloads placed on top of the Carter robot.
• Power system modularity: The DIN-rail mounted power system is designed for easy expansion with other components (e.g. additional terminal blocks or voltage regulators) to support additional equipment. Note that users make modifications at their own risk.
• Side panels: Carter v2.0 is designed with panel cutouts around the walls of the main body enclosure to facilitate easy substitution or addition of new sensors or actuators. Users are encouraged to create custom 3D-printed sensor brackets similar to those used for USB cameras in the design.
• High-power applications: When supplied with 12V DC power, the Jetson AGX Xavier Development Kit may experience power failures running demanding applications in MAXN mode. You can mitigate this issue by using a voltage with a higher supply. If you expect to run these types of applications on Carter, we recommend adjusting the output voltage of the SD-150C-12 voltage regulator to its maximum setting of ~16V. All components on this power distribution rail are compatible with input voltages up to 18 VDC.
• Threadlock: If locknuts are not used, we recommend using a medium or low-strength threadlock to prevent vibrations from loosening fasteners.
• Jetson flashing and setup: We recommend flashing the Jetson AGX Xavier Development Kit with Jetpack 4.5.1 (including Isaac SDK option) using outlet power prior to installation.
• CAD model: You can reference the OnShape CAD model throughout the assembly process to verify component locations.
• Lidar sheet metal parts: These have been modified from those shown in the assembly photos below. Reference the CAD version for complete accuracy.

Power-System Wiring

The following diagram outlines power-system wiring for Carter, along with a CAD drawing of the DIN rail assembly. Each power cable in the diagram is described below in more detail.

RMPLite Power

48V Rail Power

48V Rail to SD-150C-12 Vreg

SSR Relay Power Connections

SD-150C-12 Vreg to 12V Rail

Jetson AGX Xavier Power

VLP-16 Power

Ethernet Switch Power

USB Hub Power

Eth Switch GND

SD150C-12 GND

Voltage rails to ground blocks

Communications Wiring

The following diagram outlines communcations wiring for Carter. Each data cable in the diagram is described below in more detail.

RMPLite Data Harness

RMPLite UART

Sytem Power Signal

Jetson/Hub Ethernet

VLP-16 Ethernet

External Drop Ethernet

External Drop HDMI

Front Camera USB

Rear Camera USB

Left Camera USB

Right Camera USB

Jetson/Hub USB

External Drop USB

Wi-Fi Antenna Cable

Follow these steps to assemble Carter:

1. Fabricate cables

Reference the Carter Wiring Diagrams section above to fabricate necessary cables. You can fabricate connections between DIN rail components during assembly if desired.

The following table outlines the wiring of the RMPLite data harness:

The following table outlines the wiring of the RMPLite UART cable:

The “RMPLite Power”/”RMPLite Data Harness” and “RMPLite UART” data cable assemblies are the most complex. All other assembles only use two conductors for power.

2. Prepare 3D-printed parts

The following table lists 3D-printed parts for Carter:

Install heat-set inserts. Each camera panel part (2x front/back, 2x side) requires four M2 x 4mm heat set inserts to mount USB cameras.

Mount the USB cameras and antennas.

3. Prepare the Jetson AGX Xavier

Use a jumper to short pins 5 and 6 on the automation header. This causes the Xavier to start up automatically when connected to power. Refer to the Jetson AGX Xavier Specification document (section 3.5 (pg. 29) table 3-7) for more details.

Install the Wi-Fi card in the J505 M.2 slot on the bottom of the Xavier board. Attach the IPEX MHF4 to RP-SMA Female Coax Cable, 6in to the Intel 9260 card.

Install the 3D-printed flange-mount feet and panel-mount SMA connectors.

4. Prepare Segway RMPLite 220 wheelbase

Remove the stock top plate. Remove the E-Stop button and extend its cable to reach the rear of the RMPLite 220. You will need to cut the cable and solder it with the added extension; we recommend using heat shrink. Route and secure the cable.

Connect the RMPLite power and data cables.

5. Populate DIN rail with power distribution components

Unless the DIN rail has been purchased as a pre-cut length, cut it to a length of approximately 275mm prior to using it.

Reference the power system Diagram and DIN rail assembly CAD in the Carter Wiring Diagrams section. Use ferrule terminals for all DIN rail connections.

Make connections between components on the DIN rail and add input cables that connect to the RMPLite (48V to RMPLite, RMPLite SSR Sig).

Leave the outgoing power cable connections disconnected for now.

6. Test the power distribution system

1. Connect the SD-150C-12 regulator to the DIN rail assembly.
2. Connect 48V to the RMPLite and the RMPLite SSR Sig cables to the RMPLite.
3. Power on the RMPLite and measure the voltage rails to ensure proper output voltages. The SD-150C-12 output voltage can be adjusted if desired.
4. Verify that the 48V and 12V power rails are unpowered when the RMPLite platform is turned off. If this is not the case, check connections to the solid-state relay. These are labeled in the power system wiring diagram.
5. Disconnect the SD-150C-12 and RMPLite connections from the DIN rail.

7. Install the SD-150C-12 regulator

The SD-150C-12 can be mounted to either the top or bottom of the Body Box Enclosure. Mounting to the underside creates more space inside the enclosure, but this option requires you to bend the fork connectors to 90 degrees, requires more care, and reduces access. Mounting inside the enclosure takes up space, but makes the regulator much more accessible. The same holes are used for either configuration.

Connect the cables to the SD150C-12 and route them to the interior of the enclosure (48VIN SD150C-12, 12VOUT SD150C-12, SD150C-12 GND).

8. Mount the base plate and body box enclosure

Use 6 M5x10 SOC screws with M5 washers. First, ensure that the cables from the RMPLite and SD-150C-12 Voltage Regulator are connected and routed through the cable-access hole in the Body Box enclosure.

9. Install E-Stop button into base plate

Once installed, connect the E-Stop button with the previously extended and routed cable.

10. Install camera panels

Install the assembled 3D-printed front, rear, left, and right camera panels. Use M4 nylock nuts and M4 washers.

11. Install light panels

Install light panels in all four corners using M4 nylock nuts and M4 washers.

Note that the light panels are designed for use with NeoPixel LEDs (60 per meter version), as shown in the photos below. We do not provide instructions on enabling this, but you can do so on your own.

12. Install side panel covers

Install four side panel covers using M4 nylock nuts and M4 washers.

13. (Optional) Install side supports on the Body Box enclosure

Side supports provide additional structural integrity and mounting points (4x M6) for accessories or payloads placed on top of Carter.

14. Install HDMI, Ethernet, and USB panel connectors on the Ext Drop Box

Use M3 x 10 90deg flat-head bolts and M3 locknuts.

15. Install the Ext Drop Box on the Body Box enclosure

Install the Ext Drop Box on the Body Box enclosure using M4 nylock nuts and M4 washers.

16. Mount the DIN rail assembly

Mount the DIN rail assembly using M5 locknuts and washers.

After the assembly is mounted, reconnect cables for the SD-150C-12 and RMPLite.

17. Install the front support

Install the front support in the Body Box enclosure using M4 nylock nuts and M4 washers.

18. Mount the VLP-16 Lidar

Mount the VLP-16 Lidar onto the Lidar Plate using 1/4-20 x 1/4in SOC screws and threadlocks.

19. Install the Lidar Plate and Velodyne Interface Box

Install the Lidar Plate and Velodyne Interface Box on the Front Support using M4x8 SOC screws.

Optionally, you can install the Lidar Guard with the Lidar Plate, which requires M4x10 SOC in place of M4x8 SOC screws.

Excess cable can be coiled and secured inside the Front Support.

20. Install USB hub

Install the USB hub in the Body Box enclosure using M4 nylock nuts and M4 washers.

21. Install Jetson AGX Xavier

Install the Jetson AGX Xavier Development Kit in the Body Box enclosure using M4 nylock nuts and M4 washers.

22. Connect power cables from the DIN rail to devices

Connect and secure the power cables from the DIN rail to the following devices:

• Jetson AGX Xavier Development Kit
• USB Hub
• VLP-16 Lidar
• Ethernet Switch

23. Connect data cables

Connect and secure the following data cables:

• Jetson to USB Hub
• Jetson to RMPLite Data Harness
• Jetson to Ethernet Switch
• Jetson to HDMI External Drop
• Jetson to Wi-Fi antennas
• (4x) USB Cameras to USB Hub
• USB Hub to USB External Drop
• VLP-16 to Ethernet Switch
• Ethernet Switch to Ethernet External Drop

24. Install Ext Drop cover and Body Box cover

Carter assembly is now complete. Internal wiring is shown in the following images.

You can export the CAD files for custom sheet metal and 3D-printed parts from the Carter OnShape project.

To simplify the export process, you can use the Configurations menu in the top left to isolate only the sheet metal or 3D-printed parts.

You can export an entire assembly by right-clicking the assembly tab at the bottom of the screen. This is recommended when exporting sheet metal parts or the entire Carter v2.0 assembly in STEP or IGES format. If sheet metal parts are isolated using the configuration menu, only these parts will be exported with their captive fasteners. This is ideal for providing part files to sheet metal fabricators.

You can also export parts individually by right-clicking on their names in the left assembly list. This is recommended when exporting 3D-printed parts in STL format for printing.

If you have an OnShape account (free for personal use), you can also link or copy the Carter v2.0 assembly for use in your projects using the icons in the upper-left corner.

After you assemble Carter, go through the following steps to configure the software on it.

1. Set up the Jetson AGX Xavier

1. Install the Jetson operating system on the Jetson AGX Xavier as described in the SDK Manager documentation.
2. Obtain the IP address of the robot as described in the Getting Started With Jetson Nano guide.

3. Follow the Setup guide to install Isaac SDK, along with all of its dependencies, on the Xavier.
4. Follow the steps in the Application Console Options section to register your SSH key with the Xavier.

2. Configure the VLP-16 Lidar

Out of the box, the VLP-16 Lidar is configured with an IP address of 192.168.1.201. You will need to change this address:

1. Set the Jetson AGX Network to 192.168.1.5.
2. Use a web browser to navigate to http://192.168.1.201. This should open the configuration page for the VLP-16 Lidar.
   1. Change the Host (Destination) IP address from “255.255.255.255” to “192.168.0.5” (i.e. the IP address of the Xavier) and click the Set button. Note that each section has its own Set button.
   2. Change the IP address of the Network (Sensor) (i.e. the Lidar itself) to “192.168.0.201” and click the Set button.
   3. Click the Save Configuration button.
3. Change the Jetson AGX Network back to 192.168.0.5.

4. Ping the VLP-16 Lidar from the Xavier to confirm the new configuration:

   Copy

   Copied!

               
   
               
   $ ping 192.168.0.201
           

   If the ping fails, refer to Appendix J - Network Configuration in the VLP-16 User Manual for help.

3. Configure the Xavier

1. Connect the Jetson AGX Xavier to a display and connect a keyboard and mouse to the USB hub.

2. Log in to the Xavier.

3. Connect to a strong Wi-Fi network.

4. Install dependencies on the Xavier:

   1. Install “ssh” on the Xavier:

      Copy

      Copied!

                  
      
                  
      sudo apt-get install ssh
              

   2. Install “jstest-gtk” on the Xavier. This is required to configure and calibrate the joystick controller.

      Copy

      Copied!

                  
      
                  
      sudo apt-get install jstest-gtk
              

5. Set up the Xavier to use GPIO UART_2 pins for serial communication. To make this possible, you need to disable the serial console and modify permissions for /dev/ttyTHS0. Note that UART_2 appears as ttyTHS0 on Jetson AGX Xavier and ttyTHS1 on Jetson Nano.

   1. Disable the serial console:

      Copy

      Copied!

                  
      
                  
      systemctl stop nvgetty
      systemctl disable nvgetty
      udevadm trigger
      reboot
              

   2. Add <user> to the dialout group:

      Copy

      Copied!

                  
      
                  
      sudo adduser <user> dialout
              

   3. Verify that /dev/ttyTH0 is part of the dialout group and that correct permissions are set (rw-rw----)

      Copy

      Copied!

                  
      
                  
      ls -l /dev/ | grep ttyTHS0
              

      If the group does not have read/write access, change permissions:

      Copy

      Copied!

                  
      
                  
      $ sudo chmod 660 /dev/ttyTHS0
              

6. Set the controller to pairing mode (refer to the controller manual for instructions). From Bluetooth Settings, pair and connect the joystick.

7. On the Xavier, edit the sensor configuration files for Carter to accurately reflect the position of the sensors. The following are file locations and sample configurations for the different sensors:

   Tip

   Refer to this FAQ for a description of 3D pose values.

   1. Lidar: isaac/apps/carter/robots/

      Copy

      Copied!

                  
      
                  
      "2d_carter.scan_flattener.range_scan_flattening": {
        "isaac.perception.RangeScanFlattening": {
          "height_min": -0.28,
          "height_max": 0.53
        }
      },
      "2d_carter.carter_hardware.vlp16_initializer": {
        "lidar_initializer": {
          "lhs_frame": "robot",
          "rhs_frame": "lidar",
          "pose": [1.0, 0.0, 0.0, 0.0, -0.044874, 0.0, 0.449716]
        }
      }
              

8. Modify the Carter app to use the Segway RMPLite driver.

   1. Download the Segway RMPLite driver.

Follow the steps below to create a map and configure it for the Carter application:

1. Set up the Isaac SDK on your host machine as described on the Setup page.
2. From the Isaac SDK repository on your host machine, deploy either the GMapping Application or the Cartographer application to Carter.
3. Use the Map Editor to create waypoints and restricted areas on your map.
4. Create a config and graph file for the map. Refer to the sample files in the Isaac SDK.

This section describes how to run four different sample applications on Carter.

Application 1: Random

This application instructs Carter to travel from one waypoint to another using randomly chosen waypoints.

1. From the host machine, deploy the Carter application to the robot (from the sdk/ subdirectory):

   Copy

   Copied!

               
   
               
   ./../engine/engine/build/deploy.sh -p //apps/carter:carter-pkg -d jetpack45 -h <robot_ip> --remote_user <username_on_robot> -s
           

2. SSH into the robot (using the Xavier IP).

3. Go to the /deploy/<host_username>/carter-pkg directory.

4. Run the application:

   Copy

   Copied!

               
   
               
   ./run apps/carter/carter.py --map_json <map_file> --robot_json <robot_file>
           

5. To create a map of large environments, you need to tune the map to create a dense graph. This will speed up path planning for the robot. Refer to the Global Planner section of the Tuning the Navigation Stack document for more details.

6. Go to the Isaac Websight visualization tool (https://<robot_ip>:3000) to visualize the application.

7. Switch to a different goal behavior in Websight:

   1. Go to the “goals.goal_behavior” node.
   2. Change the “desired_behavior” from “Random” to “WayPoint”, “Pose”, “Route” etc.
   3. Go to “goals.pose_as_goal”, “goals.waypoint_as_goal”, or “goals.patrol” and click Submit to activate the application with that configuration.

Application 2: Waypoint

This application moves Carter to a given goal using the isaac.navigation.MapWaypointAsGoal component.

To use the Waypoint application, add the sample Flatsim delivery application to your Carter application. This sample application is located at isaac/apps/carter/carter_delivery/carter_delivery_flatsim.app.json.

Application 3: Pose

This application moves Carter to a given goal in Websight.

The “pose_as_goal” config is part of the “goal_generators.subgraph.json” file by default. Follow these steps to enable this application:

1. Open Websight.
2. Under “goals.goal_behavior”, change the “desired_behavior” to “pose_as_goal”.
3. Add the “pose_as_goal” marker to navigate the robot using the pose.

Changing the “desired_behavior” to “pose_as_goal”

Refer to the Additional Notes section of the Interactive Markers document for more details.

Application 4: Patrol

This application instructs Carter to patrol along a pre-defined route.

First, modify the config for “patrol” mode in isaac/sdk/packages/navigation/apps/goal_generators.subgraph.json: To define a route, add waypoints that are defined in your map.

            

            
"patrol": {
  "MapWaypointsAsPlan": {
    "waypoints": [
        "kitchen",
        "atrium"
    ],
        

After adding the route to the sample JSON, run the Carter application and open Websight. Under the “goals.goal_behavior” configuration, change the “desired_behavior” to “patrol”.

Changing the “desired_behavior” to “patrol”

VLP-16 Lidar does not connect

Ensure the Lidar has been programmed with the correct IP addresses: 192.168.0.201 for itself and 192.168.0.5 for the host/Xavier. Ping the VLP-16 from the Xavier to ensure the connection is working.