NVIDIA Kaya

Kaya is a robot developed as a platform to demonstrate the power and flexibility of the Isaac Robot Engine running on the NVIDIA Jetson Nano platform. It has been intentionally designed using 3D printed parts and hobbyist components to be as accessible as possible.

../../_images/kaya_robot.jpg

The Kaya Robot

Kaya features a three-wheeled holonomic drive which allows it to move in any direction, regardless of where it is facing. A RealSense depth camera allows Kaya to see the world around it in 3D, enabling features like mapping, localization, and object recognition. An on-board IMU and position feedback from the wheel motors provide accurate odometry information. Power is provided via an easily-swapped and fast-charging power tool battery. Outfitting the Jetson Nano with a Wi-Fi/Bluetooth card allows you to interact with the robot using Isaac Sight over Wi-Fi, and also allows you to connect a gamepad controller via Bluetooth.

Use the instructions in this section to build and operate your own Kaya robot.

Parts

For a list of 3D printed parts, and STL files for creating the parts, see the Isaac Developer Downloads website.

The following is a list of the off-the-shelf components needed for building Kaya. Prices shown are approximate and subject to change by vendors. Many of these components or suitable replacements are available from a wide variety of sources, which you are encouraged to explore.

PART NAME VENDOR PART # SOURCE QTY APPROX COST
3D Printed Parts
See 3D Printed Parts List
NVIDIA Jetson Nano Developer Kit 945-13450-0000-000 NVIDIA 1 $99.00
MicroSD Card (32GB) SDSQUNC-032G-GN6MA Amazon - SanDisk 1 $8.00
Intel Dual Band Wireless-Ac 8265 W/Bt 8265NGW Amazon - Intel 1 $25.00
Dual Band Wi-Fi Antennas & IPEX4 to RP-SMA Cables CHAOHANG 6484109 Amazon 1 $13.00
Intel RealSense Depth Camera D435 961448 Intel 1 $179.00
Bosch GY-BMI160 6DOF IMU Board (Generic Breakout) GY-BMI160 Amazon / eBay / AliExpress 1 $5.00
Dynamixel MX-12W Servo Motor 902-0086-000 Robotis 3 $197.70
USB2AX v3.2a or U2D2 XE-USB2AX (for USB2AX v3.2a) or RO-902-0132-000 (for U2D2) Trossen Robotics USB2AX v3.2a or Trossen Robotics U2D2 1 $39.95 (for USB2AX v3.2a) or $49.90 (for U2D2)
Dynamixel 6 Port AX/MX Power Hub IL-3PHUB Trossen Robotics 1 $7.95
VEX 3.25” Omni-Directional Wheel 217-4775 VEX Robotics 3 $53.97
DeWALT 20V MAX 2AH Li-ion Battery DCB203 Amazon - DeWALT 1 $45.00
DeWALT 20V MAX Li-ion Battery Charger DCB107 Amazon - DeWALT 1 $34.75
5V 6A DC-DC Step-Down Regulator D24V60F5 Pololu 1 $19.95
12V 5A DC-DC Step-Down Regulator EA60-12V Amazon 1 $9.50
2.1 x 5.5mm Male Barrel Plug Pigtail B072BXB2Y8 Amazon 1 $0.94
Rocker Switch 7395K28 McMaster-Carr 1 $6.76
Terminal Block 7527K44 McMaster-Carr 1 $2.24
Terminal Block Jumpers 7527K59 McMaster-Carr 2 $0.49
Wire End/Terminal Kit HC28-6 Amazon
$10.00
18AWG Electrical Wire (Red and Black)
Amazon
$5.00
Female-Female Electrical Jumper Wires
Amazon 4 $5.00
MakerBeam 150x10x10mm Beams bamb150p06 Amazon - MakerBeam 3 $5.00
MakerBeam 25 pieces T-slot nuts inc screws 101619 Amazon - MakerBeam
$17.00
M3 18mm threaded standoff 95947A525 McMaster-Carr 3 $3.66
M3 x 35mm Socket Head Screw 91292A033 McMaster-Carr 3 $3.54
M3 x 8mm Socket Head Screw 91292A112 McMaster-Carr
$4.29
M3 x 5mm Socket Head Screw 91292A110 McMaster-Carr
$4.72
M3 Hex Nuts 90591A250 McMaster-Carr
$2.21
M2 x 6mm Phillips Screw 92010A003 McMaster-Carr
$4.39
      $812.87

Dynamixel U2D2 & USB2AX Interchangeability

To communicate with the Dynamixel servo motors, either the USB2AX or U2D2 serial converters may be used. (The U2D2 converter is available as Trossen Robotics RO-902-1032-000 for $49.90. Price is approximate and subject to change by the vendor.)

You must specify the USB device name of the converter in the usb_port parameter of the isaac.KayaBaseDriver component API so that your application can communicate with it. For the Kaya sample applications included in Isaac SDK, this is found in \apps\kaya\robots\kaya_v<version>.config.json. In the following example, the U2D2 serial converter is specified.

{
   "kaya_driver": {
      "isaac.KayaBaseDriver": {
         "usb_port": "/dev/ttyUSB0"
      }
   }
}

The devices have the following USB device names.

  • USB2AX: ttyACM0
  • U2D2: ttyUSB0

See the Component API section of this documentation describing isaac.KayaBaseDriver for more information.

Wiring Diagram

See below for a wiring diagram for the Kaya robot. You may find it useful to reference this during the assembly process. Click for a larger version.

Wiring Diagram for the NVIDIA Kaya Robot

Wiring Diagram for the Kaya Robot

Assembly Instructions

This section describes step-by-step assembly of the Kaya robot.

The underscore-delimited names that appear in parentheses in the steps below are the names of the 3D-printed parts. See Parts for links to that list.

1 Assign IDs to the Dynamixel Motors

When first purchased, the Dynamixel motors are all be assigned ID “1,” and must be assigned unique IDs and properly configured before being connected together on Kaya. This procedure requires a USB serial adapter (USB2AX or U2D2), an external 12V power source (the 12V DC-DC converter on Kaya), and a PC which on which to run the RoboPlus v1 software.

Note

RoboPlus v2 does not work with MX-12W. See http://www.robotis.us/roboplus1/ to download the RoboPlus v1 application.

You can also complete this step after the rest of Kaya is assembled. Just be aware that you should only connect one motor at a time to avoid duplicate IDs.

To assign IDs to the motors:

  1. Start the RoboPlus v1 application.

  2. Connect the USB serial converter to the desktop.

  3. Connect each Dynamixel motor to the serial adapter.

  4. Connect the Dynamixel motor to external (12V) power.

  5. Launch Dynamixel Wizard in the RoboPlus v1 application.

  6. Open Port (top menu) to detect connected motors.

  7. Click on the correct motor, assign the ID # below and wheel mode. Click Apply.

    • Left: ID 1
    • Rear: ID 2
    • Right: ID 3
  8. Repeat step 7 for each motor, and annotate the ID placard for each motor accordingly.

2 Install Dynamixel Servo Motors (3 per Robot)

  1. Insert 4 M2 nuts from the Dynamixel MX-12W motor kits into the second slot up from the horn (servo head) on all sides. Press firmly to secure.

    ../../_images/kaya2.jpg
  2. Install each motor into a motor bottom plate (Kaya_Motor_Bottom_Plate) using 4 M2 screws from the Dynamixel MX-12W motor kits.

    ../../_images/kaya2a.jpg

3 Assemble the Inside Wheel Hub (3 per Robot)

  1. Insert an 18mm M3 standoff into hex hole on the back side of the hub (Kaya_Wheel_Hub_Inside).

  2. Using 4 M2 screws from the Dynamixel MX-12W motor kits, fasten the hub (Kaya_Wheel_Hub_Inside) to the horn of the Dynamixel. Tighten evenly. Be careful not to overtighten.

  3. Repeat for each motor.

    ../../_images/kaya3.jpg

4 Cut and Crimp Power Wiring

  1. Cut 9 sections of 18 AWG (4 black, 5 red) wire to length and strip insulation from the ends (3-5mm).

  2. Attach connectors according to the table using a suitable crimping tool. Take care to ensure that all connectors are mechanically secure and make good electrical contact.

    Refer to the following image, table and the Wiring Diagram when cutting and crimping wiring.

    ../../_images/kaya4.jpg
Wire# Color Length Terminal 1 Connector Terminal 1 Connection Terminal 2 Connector Terminal 2 Connection
1 Black 140 mm Male Spade Battery B- Fork Spade Terminal Block B-
2 Red 80 mm Male Spade Battery B+ Female Spade Switch IN
3 Red 140 mm Female Spade Switch OUT Fork Spade Terminal Block B+
4 Black 120 mm Fork Spade Terminal Block B- Fork Spade 12V Regulator -IN
5 Red 120 mm Fork Spade Terminal Block B+ Fork Spade 12V Regulator +IN
6 Black 120 mm Fork Spade Terminal Block B- Bare Wire 5V Regulator -IN
7 Red 120 mm Fork Spade Terminal Block B+ Bare Wire 5V Regulator +IN
8 Black 120 mm Fork Spade 12V Regulator -OUT Bare Wire Dynamixel Hub -IN
9 Red 120 mm Fork Spade 12V Regulator +OUT Bare Wire Dynamixel Hub +IN

5 Connect Battery Terminals to the Battery Holder

  1. Place Battery terminal cables (wires #1 and #2) into slots in the battery holder (Kaya_Battery_Holder), paddles facing each other as depicted below. The B+ (red wire) should be seated in the slot nearest to the notch cut out of the battery holder. Push firmly on the top of the crimps to seat properly in the channels.

    ../../_images/kaya5.jpg
  2. Secure the cables using the terminal holder (Kaya_Battery_Terminal_Holder) and an M3 x 8mm screw. Check alignment with a battery, and use pliers to adjust the paddles until they align and seat properly. To avoid creating an electrical short between the battery terminals, cover any loose wire ends with electrical tape.

    ../../_images/kaya5a.jpg

6 Assemble the Frame

  1. Attach each motor top plate (Kaya_Motor_Top_Plate) to MakerBeams using 4 MakerBeam nuts and 4 M3 x 5mm screws at each corner. Slide MakerBeam nuts into channels first, 2 per side, and then align with the holes before fastening with screws.

    ../../_images/kaya6.jpg
  2. Between two motor top plates (Kaya_Motor_Top_Plate), install Kaya_Pusher_Mount, remembering to slide nuts into place before attaching the second motor top plate.

  3. Assemble the full triangle and tighten all 12 screws evenly. The side with the pusher mount (Kaya_Pusher_Mount) is the front of Kaya, for the purposes of the rest of this procedure.

    ../../_images/kaya6a.jpg

7 Connect the Bottom Plate Assembly to the Motor Top Plate/MakerBeam Frame

  1. Attach each motor bottom plate (Kaya_Motor_Bottom_Plate) assemblies to the frame in the correct configuration. The motor assembly tagged with Dynamixel ID 2 should sit opposite to the pusher mount (Kaya_Pusher_Mount), with the other two motor assemblies to the adjacent sides.

  2. Insert 4 M3 nuts into a motor bottom plate (Kaya_Motor_Bottom_Plate), and attach to motor top plates (Kaya_Motor_Top_Plate) with 4 M3 x 8mm screws. Repeat for each motor. Tighten all 12 screws evenly.

    ../../_images/kaya7.jpg

8 Connect Motor Wiring

  • When all of the Dynamixel MX-12W servo motors have been assigned IDs (see step 1) and have been installed in the proper locations, connect them for power and data transmission using the cables from the Dynamixel MX-12W motor kits.

    All servo motors share common power and data lines, and communicate using data packets addressed by motor ID. For this reason, they may be connected in any order, or even wired individually to the Dynamixel Power Hub. For cable management purposes the best connection order is 1 - 3 - 2.

    ../../_images/kaya8.jpg

9 Mount the IMU

  1. Solder male header pins to the IMU board. Arrange them with long ends on the labelled side of the board. Pins for 3V3, GND, SCL, and SDA will be used for I2C communication.
  2. Mount the IMU board with header pins facing towards the center of the electronics bay (Kaya_Electronics_Bay) using 2 M2 x 6mm screws, as shown.
../../_images/kaya9.jpg

10 Mount DC Regulators

  1. Solder the blue screw terminals to the Pololu 5V 6A DC-DC Step-Down Regulator.

  2. Fasten the 5V regulator and 12V regulator to the electronics bay (Kaya_Electronics_Bay) with 6 M2 x 6mm screws. Ensure the orientation of these regulators matches the image, with IN+/- on the 5V regulator on the same side as IN+/- on the 12V regulator.

    ../../_images/kaya10.jpg

11 Connect IMU Wiring

  1. Attach female-to-female electrical jumper wires to IMU leads (3V3, GND, SCL, and SDA). Make note of what color wires are attached to each pin, these will later be connected to the Jetson Nano GPIO pins.

  2. Optionally, use a dab of hot glue on the middle side of the headers to hold to the IMU pin headers firmly in place.

    ../../_images/kaya11.jpg

See Wire the BMI160 IMU to the Jetson Nano or Xavier for more information.

12 Install Wi-Fi Antennas

  1. Connect the antenna cables to the Wi-Fi antennas using the SMA screw terminal.

  2. Bend antennas to 90 degrees, and fit them in the designated channels on either side of of the base (Kaya_Nano_Base).

  3. Fasten antennas in place using zip-ties, as shown.

    ../../_images/kaya12.jpg

13 Attach the Jetson Nano Base to the Frame

  1. Orient the Jetson Nano Base (Kaya_Nano_Base) so that the NVIDIA logo faces the Kaya pusher mount (Kaya_Pusher_Mount).

  2. Slide 1 MakerBeam nut onto the top of the front MakerBeam, and 2 onto the other two MakerBeams. Fasten the front screw first, then position the other 4 down to side beams. Tighten all screws evenly.

    ../../_images/kaya13.jpg

14 Connect the Battery Chassis Assembly to the New Frame Assembly

  1. Stack the electronics bay (Kaya_Electronics_Bay) on top of the battery holder (Kaya_Battery_Holder) with the flat faces together, DC regulators face-up.

  2. Bring the top of the electronics bay (Kaya_Electronics_Bay) into contact with the bottom of the base (Kaya_Nano_Base), so that it sits in the center of the frame assembly, between the motors.

  3. Ensure that no cables are being pinched. Check that the front dynamixel cables are not on top of the 12V power supply. Then attach using 4 M3 x 35mm screws down through the base (Kaya_Nano_Base) and into 4 M3 nuts in the battery holder (Kaya_Battery_Holder).

    ../../_images/kaya14.jpg

15 Prepare the Terminal Block

  • Usinge terminal block jumpers, join the two leftmost terminal block connections together, then the two rightmost connections. This allows battery B+ and B- to be routed to the 12V and 5V DC regulators.

    Reference the Wiring Diagram when making electrical connections as outlined in the following steps.

    ../../_images/kaya15.jpg

16 Install the Power Switch

  1. Locate the B+ wire (#2, Red) from the battery holder, and feed it through the switch hole at the rear of of the base (Kaya_Nano_Base). Connect the female spade connector to the middle switch terminal.

  2. Attach a second wire (#3, Red, female spade to fork spade) to the outer switch terminal.

  3. Orient the switch so that the two terminals are closest to the inside of Kaya. Slightly bend the connected terminal wires to face the outside of Kaya as in the image. This helps them to clear the battery holder (Kaya_Battery_Holder) once installed.

  4. Feed the unattached fork spade connector through the switch hole and toward the rear of Kaya.

  5. Being careful to avoid pinched wires, snap the switch down into place.

    ../../_images/kaya16.jpg

17 Connect Regulator Inputs

  1. Pull the B- wire (#1, Black) from the battery and the B+ wire from the switch (#3, Red) to the rear of Kaya. Connect the fork spade connectors of these wires to the terminal block, referencing the Wiring Diagram. These wires should route neatly between the MakerBeams below the terminal block.

    ../../_images/kaya17a.jpg
  2. Connect wires #4 through #7 to the terminal block using fork spade connectors. Reference photos and the Wiring Diagram for cable order. Bend the forks downward after they are connected onto the terminal block for a better fit.

    ../../_images/kaya17b.jpg
  3. Tuck the other cables underneath the base (Kaya_Nano_Base) and into the electronics bay (Kaya_Electronics_Bay). Connect cables #4 and #5 to the 12V regulator IN+/-, and cables #6 and #7 to the 5V regulator IN+/-. Note that the connections on the 12V regulator are ordered +IN, -IN, -OUT, +OUT, as indicated in the Wiring Diagram.

18 Connect Regulator Outputs

  1. Attach a 2.1 x 5.5mm male barrel plug pigtail to the 5V regulator out screw terminals. Note that the Brown wire is +, and the Blue wire is -. (Center positive polarity.)

  2. Connect the fork spade connectors of cables #8 and #9 to the 12V regulator OUT- and OUT+. Connect bare wire ends to the Dynamixel Power Hub screw terminals, routing the cables either back through or around the base (Kaya_Nano_Base).

    ../../_images/kaya18.jpg

19 Mount the Terminal Block

  1. Mount the terminal block to the frame with 3 MakerBeam nuts and 2 M3 x 8mm screws. Slide one extra nut ahead of right side of terminal block for later use with the terminal block cover (Kaya_Terminal_Block_Cover).

  2. Position the Terminal Block as far back as possible by using only the front set of mounting holes.

    ../../_images/kaya19.jpg

20 Mount the Dynamixel Power Hub

  1. Connect the loose Dynamixel cable from servo motor ID3 to any port on the Dynamixel Power Hub.

  2. Use another Dynamixel cable to connect the USB2AX to the Dynamixel Power Hub. Mount the Dynamixel Power Hub to the base (Kaya_Nano_Base) with 2 M2x6 screws, as shown below.

    ../../_images/kaya20.jpg

21 Connect the Wi-Fi Card

  • Connect the small antenna cable connectors (U.FL / AMC / UMCC / IPX) to the Wi-Fi card. These should snap into place with vertical pressure, but can be difficult to work with. It may be helpful to lay the card on a flat surface for best leverage. Take care not to damage the connectors on the card, and avoid disconnecting and reconnecting if possible.

    ../../_images/kaya21.jpg

22 Install the Wi-Fi Card

  1. Remove the Jetson Nano module from the carrier board by first removing 2 Phillips screws, then the 2 side latches.
  2. Connect the Wi-Fi card and secure with the included screw.
../../_images/kaya22a.jpg
  1. Reattach and secure the Jetson Nano module the carrier board.

23 Mount the Jetson Nano

  1. Mount the Jetson Nano Developer Kit to Kaya_Nano_Base using 4 M2 x 6mm screws, as shown in the image below. For cable management, it may be helpful to coil the antenna cables beneath the Jetson Nano.
  2. Add a jumper across the header pins labeled “ADD JMPR TO DISABLE USB PWR” on the carrier board to allow the Jetson Nano to be powered via the 5V barrel jack connection.
../../_images/kaya23.jpg

24 Mount RealSense Camera

  1. Attach the RealSense camera to Kaya_Realsense_Mount with 2 M3 x 5mm screws.

  2. Use 3 M3 x 8mm screws and 3 M3 nuts to attach the camera mount (Kaya_Realsense_Mount) to the camera bridge (Kaya_Realsense_Bridge).

  3. Connect the USB-C cable to the RealSense camera.

    ../../_images/kaya24a.jpg ../../_images/kaya24b.jpg

25 Attach the RealSense Bridge to the Jetson Nano Base

  • Use 4 M3 x 8mm and 4 M3 nuts to attach the camera bridge (Kaya_Realsense_Bridge) to the base (Kaya_Nano_Base).

    ../../_images/kaya25.jpg

26 Mount the Wheels

  • Attach each Omniwheel using 1 M3 x 35mm screw through the outside wheel hub (Kaya_Wheel_Hub_Outside), the wheel itself, and then into the inside hub (Kaya_Wheel_Hub_Inside). Take care not to overtighten.

    Threadlocker may be used to prevent loosening of these fasteners.

    ../../_images/kaya26.jpg

27 Attach the Terminal Block Cover

  • Using the extra MakerBeam nut from step 19, attach the terminal block cover (Kaya_Terminal_Block_Cover) using a M3 x 5mm screw.

    ../../_images/kaya27.jpg

28 Connect Cables to Jetson Nano

  1. Connect the RealSense camera and USB2AX to the top 2 USB connectors on the Nano.

  2. Ensure the power switch is not set to “ON”.

  3. Route the barrel plug cable from the 5V regulator behind the terminal block cover (Kaya_Terminal_Block_Cover) and plug into the barrel jack input of Jetson Nano.

    Note

    Never connect the 5V barrel plug cable to the barrel jack input of the Dynamixel Power Hub. This may cause irreversible damage to the voltage regulators.

  4. Connect the jumper wires from the IMU to the Jetson Nano GPIO pins, referencing the Wiring Diagram and your earlier notes on wire colors.

    ../../_images/kaya28.jpg

29 Clean Up Cables and Power On

  1. Tuck the front Dynamixel cables up in front of 12V regulator and MakerBeam.

  2. Zip-tie other Dynamixel and power supply cables as desired.

  3. Install Battery and ensure it snaps into place. If there is resistance, check for cables and any remaining 3D printed support material, then ensure your terminals are not bent.

  4. Install the pusher (Kaya_Pusher) by sliding it down into the pusher mount (Kaya_Pusher_Mount).

    Congratulations, you have fully assembled a Kaya robot! Have fun tinkering with your new creation and the Isaac Robot Engine!

    ../../_images/kaya29.jpg

Notes on Power-Hungry Applications

You may encounter issues when running more demanding applications on Jetson Nano, especially when running in “MAX-N” mode.

Be careful that 3D printed PLA plastic parts do not come into contact with the heatsink if there are concerns with Jetson Nano overheating. Kaya is not equipped with a fan to aid with cooling. The Jetson Nano Developer Kit provides a 4-pin connector to accommodate 5V PWM fans. However, adding a fan to Kaya requires removal or custom modification of several 3D printed parts.

Raising the nominal output voltage of the 5V regulator can help safeguard against the effects of transient voltage dips. This can be accomplished by soldering a thru-hole resistor between the FB and GND pins on the Pololu 5V regulator board. A resistor value of 82.5 kOhm increases the nominal output voltage to 5.143V.

../../_images/kaya_opt_a.jpg

The following image shows the resistor correctly installed.

../../_images/kaya_opt_b.jpg

A capacitor (100-1000 uF) may also be placed across the regulator output terminals to further safeguard against transient voltage fluctuations. For more information, please reference the datasheet for the Microchip MIC2101.

Running Isaac SDK on Kaya

Once you have assembled a your Kaya, use the procedures in Getting Started With Jetson Nano get started with Isaac on the Jetson Nano, and deploy and run a couple of sample applications.

Note

A Kaya robot assembled using the listed parts and procedures in this documentation is version 2. Applications for this robot must load the version-appropriate configuration, as shown in the following lines:

"config_files": [
  "apps/kaya/robots/kaya_v2.config.json"
],

The Joystick Application

The following steps deploy a simple joystick application that can be used to control the Kaya robot.

  1. Deploy //apps/kaya:joystick-pkg to the robot as explained in Deploying and Running on Jetson.

  2. Change to the directory on your Nano and run the application with the following commands:

    cd deploy/<your_username>/joystick-pkg/
    ./apps/kaya/joystick
    

The Follow Me Application

The following steps deploy an application to have the Kaya robot autonomously move towards a designated AprilTag fiducial. This application combines AprilTag detection, path planning, control, and the Kaya driver. It requires a working realsense camera for AprilTag detection and path planning, and a joystick to enable autonomous mode.

  1. Deploy //apps/kaya:follow_me-pkg to the robot as explained in Deploying and Running on Jetson.

  2. Change to the directory on your Nano and run the application with the following commands:

    cd deploy/<your_username>/follow_me-pkg/
    ./apps/kaya/follow_me
    
  3. Open Isaac Sight on the desktop browser at <nano_ip>:3000. In the Application Configuration panel on the right, click on fiducial_as_goal, and change the target_fiducial_id (default is “tag36h11_9”) to the id of the AprilTag in use.

    If you put the AprilTag within the realsense camera’s field of view, you should see in the Follower Kaya - Camera window that the april tag is detected, and a planned path shown in a blue line from the Kaya robot to the april tag.

  4. If you hold down the R1 button on the joystick, the Kaya robot enters autonomous mode, and moves towards the april tag. Move the april tag around while keeping it within the camera’s view, and Kaya follows it around.

The Object Detection Application

This application uses YOLO (tiny-yolov3) on Kaya to perform object detection. The YOLO model is trained to recognize a few common objects in these three classes for trash classification: compost (apple, banana), metal (soda can), paper (paper box and small cardboard box). Here are the steps for running this application.

  1. Deploy //apps/kaya:object_detection-pkg to the robot as explained in Deploying and Running on Jetson.

  2. Change to the directory on your Nano and run the application with the following commands:

    cd deploy/<your_username>/object_detection-pkg/
    ./apps/kaya/object_detection_kaya
    

    Open Isaac Sight on the desktop browser at <nano_ip>:3000. You should be able to see the live camera image.

  3. Place an apple or a soda can in front of Kaya’s camera, and you should see a bounding box around the object labeled as compost or metal, respectively.

The Mapping Application

The gmapping application demonstrates use of the Kaya robot to build an occupancy map of its environment. Kaya uses the Intel RealSense camera to perceive depth of obstacles, and wheel odometry and IMU to update its own state. Obstacle depth and robot state information are published to the host to build up a map.

This application has two parts: gmapping_distributed_kaya that runs on the Kaya robot, and gmapping_distributed_host that runs on the host machine.

  1. Deploy //apps/kaya:gmapping_distributed_kaya-pkg to the robot as explained in Deploying and Running on Jetson.

  2. Change to the directory on your Nano and run the application with the following commands:

    cd deploy/<your_username>/gmapping_distributed_kaya-pkg/
    ./apps/kaya/gmapping_distributed_kaya
    

    Open Isaac Sight on the desktop browser at <nano_ip>:3000. You should be able to see the camera RGB and depth image, as well as plot of the robot state.

  3. Use the joystick to drive Kaya around and you should see the robot state being updated.

  4. On the desktop system, open app/kaya/gmapping_distributed_host.app.json and change the tcp-subscriber host from “YOUR_NANO_IP_HERE” to the IP address of the Jetson Nano on the robot (<nano_ip>).

  5. Build and run the host application with the following commands:

    bazel build //apps/kaya:gmapping_distributed_host
    bazel run //apps/kaya:gmapping_distributed_host
    
  6. Open Isaac Sight on the desktop at localhost:3000. You should be able to see the map Kaya is building. Use the joystick to drive Kaya around and observe the map updates.

    If you see performance issues with the map update, go to the Kaya page of Isaac Sight webpage and disable camera RGB and depth visualization (uncheck Channels - viewer).

IMU and Motor Permissions

Kaya uses an IMU that connects using I2C protocol and motors that are connected through USB. Therefore the following commands need to be executed on Nano once. Please start a new terminal after running these commands.

sudo adduser $USER dialout
sudo adduser $USER i2c

For more information on the IMU, see the Wire the BMI160 IMU to the Jetson Nano or Xavier section of this document.