Developing Codelets in Python

While in terms of performance, the best language for writing codelets is C++, not all codelets of an application need to be in the same language. The Isaac SDK also supports Python codelets, or pyCodelets, for those who are more familiar with Python. This section shows you how to do the following:

  • Run Python codelets, using ping_python included in the Isaac SDK as an example
  • Create Python codelets

This section also describes the run script deployed with Python codelets to the target system, and the differences between JSON and Bazel BUILD files for C++ codelets and JSON and Bazel BUILD files for Python codelets.

A Python version of the Ping codelet described in the Basics section can be found in the apps/tutorials/ping_python/ directory.

This application can be run on your system by executing the following command:


bob@desktop:~/isaac/sdk$ bazel run //apps/tutorials/ping_python

If you want to run the application on a Jetson device you can have to follow these instructions

  1. Deploy //apps/tutorials/ping_python:ping_python-pkg to the robot as explained in Deploying and Running on Jetson.

  2. Change to the directory of the deployed package on Jetson with the following command:


    bob@desktop:~/$ cd ~/deploy/bob/ping_python-pkg Where "bob" is your username on the host system.

  3. Run the application by executing the following command:


    bob@desktop:~/deploy/bob/ping_python-pkg/$ ./run apps/tutorials/ping_python/

When you run the codelet, by either method, a “Hello World!” message is printed every 1.5 seconds. Modify the script at apps/tutorials/ping_python/ and run it again to see the effects of your changes.


If a “ImportError: No module named capnp” error is displayed, make sure pycapnp is installed with the install_dependencies_jetson script mentioned in Installing Dependencies on Robots.

A more complete example, the Python version of the Proportional Control codelet described in the A Complete Application section is shown below. The following Python script is functionally equivalent to a combination of main.cpp, ProportionalControlCpp.hpp, and ProportionalControlCpp.cpp:


''' Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved. NVIDIA CORPORATION and its licensors retain all intellectual property and proprietary rights in and to this software, related documentation and any modifications thereto. Any use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from NVIDIA CORPORATION is strictly prohibited. ''' from isaac import * # A Python codelet for proportional control # For comparison, please see the same logic in C++ at "ProportionalControlCpp.cpp". # # We receive odometry information, from which we extract the x position. Then, using refence and # gain parameters that are provided by the user, we compute and publish a linear speed command # using `control = gain * (reference - position)` class ProportionalControlPython(Codelet): def start(self): # This part will be run once in the beginning of the program # Input and output messages for the Codelet. We'll make connections in the json file. self.rx = self.isaac_proto_rx("Odometry2Proto", "odometry") self.tx = self.isaac_proto_tx("StateProto", "cmd") # DifferentialBaseControl type # Print some information print("Please head to the Sight website at <IP>:<PORT> to see how I am doing.") print("<IP> is the Internet Protocol address where the app is running,") print("and <PORT> is set in the config file, typically to '3000'.") print("By default, local link is 'localhost:3000'.") # We can tick periodically, on every message, or blocking. See documentation for details. self.tick_periodically(0.01) def tick(self): # This part will be run at every tick. We are ticking periodically in this example. # Try to get the odometry message. # Nothing to do if we haven't received odometry data yet rx_message = self.rx.message if rx_message is None: return # Read parameters that can be set through Sight webpage reference = self.config["desired_position_meters"] gain = self.config["gain"] # Read odometry message received position = rx_message.proto.odomTRobot.translation.x # Compute the control action control = gain * (reference - position) # Show some data in Sight"reference (m)", reference)"position (m)", position)"control", control)"gain", gain) # Publish control command tx_message = self.tx.init() data = tx_message.proto.init('data', 2) data[0] = control # linear speed data[1] = 0.0 # This simple example sets zero angular speed self.tx.publish() def main(): app = Application( "apps/tutorials/proportional_control_python/") app.nodes["py_controller"].add(ProportionalControlPython) app.connect('odometry/isaac.navigation.DifferentialBaseOdometry', 'odometry', 'py_controller/PyCodelet', 'odometry') app.connect('py_controller/PyCodelet', 'cmd', 'commander.subgraph/interface', 'control') if __name__ == '__main__': main()

Import statements in Python are analogous to preprocessor #include statements in C++. Like in ProportionalControlCpp.cpp, the codelet is defined in start and tick functions. The Isaac parameters desired_position_meters and gain are used, with values either configured in JSON files or set through Sight at runtime.

At every tick, if an odometry message is received, the appropriate command is computed and published for the robot. Some important data is displayed in Sight.

The main function simply loads the graph and configuration files before running the application, the way that main.cpp does in the C++ codelet. In addition, the Python main function also creates the edges for the PyCodelet.

Follow a procedure similar to the following when creating your Python codelets.

Create a Workspace

  1. Copy apps/tutorials/proportional_control_python/ or another existing Python-based codelet to apps/<your_app_name> as a template or starting point.

    If you use the Proportional Control codelet unmodified for this tutorial, a Carter robot or equivalent is required. See NVIDIA Carter for more information.

  2. Rename the files to reflect the name of your codelet instead of the codelet you copied.

Create a Bazel BUILD File

  1. In apps/<your_app_name>/BUILD copied in with the other files used as a starting point, replace all proportional_control_python strings with <your_app_name>.

  2. Modify the data property in the py_binary rule depending on the C++ codelets you use.

    For example, if you were to omit or remove //packages/segway in apps/tutorials/proportional_control_python/BUILD, and run the codelet, the error Component with typename ‘isaac::SegwayRmpDriver’ not registered would be displayed, because the Proportional Control codelet (proportional_control_python.graph.json) uses the C++ based segway codelet.

  3. Since our application is located in apps and not apps/tutorials, remove the specification of //apps/tutorials:py_init, leaving //apps:py_init in place.

    If instead of moving the application up to apps, you move it to apps/tutorials/tutorials_sub, the BUILD file in apps/tutorials/tutorials_sub must specify py_init in all three directories, //apps:py_init, //apps/tutorials:py_init, and //apps/tutorials/tutorials_sub:py_init. Each directory would also need a copy of

Create a Python Codelet

  1. In your <your_app_name>.py, replace import apps.tutorials.proportional_control_python with import apps.<your_app_name>.

  2. Rename and modify the ProportionalControlPython class as needed. You can define multiple Python codelets in this file.

  3. In the main function, replace all proportional_control_python strings with <your_app_name>. You must register all pyCodelets using their class names, such as ProportionalControlPython in the files we used as a starting point. Modify node names, py_controller in this case, to match the name you chose in your graph.json file.

    Your main function will be similar to the following:


def main(): app = Application(app_filename = "") app.register({"my_py_node1": PyCodeletType1, "my_py_node2a": PyCodeletType2, "my_py_node2b": PyCodeletType2})

  1. Add or remove nodes, components, or edges in the graph section of apps/<your_app_name>/<your_app_name>.app.json depending on your codelet.
  2. Configure nodes and components in the config section of apps/<your_app_name>/<your_app_name>.app.json as needed. Make sure to replace all instances of the codelet name with the name of your new codelet.

Run the codelet locally or deploy and run it on a Jetson system as described in Running a Python Codelet.

The run script, provided along with deployment (using of an Isaac application that includes a Python codelet or codelets, performs the following functions:

  • Checks that the filename of the Python script ends in “.py”

  • Verifies that every directory has an file

  • Runs the application using the following command.


    PYTHONPATH=$PWD:$PWD/packages/pyalice:$PWD/packages/ml python

These functions are performed by the run script when we use the following command:


./run apps/tutorials/proportional_control_python/

JSON files for Python codelets are very similar to those for C++ codelets, except that the component type of Python codelets is always isaac::alice::PyCodelet.

Bazel BUILD files are somewhat different, as shown in the following example:


load("@com_nvidia_isaac_sdk//bzl:module.bzl", "isaac_pkg") py_binary( name = "proportional_control_python", srcs = [ "", "", ], data = [ "proportional_control_python.config.json", "proportional_control_python.graph.json", "//apps:py_init", "//apps/tutorials:py_init", "//packages/", "//packages/", "//packages/", ], deps = ["//packages/pyalice"], ) isaac_pkg( name = "proportional_control_python-pkg", srcs = ["proportional_control_python"], )

Use of C++ codelets is enabled by specifying the corresponding modules in data in the py_binary rule. For example, //packages/ is required to use C++ Codelet of type isaac::SegwayRmpDriver. Omitting or forgetting this dependency causes the error Component with typename ‘isaac::SegwayRmpDriver’ not registered to be displayed when the application is run.

The isaac_pkg rule is responsible for packing all the files up and creating an archive which can be transferred to the target device with the deploy script.

© Copyright 2018-2020, NVIDIA Corporation. Last updated on Oct 30, 2023.