Python API

Though most parts of Isaac SDK are coded in C++, you have the option to build your applications with Python. This document introduces the Python API for Isaac SDK. The Python API allows you to do the following:

  • Create, manage, and run Isaac application in Python.
  • Access recorded Isaac Log data in Python.
  • Implement complicated robotics behavior with Behavior Tree in Python.

Isaac SDK includes several sample applications coded with the Python API, for example, packages/flatsim/apps:flatsim. The example below begins with a skeleton Python app.

To begin with, a Bazel target is needed along with the Python code itself:

Copy
Copied!
            

load("@com_nvidia_isaac_sdk//bzl:py.bzl", "isaac_py_app") isaac_py_app( name = "foo_app", srcs = [ "foo_app.py" ], data = [ "foo_subgraph" ], modules=[ 'viewers' ], deps = [ "//packages/pyalice" ], )


Along with common definitions like srcs and data, modules pulls in specified modules that come with Isaac SDK.

To run the app on a PC, use following command:

Copy
Copied!
            

bazel run //apps/foo_app:foo_app


To run it on Jetson devices, first deploy it with the deploy.sh script (from the sdk/ subdirectory):

Copy
Copied!
            

./../engine/engine/build/deploy.sh -h <jetson_ip> -p //apps/foo_app:foo_app-pkg -d jetpack44


where jetson_ip is the IP of the Jetson device.

Then, on the Jetson device, run this app:

Copy
Copied!
            

./run ./apps/foo_app/foo_app.py


To create an instance of Isaac Application with the Python API, import and create the instance using ‘engine.pyalice.Application’. As with C++, the init function may take the path of the application JSON file as an argument:

Copy
Copied!
            

from engine.pyalice import Application app = Application(name="foo_app")


With the application instance, you can load Isaac subgraphs as can be done with *.app.json files:

Copy
Copied!
            

app.load("apps/foo_app/foo.subgraph.json", prefix="foo")


Besides loading pre-authored computing graphs, Python API can make things more flexible. For example, many codelets in Isaac SDK are provided by modules located in the packages folder, and these modules have to be loaded via *.app.json and *.subgraph.json files just like in C++ applications:

Copy
Copied!
            

{ "name": "foo_app", "modules": [ "message_generators", "viewers" ] }


Here the message_generators module provides dummy codelets that publish pre-configured messages for testing purposes. The viewers module provides codelets that visualizes messages in Sight.

With the Python API, besides specifying modules in JSON files, you can also load modules when creating application and/or when they are deemed necessary:

Copy
Copied!
            

app = Application(name="foo_app", modules=["message_generators"]) app.load_module('viewers')


Note

Ensure that the modules are loaded before creating instances from the codelets provided by the modules or loading any subgraph that uses the codelets.

Like its C++ counterpart, Application manages the computing graph with nodes consisting of components. Now, let’s create a node and attach a component from the ImageViewer codelet provided by the viewers module we just loaded above:

Copy
Copied!
            

node = app.add(name='viewer') component = node.add(name='ImageViewer', ctype=app.registry.isaac.viewers.ImageViewer)


Here, the app.add() function returns a node instance while the node.add() function returns a component instance. These instances can also be retrieved as follows:

Copy
Copied!
            

node = app.nodes['viewer'] component = node['ImageViewer']


Now set the config parameter of target_fps to 15fps. Refer to the isaac.viewers.ImageViewer API entry for details about its config parameters.

Copy
Copied!
            

component.config.target_fps = 15.0


Similarly, you can create a component from the CameraGenerator codelet that publishes messages and configure it as follows. Note that CameraGenerator is provided by the module of message_generators, which needs to be loaded beforehand.

Copy
Copied!
            

image_node = app.add(name='camera') camera_generator = node.add(name='CameraGenerator', ctype=app.registry.isaac.message_generators.CameraGenerator) camera_generator.config.rows = 480 camera_generator.config.cols = 640 camera_generator.config.tick_period = '15Hz'


Now we have a generator component that publishes messages and a viewer component that visualizes messages. We can connect these components so that the generated messages are sent to the viewer component:

Copy
Copied!
            

app.connect(camera_generator, "color_left", component, "color") app.connect('camera/CameraGenerator', 'color_left', 'viewer/ImageViewer', 'image')


Here, the components can be specified either with the instance mentioned above or their names.

With the code above, we now have a complete application graph. You can run it with the run() function. Calling run() without an argument allows it to run indefinitely. You can also specify that it run for a certain duration (in seconds) or stop when a specific node is not running anymore:

Copy
Copied!
            

app.run() app.run(10.0) app.run('foo_node')


In all cases, pressing Ctrl-C will stop the application.

Cask is the recording format used in Isaac SDK. Refer to :Record and Replay for recording and replaying logs. A sample application for recording logs can be found at apps/samples/camera/record_dummy.py.

Assuming that you have a recorded log in the /path/to/log/ folder, you can load the log in Python as follows:

Copy
Copied!
            

from isaac import Cask, Message cask = Cask('/path/to/log/') # List all channels recorded series = cask.channels['foo_channel']: # Looks for channel named 'foo_channel' for msg in series: # Goes through every messages one by one in recorded order print(msg.proto) print(msg.uuid) print(msg.acqtime) print(msg.pubtime)


Isaac SDK features a special module called Behavior Tree, which provides different codelets that can be used to manage other codelets for complicated application behavior. TimerBehavior, for example, can start a specific codelet and keep it running for a specified duration before shutting it down. SwitchBehavior, on the other hand, could be used to switch behavior between pre-configured modes.

Before creating and manipulating Behavior codelets, ensure the module is loaded:

Copy
Copied!
            

app.load_module('behavior_tree')


A Behavior codelet can also be managed by other Behaivor codelets–you can create quite complicated functionality by stacking Behaviors.

You can achieve more flexibility by creating and configuring these Behavior codelets with the Python API.

Please refer to :Developing Codelets in Python for sample of developing codelets with Python.

© Copyright 2018-2020, NVIDIA Corporation. Last updated on Oct 30, 2023.