Python API

Though most parts of Isaac SDK are coded in C++, you have the option to build your applications with Python. This document introduces the Python API for Isaac SDK. The Python API allows you to do the following:

• Create, manage, and run Isaac application in Python.
• Access recorded Isaac Log data in Python.
• Implement complicated robotics behavior with Behavior Tree in Python.

Isaac SDK includes several sample applications coded with the Python API, for example, packages/flatsim/apps:flatsim. The example below begins with a skeleton Python app.

To begin with, a Bazel target is needed along with the Python code itself:

            

            
load("@com_nvidia_isaac_sdk//bzl:py.bzl", "isaac_py_app")

isaac_py_app(
    name = "foo_app",
    srcs = [ "foo_app.py" ],
    data = [ "foo_subgraph" ],
    modules=[ 'viewers' ],
    deps = [ "//packages/pyalice" ],
)
        

Along with common definitions like srcs and data, modules pulls in specified modules that come with Isaac SDK.

To run the app on a PC, use following command:

            

            
bazel run //apps/foo_app:foo_app
        

To run it on Jetson devices, first deploy it with the deploy.sh script (from the sdk/ subdirectory):

            

            
./../engine/engine/build/deploy.sh -h <jetson_ip> -p //apps/foo_app:foo_app-pkg -d jetpack45
        

where jetson_ip is the IP of the Jetson device.

Then, on the Jetson device, run this app:

            

            
./run ./apps/foo_app/foo_app.py
        

To create an instance of Isaac Application with the Python API, import and create the instance using ‘engine.pyalice.Application’. As with C++, the init function may take the path of the application JSON file as an argument:

            

            
from engine.pyalice import Application

app = Application(name="foo_app")
        

With the application instance, you can load Isaac subgraphs as can be done with *.app.json files:

            

            
app.load("apps/foo_app/foo.subgraph.json", prefix="foo")
        

Besides loading pre-authored computing graphs, Python API can make things more flexible. For example, many codelets in Isaac SDK are provided by modules located in the packages folder, and these modules have to be loaded via *.app.json and *.subgraph.json files just like in C++ applications:

            

            
{
  "name": "foo_app",
  "modules": [
    "message_generators",
    "viewers"
  ]
}
        

Here the message_generators module provides dummy codelets that publish pre-configured messages for testing purposes. The viewers module provides codelets that visualizes messages in Sight.

With the Python API, besides specifying modules in JSON files, you can also load modules when creating application and/or when they are deemed necessary:

            

            
app = Application(name="foo_app", modules=["message_generators"])
app.load_module('viewers')
        

Like its C++ counterpart, Application manages the computing graph with nodes consisting of components. Now, let’s create a node and attach a component from the ImageViewer codelet provided by the viewers module we just loaded above:

            

            
node = app.add(name='viewer')
component = node.add(name='ImageViewer',
                     ctype=app.registry.isaac.viewers.ImageViewer)
        

Here, the app.add() function returns a node instance while the node.add() function returns a component instance. These instances can also be retrieved as follows:

            

            
node = app.nodes['viewer']
component = node['ImageViewer']
        

Now set the config parameter of target_fps to 15fps. Refer to the isaac.viewers.ImageViewer API entry for details about its config parameters.

            

            
component.config.target_fps = 15.0
        

Similarly, you can create a component from the CameraGenerator codelet that publishes messages and configure it as follows. Note that CameraGenerator is provided by the module of message_generators, which needs to be loaded beforehand.

            

            
image_node = app.add(name='camera')
camera_generator = node.add(name='CameraGenerator',
                            ctype=app.registry.isaac.message_generators.CameraGenerator)
camera_generator.config.rows = 480
camera_generator.config.cols = 640
camera_generator.config.tick_period = '15Hz'
        

Now we have a generator component that publishes messages and a viewer component that visualizes messages. We can connect these components so that the generated messages are sent to the viewer component:

            

            
app.connect(camera_generator, "color_left", component, "color")
app.connect('camera/CameraGenerator', 'color_left', 'viewer/ImageViewer', 'image')
        

Here, the components can be specified either with the instance mentioned above or their names.

With the code above, we now have a complete application graph. You can run it with the run() function. Calling run() without an argument allows it to run indefinitely. You can also specify that it run for a certain duration (in seconds) or stop when a specific node is not running anymore:

            

            
app.run()

app.run(10.0)

app.run('foo_node')
        

In all cases, pressing Ctrl-C will stop the application.

Cask is the recording format used in Isaac SDK. Refer to :Record and Replay for recording and replaying logs. A sample application for recording logs can be found at apps/samples/camera/record_dummy.py.

Assuming that you have a recorded log in the /path/to/log/ folder, you can load the log in Python as follows:

            

            
from isaac import Cask, Message
cask = Cask('/path/to/log/')

# List all channels recorded
series = cask.channels['foo_channel']:    # Looks for channel named 'foo_channel'
for msg in series:                        # Goes through every messages one by one in recorded order
   print(msg.proto)
   print(msg.uuid)
   print(msg.acqtime)
   print(msg.pubtime)
        

Isaac SDK features a special module called Behavior Tree, which provides different codelets that can be used to manage other codelets for complicated application behavior. TimerBehavior, for example, can start a specific codelet and keep it running for a specified duration before shutting it down. SwitchBehavior, on the other hand, could be used to switch behavior between pre-configured modes.

Before creating and manipulating Behavior codelets, ensure the module is loaded:

            

            
app.load_module('behavior_tree')
        

A Behavior codelet can also be managed by other Behaivor codelets–you can create quite complicated functionality by stacking Behaviors.

You can achieve more flexibility by creating and configuring these Behavior codelets with the Python API.

Please refer to :Developing Codelets in Python for sample of developing codelets with Python.