Though most parts of Isaac SDK are coded in C++, you have the option to build your applications with Python. This document introduces the Python API for Isaac SDK. The Python API allows you to do the following:
- Create, manage, and run Isaac application in Python.
- Access recorded Isaac Log data in Python.
- Implement complicated robotics behavior with Behavior Tree in Python.
Isaac SDK includes several sample applications coded with the Python API, for example,
packages/flatsim/apps:flatsim
. The example below begins with a skeleton Python app.
To begin with, a Bazel target is needed along with the Python code itself:
load("@com_nvidia_isaac_sdk//bzl:py.bzl", "isaac_py_app")
isaac_py_app(
name = "foo_app",
srcs = [ "foo_app.py" ],
data = [ "foo_subgraph" ],
modules=[ 'viewers' ],
deps = [ "//packages/pyalice" ],
)
Along with common definitions like srcs
and data
, modules
pulls in
specified modules that come with Isaac SDK.
To run the app on a PC, use following command:
bazel run //apps/foo_app:foo_app
To run it on Jetson devices, first deploy it with the deploy.sh
script (from the
sdk/
subdirectory):
./../engine/engine/build/deploy.sh -h <jetson_ip> -p //apps/foo_app:foo_app-pkg -d jetpack45
where jetson_ip
is the IP of the Jetson device.
Then, on the Jetson device, run this app:
./run ./apps/foo_app/foo_app.py
To create an instance of Isaac Application with the Python API, import and create the instance
using ‘engine.pyalice.Application’. As with C++, the init
function may take the path of the
application JSON file as an argument:
from engine.pyalice import Application
app = Application(name="foo_app")
With the application instance, you can load Isaac subgraphs as can be done with
*.app.json
files:
app.load("apps/foo_app/foo.subgraph.json", prefix="foo")
Besides loading pre-authored computing graphs, Python API can make things more flexible.
For example, many codelets in Isaac SDK are provided by modules located in the packages
folder, and these modules have to be loaded via *.app.json
and *.subgraph.json
files
just like in C++ applications:
{
"name": "foo_app",
"modules": [
"message_generators",
"viewers"
]
}
Here the message_generators
module provides dummy codelets that publish pre-configured
messages for testing purposes. The viewers
module provides codelets that visualizes messages
in Sight.
With the Python API, besides specifying modules in JSON files, you can also load modules when creating application and/or when they are deemed necessary:
app = Application(name="foo_app", modules=["message_generators"])
app.load_module('viewers')
Ensure that the modules are loaded before creating instances from the codelets provided by the modules or loading any subgraph that uses the codelets.
Like its C++ counterpart, Application manages the computing graph with nodes consisting of
components. Now, let’s create a node and attach a component from the ImageViewer
codelet provided by the viewers
module we just loaded above:
node = app.add(name='viewer')
component = node.add(name='ImageViewer',
ctype=app.registry.isaac.viewers.ImageViewer)
Here, the app.add()
function returns a node instance while the node.add()
function
returns a component instance. These instances can also be retrieved as follows:
node = app.nodes['viewer']
component = node['ImageViewer']
Now set the config parameter of target_fps
to 15fps. Refer to the
isaac.viewers.ImageViewer API entry for details about its config parameters.
component.config.target_fps = 15.0
Similarly, you can create a component from the CameraGenerator
codelet that publishes
messages and configure it as follows. Note that CameraGenerator
is provided by the
module of message_generators
, which needs to be loaded beforehand.
image_node = app.add(name='camera')
camera_generator = node.add(name='CameraGenerator',
ctype=app.registry.isaac.message_generators.CameraGenerator)
camera_generator.config.rows = 480
camera_generator.config.cols = 640
camera_generator.config.tick_period = '15Hz'
Now we have a generator component that publishes messages and a viewer component that visualizes messages. We can connect these components so that the generated messages are sent to the viewer component:
app.connect(camera_generator, "color_left", component, "color")
app.connect('camera/CameraGenerator', 'color_left', 'viewer/ImageViewer', 'image')
Here, the components can be specified either with the instance mentioned above or their names.
With the code above, we now have a complete application graph. You can run it with the run()
function. Calling run()
without an argument allows it to run indefinitely. You can also
specify that it run for a certain duration (in seconds) or stop when a specific node is not running anymore:
app.run()
app.run(10.0)
app.run('foo_node')
In all cases, pressing Ctrl-C will stop the application.
Cask is the recording format used in Isaac SDK. Refer to :Record and Replay for recording and
replaying logs. A sample application for recording logs can be found at
apps/samples/camera/record_dummy.py
.
Assuming that you have a recorded log in the /path/to/log/
folder, you can load the log in
Python as follows:
from isaac import Cask, Message
cask = Cask('/path/to/log/')
# List all channels recorded
series = cask.channels['foo_channel']: # Looks for channel named 'foo_channel'
for msg in series: # Goes through every messages one by one in recorded order
print(msg.proto)
print(msg.uuid)
print(msg.acqtime)
print(msg.pubtime)
Isaac SDK features a special module called Behavior Tree, which provides different codelets that can be used to manage other codelets for complicated application behavior. TimerBehavior, for example, can start a specific codelet and keep it running for a specified duration before shutting it down. SwitchBehavior, on the other hand, could be used to switch behavior between pre-configured modes.
Before creating and manipulating Behavior codelets, ensure the module is loaded:
app.load_module('behavior_tree')
A Behavior codelet can also be managed by other Behaivor codelets–you can create quite complicated functionality by stacking Behaviors.
You can achieve more flexibility by creating and configuring these Behavior codelets with the Python API.
Please refer to :Developing Codelets in Python for sample of developing codelets with Python.