Python Codelets

Python Codelets allow users to build parts of their application in Python. This section gives an overview of the how to implement codelets in python and how to use them in an application.

Codelets are user implemented units which can be run as part of the graph. Python codelets allow the users to implement this functionality in python. Like C++ codelets, python codelets have the following functions users can implement:

• start() - called once when the codelet starts

• tick() - called on every tick

• stop() - called once when the codelet is stopped

Unlike the C++ codelets, all the python codelets are registered via a same interface namely: nvidia::gxf::PyCodeletV0.

To implement a codelet in python, users have to implement the class CodeletAdapter. As explained above, the users implement start(), stop() and tick() methods in Python. This class also provides basic functionality to obtain the parameters.

Running an application containing a Python Codelet has to be done using gxe.py. Graphs containing Python Codelets cannot run directly using the gxe binary because a python interpreter has to be started before running any Python Codelet.

A graph can also be run if graph-composer and regsitry is already installed. Use the following command to install the relevant extensions:

registry graph insall -g path/to/graph.yaml -m output/path/to/generated/manifest.yaml -d path/to/target.yaml --output-directory output/directory

A sample target file can be found in /opt/nvidia/graph-composer/.

The above command will install the relevant extensions and other required files to output/directory.

Once that is done, run the following command:

cp /opt/nvidia/graph-composer/core.so /opt/nvidia/graph-composer/core.py output/directory/gxf

Finally add output/directory to PYTHONPATH using the following command:

export PYTHONPATH=output/directory

Running a graph using gxe.py:

python3 output/directory/gxf/std/gxe.py --app path/to/graph.yaml --manifest path/to/manifest/file.yaml

Following are some of the Python Codelet examples which can be found in gxf/python/tests:

Ping Codelets demonstrate the basic usage of python codelets

1. PingTx.py

   • Constructs an empty message which is an Entity

   • Transmits it on the first transmitter

2. PingRx.py

   • Receives a message on first transmitter

   • Passes if it’s non-null else raises an exception

Obtaining tensor data in python

3. VerifyEqual.py

   • This codelet has two receivers.

   • It receives two messages one from each receiver.

   • Extracts tensor data from both the message.

   • Copies the data on the host if the tensor is on the device.

   • Asserts that the data on the tensors is equal.

Generating tensor data from python

4. StreamGenerator

   • On every tick, this codelet generates four tensors: two on host and two on device.

   • It uses TensorDescription object to reshape the tensor to desired shape.

   • Creates a host message and a device message.

   • Adds the device tensors to device messages and host tensors to host messages

   • Publishes the host message on the first transmitter and the device message on the second transmitter.

Please refer to the sample graph files present in the gxf/python/tests directory for examples on how to use the Python Codelets in an application.

General Concepts

Python enables users to add a lot of functionality with very less boiler plate code and hence is heavily used in the Machine Learning Community.

For e.g. simulating a sensor for data, vizualizing the generated output or running various ML models on the data can be easily implemented in python. Python Codelets allow users to implement these functionalities in python.

Unlike C++ codelets, python codelets are not registered individually. To create a python codelet users implement CodeletAdapter (a python base class) and all python codelets are registered in the registry as PyCodeletV0 which is a classic C++ codelet. This C++ codelet calls the start(), tick() and stop() methods of the python codelet implementation.

PyCodeletV0	User’s Python Codelet
start()	start()
tick()	tick()
stop()	stop()

Implenting a Python Codelet

Implementing Class

To implement a python codelet, users implement CodeletAdapter. This is available in gxf.python_codelet.codelet. Specifically, users implement the following functions:

start() - Setting up the codelet. Called once when the codelet starts.

tick() - Business logic. Called based on Scheduling Terms.

stop() - Clean up. Called when entity containing the python codelet is terminated.

Adding Python Codelet to the Graph

Adding a python codelet to the graph is different from adding a C++ codelet. Unlike C++ codelets, the type of python codelet is nvidia::gxf::PyCodeletV0 and not nvidia::gxf::Codelet. Linking the implementation of the codelet is done via params directly. Also, python codelets only support registering only a selected types of parameters:

nvidia::gxf::PyCodeletV0 accepts the following parameters:

Parameter	Mandatory/Optional	Description
codelet_name	Mandatory	Name of the user’s python codelet class which implements CodeletAdapter
codelet_file	Mandatory	Absolute path to the file containing the implementation
receivers	Optional	A list of receivers
trasnmitters	Optional	A list of transmitters
allocators	Optional	A list of allocators
cuda_stream_pools	Optional	A list of cuda stream pools
codelet_params	Optional	A string which the users can parse for setting additional params

Following is an entity named rx which contains the following components:

• A python codelet called python_receiver

• A DoubleBufferReceiver called signal

• A nameless MessageAvailableSchedulingTerm

The implementation of the python codelet is present in some/path/to/PythonCodelets.py file under the class called PingRx which should implement CodeletAdapter. The python codelet also accept two other parameters:

• receivers: A list containing single item, signal, which is a double buffer receiver component.

• codelet_params: a custom string which the user can parse in the start(), tick() or stop() method and use accordingly.

---
name: rx
components:
- name: signal
type: nvidia::gxf::DoubleBufferReceiver
- type: nvidia::gxf::MessageAvailableSchedulingTerm
parameters:
   receiver: signal
   min_size: 1
- name: python_receiver
type: nvidia::gxf::PyCodeletV0
parameters:
   codelet_name: "PingRx"
   codelet_file: "some/path/to/PingRx.py"
   receivers: [signal]
   codelet_params: "log_count=5"

Accessing Parameters

CodeletAdapter implements the following methods which let’s users access the parameters using self.method_name():

Method	Description
get_allocators()	returns a list of gxf.StandardExtension.Allocator objects
get_cuda_stream_pools()	returns a list of gxf.CudaExtension.CudaStreamPool objects
get_transmitters()	returns a list of gxf.StandardExtension.Transmitter objects
get_receivers()	returns a list of gxf.StandardExtension.Receiver objects
get_clock()	returns a gxf.StandardExtension.Clock
get_params()	returns a string containing the additional params

CodeletAdapter also implements the following utility methods:

Method	Description
eid()	returns the unique ID of the entity containing this codelet.
cid()	returns the unique ID of the codelet component.
name()	returns the name of the python codelet.
get_execution_timestamp()	returns the last timestamp in nanoseconds when the codelet was either started, ticked, or stopped.
get_execution_time()	same as get_execution_timestamp() returns a floating points with seconds as unit.
get_delta_time()	returns the time difference between the current call (start(), tick() or stop()) and the last call.
get_execution_count()	returns the number of times the codelet has been executed.
is_first_tick()	returns True if tick() is called for the first time after start().