.. Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved. NVIDIA CORPORATION and its licensors retain all intellectual property and proprietary rights in and to this software, related documentation and any modifications thereto. Any use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from NVIDIA CORPORATION is strictly prohibited. .. _python_codelet: Python Codelets ============================== Python Codelets allow users to build parts of their application in Python. This section gives an overview of the how to implement codelets in python and how to use them in an application. Codelets are user implemented units which can be run as part of the graph. Python codelets allow the users to implement this functionality in python. Like C++ codelets, python codelets have the following functions users can implement: * ``start()`` - called once when the codelet starts * ``tick()`` - called on every tick * ``stop()`` - called once when the codelet is stopped Unlike the C++ codelets, all the python codelets are registered via a same interface namely: ``nvidia::gxf::PyCodeletV0``. To implement a codelet in python, users have to implement the class ``CodeletAdapter``. As explained above, the users implement ``start()``, ``stop()`` and ``tick()`` methods in Python. This class also provides basic functionality to obtain the parameters. Running an application containing a Python Codelet has to be done using ``gxe.py``. Graphs containing Python Codelets cannot run directly using the ``gxe`` binary because a python interpreter has to be started before running any Python Codelet. A graph can also be run if ``graph-composer`` and ``regsitry`` is already installed. Use the following command to install the relevant extensions: ``registry graph insall -g path/to/graph.yaml -m output/path/to/generated/manifest.yaml -d path/to/target.yaml --output-directory output/directory`` A sample target file can be found in ``/opt/nvidia/graph-composer/``. The above command will install the relevant extensions and other required files to ``output/directory``. Once that is done, run the following command: ``cp /opt/nvidia/graph-composer/core.so /opt/nvidia/graph-composer/core.py output/directory/gxf`` Finally add output/directory to PYTHONPATH using the following command: ``export PYTHONPATH=output/directory`` Running a graph using ``gxe.py``: ``python3 output/directory/gxf/std/gxe.py --app path/to/graph.yaml --manifest path/to/manifest/file.yaml`` Following are some of the Python Codelet examples which can be found in ``gxf/python/tests``: Ping Codelets demonstrate the basic usage of python codelets 1. PingTx.py - Constructs an empty message which is an ``Entity`` - Transmits it on the first transmitter 2. PingRx.py - Receives a message on first transmitter - Passes if it's non-null else raises an exception Obtaining tensor data in python 3. VerifyEqual.py - This codelet has two receivers. - It receives two messages one from each receiver. - Extracts tensor data from both the message. - Copies the data on the host if the tensor is on the device. - Asserts that the data on the tensors is equal. Generating tensor data from python 4. StreamGenerator - On every tick, this codelet generates four tensors: two on host and two on device. - It uses TensorDescription object to reshape the tensor to desired shape. - Creates a host message and a device message. - Adds the device tensors to device messages and host tensors to host messages - Publishes the host message on the first transmitter and the device message on the second transmitter. Please refer to the sample graph files present in the ``gxf/python/tests`` directory for examples on how to use the Python Codelets in an application. General Concepts ------------------------------ Python enables users to add a lot of functionality with very less boiler plate code and hence is heavily used in the Machine Learning Community. For e.g. simulating a sensor for data, vizualizing the generated output or running various ML models on the data can be easily implemented in python. Python Codelets allow users to implement these functionalities in python. Unlike C++ codelets, python codelets are not registered individually. To create a python codelet users implement ``CodeletAdapter`` (a python base class) and all python codelets are registered in the registry as ``PyCodeletV0`` which is a classic C++ codelet. This ``C++`` codelet calls the ``start()``, ``tick()`` and ``stop()`` methods of the python codelet implementation. +-------------+-----------------------+ | PyCodeletV0 | User's Python Codelet | +=============+=======================+ | ``start()`` | ``start()`` | +-------------+-----------------------+ | ``tick()`` | ``tick()`` | +-------------+-----------------------+ | ``stop()`` | ``stop()`` | +-------------+-----------------------+ Implenting a Python Codelet ------------------------------------ Implementing Class ~~~~~~~~~~~~~~~~~~~~~~~~ To implement a python codelet, users implement ``CodeletAdapter``. This is available in ``gxf.python_codelet.codelet``. Specifically, users implement the following functions: ``start()`` - Setting up the codelet. Called once when the codelet starts. ``tick()`` - Business logic. Called based on Scheduling Terms. ``stop()`` - Clean up. Called when entity containing the python codelet is terminated. Adding Python Codelet to the Graph ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Adding a python codelet to the graph is different from adding a C++ codelet. Unlike C++ codelets, the ``type`` of python codelet is ``nvidia::gxf::PyCodeletV0`` and not ``nvidia::gxf::Codelet``. Linking the implementation of the codelet is done via params directly. Also, python codelets only support registering only a selected types of parameters: ``nvidia::gxf::PyCodeletV0`` accepts the following parameters: +-----------------------+--------------------+-------------------------------------------------------------------------+ | Parameter | Mandatory/Optional | Description | +=======================+====================+=========================================================================+ | ``codelet_name`` | Mandatory | Name of the user's python codelet class which implements CodeletAdapter | +-----------------------+--------------------+-------------------------------------------------------------------------+ | ``codelet_file`` | Mandatory | Absolute path to the file containing the implementation | +-----------------------+--------------------+-------------------------------------------------------------------------+ | ``receivers`` | Optional | A list of receivers | +-----------------------+--------------------+-------------------------------------------------------------------------+ | ``trasnmitters`` | Optional | A list of transmitters | +-----------------------+--------------------+-------------------------------------------------------------------------+ | ``allocators`` | Optional | A list of allocators | +-----------------------+--------------------+-------------------------------------------------------------------------+ | ``cuda_stream_pools`` | Optional | A list of cuda stream pools | +-----------------------+--------------------+-------------------------------------------------------------------------+ | ``codelet_params`` | Optional | A string which the users can parse for setting additional params | +-----------------------+--------------------+-------------------------------------------------------------------------+ Following is an entity named ``rx`` which contains the following components: - A python codelet called ``python_receiver`` - A ``DoubleBufferReceiver`` called signal - A nameless ``MessageAvailableSchedulingTerm`` The implementation of the python codelet is present in ``some/path/to/PythonCodelets.py`` file under the class called ``PingRx`` which should implement ``CodeletAdapter``. The python codelet also accept two other parameters: - ``receivers``: A list containing single item, ``signal``, which is a double buffer receiver component. - ``codelet_params``: a custom string which the user can parse in the ``start()``, ``tick()`` or ``stop()`` method and use accordingly. .. code-block:: yaml --- name: rx components: - name: signal type: nvidia::gxf::DoubleBufferReceiver - type: nvidia::gxf::MessageAvailableSchedulingTerm parameters: receiver: signal min_size: 1 - name: python_receiver type: nvidia::gxf::PyCodeletV0 parameters: codelet_name: "PingRx" codelet_file: "some/path/to/PingRx.py" receivers: [signal] codelet_params: "log_count=5" Accessing Parameters ~~~~~~~~~~~~~~~~~~~~~~ ``CodeletAdapter`` implements the following methods which let's users access the parameters using ``self.method_name()``: +-----------------------------+-------------------------------------------------------------+ | Method | Description | +=============================+=============================================================+ | ``get_allocators()`` | returns a list of gxf.StandardExtension.Allocator objects | +-----------------------------+-------------------------------------------------------------+ | ``get_cuda_stream_pools()`` | returns a list of gxf.CudaExtension.CudaStreamPool objects | +-----------------------------+-------------------------------------------------------------+ | ``get_transmitters()`` | returns a list of gxf.StandardExtension.Transmitter objects | +-----------------------------+-------------------------------------------------------------+ | ``get_receivers()`` | returns a list of gxf.StandardExtension.Receiver objects | +-----------------------------+-------------------------------------------------------------+ | ``get_clock()`` | returns a gxf.StandardExtension.Clock | +-----------------------------+-------------------------------------------------------------+ | ``get_params()`` | returns a string containing the additional params | +-----------------------------+-------------------------------------------------------------+ ``CodeletAdapter`` also implements the following utility methods: +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | Method | Description | +===============================+=================================================================================================================+ | ``eid()`` | returns the unique ID of the entity containing this codelet. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``cid()`` | returns the unique ID of the codelet component. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``name()`` | returns the name of the python codelet. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``get_execution_timestamp()`` | returns the last timestamp in nanoseconds when the codelet was either started, ticked, or stopped. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``get_execution_time()`` | same as ``get_execution_timestamp()`` returns a floating points with seconds as unit. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``get_delta_time()`` | returns the time difference between the current call (``start()``, ``tick()`` or ``stop()``) and the last call. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``get_execution_count()`` | returns the number of times the codelet has been executed. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+ | ``is_first_tick()`` | returns ``True`` if ``tick()`` is called for the first time after ``start()``. | +-------------------------------+-----------------------------------------------------------------------------------------------------------------+