.. include:: Python Sample Apps and Bindings Source Details =================================== Sample Application Source Details ---------------------------------- The following table shows the location of the Python sample applications under https://github.com/NVIDIA-AI-IOT/deepstream_python_apps .. csv-table:: Python sample application source details :file: ../text/tables/DS_python_sample_app_details.csv :widths: 20, 20, 45 :header-rows: 1 Python Bindings and Application Development ----------------- This section provides details about DeepStream application development in Python. Python bindings are available here: https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/bindings . Read more about Pyds API `here `_. Prerequisites ~~~~~~~~~~~~~~ * Ubuntu 18.04 * DeepStream SDK 6.0.1 or later * Python 3.6 * Gst Python v1.14.5 If Gst python installation is missing on Jetson, install using the following commands: :: $ sudo apt-get install python-gi-dev $ export GST_LIBS="-lgstreamer-1.0 -lgobject-2.0 -lglib-2.0" $ export GST_CFLAGS="-pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include" $ git clone https://github.com/GStreamer/gst-python.git $ cd gst-python $ git checkout 1a8f48a $ ./autogen.sh PYTHON=python3 $ ./configure PYTHON=python3 $ make $ sudo make install Running Sample Applications ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Clone the ``deepstream_python_apps`` repo under ``/sources``: :: git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps 2. This will create the following directory: :: /sources/deepstream_python_apps 3. The Python apps are under the ``apps`` directory. Go into each app directory and follow instructions in the README. .. note:: The app configuration files contain relative paths for models. Pipeline Construction ~~~~~~~~~~~~~~~~~~~~~~~~ DeepStream pipelines can be constructed using Gst Python, the GStreamer framework's Python bindings. See sample applications main functions for pipeline construction examples. MetaData Access ------------------ DeepStream MetaData contains inference results and other information used in analytics. The MetaData is attached to the ``Gst Buffer`` received by each pipeline component. The metadata format is described in detail in the SDK MetaData documentation and API Guide. The SDK MetaData library is developed in C/C++. Python bindings provide access to the MetaData from Python applications. Please find Python bindings source and packages at https://github.com/NVIDIA-AI-IOT/deepstream_python_apps. Memory Management ~~~~~~~~~~~~~~~~~~ Memory for MetaData is shared by the Python and C/C++ code paths. For example, a MetaData item may be added by a probe function written in Python and needs to be accessed by a downstream plugin written in C/C++. The deepstream-test4 app contains such usage. The Python garbage collector does not have visibility into memory references in C/C++, and therefore cannot safely manage the lifetime of such shared memory. Because of this complication, Python access to MetaData memory is typically achieved via references without claiming ownership. Allocations ~~~~~~~~~~~~ When MetaData objects are allocated in Python, an allocation function is provided by the bindings to ensure proper memory ownership of the object. If the constructor is used, the the object will be claimed by the garbage collector when its Python references terminate. However, the object will still need to be accessed by C/C++ code downstream, and therefore must persist beyond those Python references. Example: To allocate an ``NvDsEventMsgMeta`` instance, use this: :: msg_meta = pyds.alloc_nvds_event_msg_meta() *# get reference to allocated instance without claiming memory ownership* NOT this: :: msg_meta = NvDsEventMsgMeta() *# memory will be freed by the garbage collector when msg_meta goes out of scope in Python* Allocators are available for the following structs: * ``NvDsVehicleObject: alloc_nvds_vehicle_object()`` * ``NvDsPersonObject: alloc_nvds_person_object()`` * ``NvDsFaceObject: alloc_nvds_face_object()`` * ``NvDsEventMsgMeta: alloc_nvds_event_msg_meta()`` * ``NvDsEvent: alloc_nvds_event()`` * ``NvDsPayload: alloc_nvds_payload()`` * ``Generic buffer: alloc_buffer(size)`` String Access ~~~~~~~~~~~~~~~ Some MetaData structures contain string fields. Sections below provide details on accessing them. Setting String Fields ^^^^^^^^^^^^^^^^^^^^^^^ Setting a string field results in the allocation of a string buffer in the underlying C++ code. :: obj.type = "Type" This will cause a memory buffer to be allocated, and the string "TYPE" will be copied into it. This memory is owned by the C code and will be freed later. To free the buffer in Python code, use: :: pyds.free_buffer(obj.type) .. note:: ``NvOSD_TextParams.display_text`` string now gets freed automatically when a new string is assigned. Reading String Fields ^^^^^^^^^^^^^^^^^^^^^^^ Directly reading a string field returns C address of the field in the form of an int, for example: :: obj = pyds.NvDsVehicleObject.cast(data); print(obj.type) This will print an int representing the address of ``obj.type`` in C (which is a char*). To retrieve the string value of this field, use ``pyds.get_string()``, for example: :: print(pyds.get_string(obj.type)) Casting ~~~~~~~~~ Some MetaData instances are stored in GList form. To access the data in a GList node, the data field needs to be cast to the appropriate structure. This casting is done via cast() member function for the target type: :: NvDsBatchMeta.cast NvDsFrameMeta.cast NvDsObjectMeta.cast NvDsUserMeta.cast NvDsClassifierMeta.cast NvDsDisplayMeta.cast NvDsLabelInfo.cast NvDsEventMsgMeta.cast NvDsVehicleObject.cast NvDsPersonObject.cast In version v0.5, standalone cast functions were provided. Those are now deprecated and superseded by the cast() functions above: :: glist_get_nvds_batch_meta glist_get_nvds_frame_meta glist_get_nvds_object_meta glist_get_nvds_user_meta glist_get_nvds_classifier_meta glist_get_nvds_display_meta glist_get_nvds_label_info glist_get_nvds_event_msg_meta glist_get_nvds_vehicle_object glist_get_nvds_person_object Example: :: l_frame = batch_meta.frame_meta_list frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data) Callback Function Registration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Custom MetaData added to NvDsUserMeta require custom copy and release functions. The MetaData library relies on these custom functions to perform deep-copy of the custom structure, and free allocated resources. These functions are registered as callback function pointers in the NvDsUserMeta structure. Callback functions are registered using these functions: :: pyds.set_user_copyfunc(NvDsUserMeta_instance, copy_function) pyds.set_user_releasefunc(NvDsUserMeta_instance, free_func) .. note:: Callbacks need to be unregistered with the bindings library before the application exits. The bindings library currently keeps global references to the registered functions, and these cannot last beyond bindings library unload which happens at application exit. Use the following function to unregister all callbacks: :: pyds.unset_callback_funcs() See the deepstream-test4 sample application for an example of callback registration and deregistration. **Limitation**: The bindings library currently only supports a single set of callback functions for each application. The last registered function will be used. Optimizations and Utilities ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Python interpretation is generally slower than running compiled C/C++ code. To provide better performance, some operations are implemented in C and exposed via the bindings interface. This is currently experimental and will expand over time. The following optimized functions are available: * ``pyds.NvOSD_ColorParams.set(double red, double green, double blue, double alpha)`` This is a simple function that performs the same operations as the following: :: txt_params.text_bg_clr.red = red txt_params.text_bg_clr.green = green txt_params.text_bg_clr.blue = blue txt_params.text_bg_clr.alpha = alpha These are performed on each object in deepstream_test_4.py, causing the aggregate processing time to slow down the pipeline. Pushing this function into the C layer helps to increase performance. * ``generate_ts_rfc3339 (buffer, buffer_size)`` This function populates the input buffer with a timestamp generated according to RFC3339: ``%Y-%m-%dT%H:%M:%S.nnnZ\0`` Image Data Access ~~~~~~~~~~~~~~~~~~ Decoded images are accessible as ``NumPy`` arrays via the ``get_nvds_buf_surface`` function. This function is documented in the API Guide. See the ``deepstream-imagedata-multistream`` sample application for an example of image data usage.