|
|
NVIDIA DeepStream SDK API Reference
|
9.2 Release
|
Go to the documentation of this file.
24 namespace BatchMetaDoc
26 constexpr
const char*
descr = R
"pydeepstream(
27 Holds information about a formed batch containing frames from different sources.
29 NOTE: Both Video and Audio metadata uses the same :class:`BatchMetadata` type.
31 NOTE: Audio batch metadata is formed within nvinferaudio plugin and will not be corresponding to any one buffer output from nvinferaudio.
32 The batch metadata for audio is attached to the last input buffer when the audio batch buffering reach configurable threshold (audio frame length)
33 and this is when inference output is available.
35 :ivar n_frames: *int* Number of frames present in the current batch.
36 :ivar is_audio: *bool* True when the batch contains :class:`AudioFrameMetadata` entries.
37 :ivar frame_items: *iterator* Iterator over items of type :class:`FrameMetadata` in use in the current batch.
38 :ivar audio_frame_items: *iterator* Iterator over items of type :class:`AudioFrameMetadata` in use in the current audio batch.
39 :ivar preprocess_batch_items: *iterator* Iterator over preprocess batch metadata in use in the current batch as items of type :class:`UserMetadata`
40 Each item *user_meta* can be called as :class:`PreprocessBatchUserMetadata` by calling *user_meta.as_preprocess_batch()*.)pydeepstream";
41 constexpr const char*
acquire_display_meta = R
"pydeepstream(Acquires a default :class:`DisplayMetadata` object from the pool.)pydeepstream";
42 constexpr const char*
acquire_frame_meta = R
"pydeepstream(Acquires a default :class:`FrameMetadata` object from the pool.)pydeepstream";
43 constexpr const char*
acquire_object_meta = R
"pydeepstream(Acquires a default :class:`ObjectMeta` object from the pool.)pydeepstream";
44 constexpr const char*
acquire_event_message_meta = R
"pydeepstream(Acquires a default :class:`EventMessageUserMetadata` object from the pool.)pydeepstream";
45 constexpr const char*
acquire_preprocess_batch_meta = R
"pydeepstream(Acquires a default :class:`PreprocessBatchUserMetadata` object from the pool.)pydeepstream";
46 constexpr const char*
acquire_user_meta = R
"pydeepstream(Acquires a :class:`UserMetadata` object from the batch meta's user meta pool.
48 The acquired user meta can be filled with custom data and then attached to a batch, frame, or object metadata.)pydeepstream";
49 constexpr const char*
acquire_obj_reid_meta = R
"pydeepstream(Acquires an :class:`ObjectReidUserMetadata` object from the batch meta's user meta pool.
51 The acquired ReID metadata can be filled with feature vectors and attached to object metadata
52 for use in re-identification tasks.
54 :returns: :class:`ObjectReidUserMetadata` object ready to be configured.)pydeepstream";
56 Gets an iterator over user metadata items of a specific type attached to the batch.
58 This generic method allows accessing any user metadata type without requiring
59 specific bindings for each type.
61 :arg meta_type: *int*, The NvDsMetaType value (e.g., NVDS_TRACKER_OBJ_REID_META).
62 :returns: Iterator over :class:`UserMetadata` items matching the specified type.)pydeepstream";
63 constexpr const char*
append = R
"pydeepstream(Append a :class:`PreprocessBatchUserMetadata` object to the current batch metadata.)pydeepstream";
64 constexpr const char*
extract = R
"pydeepstream(
65 Extracts metadata from the batch into a list of dictionaries.
67 This method efficiently extracts all metadata (bounding boxes, labels, confidences, segmentation maps, audio labels, etc.)
68 from all frames in the batch. The extraction, data copying, and type conversions are performed in C++ for optimal performance.
70 :arg max_items: *int* - Maximum number of items in the returned list. The list will have this capacity,
71 indexed by frame *pad_index*.
73 :returns: *list of dict* - A list with size equal to *max_items*. Each element corresponds to a frame
74 indexed by *pad_index*. Elements without data will be *None*.
76 Each dictionary contains:
77 - **shape**: *list of int* - Frame dimensions [height, width]
78 - **bboxes**: *list of list of int* - Bounding boxes in [left, top, right, bottom] format
79 - **probs**: *list of float* - Confidence scores for each detected object
80 - **labels**: *list of list of str* - Labels for each object (including classifier outputs)
81 - **objects**: *list of int* - Object IDs for tracking
82 - **seg_maps**: *list of numpy.ndarray* - Segmentation maps (instance masks or semantic segmentation) as int arrays
83 - **timestamp**: *int* - Frame presentation timestamp (PTS)
85 Audio dictionaries also include **media_type**, **sample_rate**, **num_channels**,
86 **num_samples_per_frame**, **format**, **layout**, **infer_done**, **class_id**,
87 **class_label**, and **confidence**.
91 namespace FrameMetaDoc
93 constexpr
const char*
descr = R
"pydeepstream(
94 Holds metadata for a frame in a batch.
96 :ivar frame_number: *int*, Current frame number of the source.
97 :ivar pad_index: *int*, Pad or port index of stream muxer component for the frame in the batch.
98 :ivar batch_id: *int*, Location of the frame in the batch.
99 :ivar source_id: *int*, ID of the source to which the frame belongs, generated through source config
100 :ivar source_width: *int*, Original width of the frame at input to Gst-streammux.
101 :ivar source_height: *int*, Original height of the frame at input to Gst-streammux.
102 :ivar pipeline_width: *int*, Width of the frame after batching.
103 :ivar pipeline_height: *int*, Height of the frame after batching.
104 :ivar buffer_pts: *int*, Presentation timestamp (PTS) of the frame.
105 :ivar ntp_timestamp: *int*, Network Time Protocol (NTP) timestamp.
106 :ivar object_items: *iterator* Iterator over items of type :class:`ObjectMetadata` in use in the current frame.
107 :ivar tensor_items: *iterator* Iterator over items of type :class:`UserMetadata` in use in the current frame as items of type :class:`TensorOutputUserMetadata`
108 Each item *user_meta* can be called as :class:`TensorOutputUserMetadata` by calling *user_meta.as_tensor_output()*.
109 :ivar segmentation_items: *iterator* Iterator over items of type :class:`UserMetadata` in use in the current frame as items of type :class:`SegmentationUserMetadata`
110 Each item *user_meta* can be called as :class:`SegmentationUserMetadata` by calling *user_meta.as_segmentation()*.)pydeepstream";
112 constexpr const char*
append = R
"pydeepstream(Append a :class:`DisplayMetadata`, :class:`EventMessageUserMetadata`, :class:`ObjectMetadata` or :class:`TensorOutputUserMetadata` object to the current frame metadata.)pydeepstream";
114 Gets an iterator over user metadata items of a specific type attached to the frame.
116 This generic method allows accessing any user metadata type without requiring
117 specific bindings for each type.
119 :arg meta_type: *int*, The NvDsMetaType value (e.g., NVDS_TRACKER_OBJ_REID_META).
120 :returns: Iterator over :class:`UserMetadata` items matching the specified type.)pydeepstream";
123 namespace AudioFrameMetaDoc
125 constexpr
const char*
descr = R
"pydeepstream(
126 Holds metadata for an audio frame in a batch.
128 :ivar frame_number: *int*, Current frame number of the source.
129 :ivar pad_index: *int*, Pad or port index of stream muxer component for the audio frame in the batch.
130 :ivar batch_id: *int*, Location of the audio frame in the batch.
131 :ivar source_id: *int*, ID of the source to which the audio frame belongs.
132 :ivar sample_rate: *int*, Audio sample rate.
133 :ivar num_channels: *int*, Number of audio channels.
134 :ivar num_samples_per_frame: *int*, Number of audio samples in this frame.
135 :ivar format: *int*, Audio sample format.
136 :ivar layout: *int*, Audio channel layout.
137 :ivar buffer_pts: *int*, Presentation timestamp (PTS) of the audio frame.
138 :ivar ntp_timestamp: *int*, Network Time Protocol (NTP) timestamp.
139 :ivar infer_done: *bool*, Whether inference has been performed on this audio frame.
140 :ivar class_id: *int*, Class ID of the last classified event.
141 :ivar confidence: *float*, Confidence of the last classified event.
142 :ivar class_label: *str*, Label of the last classified event.
143 :ivar classifier_items: *iterator* Iterator over :class:`ClassifierMetadata` items attached by nvinferaudio.
144 :ivar tensor_items: *iterator* Iterator over tensor output user metadata items attached to the audio frame.)pydeepstream";
145 constexpr const char*
append = R
"pydeepstream(Append a :class:`UserMetadata` object to the current audio frame metadata.)pydeepstream";
147 Gets an iterator over user metadata items of a specific type attached to the audio frame.
149 :arg meta_type: *int*, The NvDsMetaType value.
150 :returns: Iterator over :class:`UserMetadata` items matching the specified type.)pydeepstream";
153 namespace ObjectMetaDoc
155 constexpr
const char*
descr = R
"pydeepstream(
156 Holds information of object metadata in the frame.
158 :ivar class_id: *int*, Index of the object class infered by the primary detector/classifier
159 :ivar object_id: *int*, Unique ID for tracking the object.
160 :ivar unique_component_id: *int*, Unique id of the component that generates the object metadata.
161 :ivar confidence: *float*, Holds a confidence value for the object, set by the inference component.
162 Confidence will be set to -0.1, if "Group Rectangles" mode of clustering is chosen since the algorithm does not preserve confidence values.
163 Also, for objects found by tracker and not inference component, confidence will be set to -0.1
164 :ivar tracker_confidence: *float*, Holds a confidence value for the object, set by the tracker component.
165 :ivar rect_params: :class:`NvOSD_RectParams`, Structure containing the positional parameters of the object in the frame.
166 Can also be used to overlay borders / semi-transparent boxes on top of objects. See :class:`NvOSD_RectParams`
167 :ivar mask_params: :class:`NvOSD_MaskParams`, Mask parameters for the object. This mask is overlaid over the object See :class:`NvOSD_MaskParams`
168 :ivar text_params: :class:`NvOSD_TextParams`, Text for describing the object. See :class:`NvOSD_TextParams`
169 :ivar label: *str*, Label describing the class of the detected object.
170 :ivar classifier_items: *iterator* Iterator over items of type :class:`ClassifierMetadata` in use in the current object.
171 :ivar tensor_items: *iterator* Iterator over items of type :class:`UserMetadata` in use in the current object as items of type :class:`TensorOutputUserMetadata`
172 Each item *user_meta* can be called as :class:`TensorOutputUserMetadata` by calling *user_meta.as_tensor_output()*.
173 :ivar obj_reid_items: *iterator* Iterator over ReID user metadata items of type :class:`UserMetadata`.
174 Each item *user_meta* can be cast to :class:`ObjectReidUserMetadata` by calling *user_meta.as_obj_reid()*.)pydeepstream";
176 constexpr const char*
append = R
"pydeepstream(Append a :class:`UserMetadata` object to the current object metadata.)pydeepstream";
178 Gets an iterator over user metadata items of a specific type attached to the object.
180 This generic method allows accessing any user metadata type without requiring
181 specific bindings for each type.
183 :arg meta_type: *int*, The NvDsMetaType value (e.g., NVDS_TRACKER_OBJ_REID_META).
184 :returns: Iterator over :class:`UserMetadata` items matching the specified type.)pydeepstream";
187 namespace LabelInfoDoc
189 constexpr
const char*
descr = R
"pydeepstream(
190 Holds all fields of a single label entry produced by a classifier.
192 :ivar label: *str*, Label string of the result.
193 :ivar class_id: *int*, Class ID of the best result.
194 :ivar prob: *float*, Probability of the best result.
195 :ivar label_id: *int*, Label ID for multi-label classifiers.
196 :ivar num_classes: *int*, Number of classes for this label.)pydeepstream";
199 namespace ClassifierMetaDoc
201 constexpr
const char*
descr = R
"pydeepstream(
202 Holds classifier metadata for an object.
204 :ivar n_labels: *int*, Number of output labels of the classifier.
205 :ivar unique_component_id: *int*, Unique id of the component that generates the classifier metadata.
206 :ivar classifier_type: *str*, Type of the classifier. Empty string if not set.)pydeepstream";
208 constexpr const char*
classifier_type = R
"pydeepstream(Type of the classifier. Returns an empty string if the classifier type was not set.)pydeepstream";
209 constexpr const char*
get_label_info = R
"pydeepstream(Get all fields of the nth label as a LabelInfo object.)pydeepstream";
210 constexpr const char*
get_label = R
"pydeepstream(Get the label string of the nth label.)pydeepstream";
213 namespace DisplayMetaDoc
215 constexpr
const char*
descr = R
"pydeepstream(
216 Holds information of display metadata that user can specify in the frame.
218 :ivar n_rects: *int*, Number of rectangles present in display meta.
219 :ivar n_labels: *int*, Number of labels/strings present in display meta.
220 :ivar n_lines: *int*, Number of lines present in display meta.
221 :ivar n_arrows: *int*, Number of arrows present in display meta.
222 :ivar n_circles: *int*, Number of circles present in display meta.)pydeepstream";
223 constexpr const char*
add_text = R
"pydeepstream(Adds a :class:`Text` object to the display meta.)pydeepstream";
224 constexpr const char*
add_rect = R
"pydeepstream(Adds a :class:`NvOSD_RectParams` object to the display meta.)pydeepstream";
225 constexpr const char*
add_line = R
"pydeepstream(Adds a :class:`NvOSD_LineParams` object to the display meta.)pydeepstream";
226 constexpr const char*
add_arrow = R
"pydeepstream(Adds a :class:`NvOSD_ArrowParams` object to the display meta.)pydeepstream";
227 constexpr const char*
add_circle = R
"pydeepstream(Adds a :class:`NvOSD_CircleParams` object to the display meta.)pydeepstream";
230 namespace UserMetaDoc
232 constexpr
const char*
descr = R
"pydeepstream(
233 Holds the user metadata. Can be casted to :class:`TensorOutputUserMetadata`, :class:`SegmentationUserMetadata`, or :class:`PreprocessBatchUserMetadata`.
235 :ivar meta_type: *int*, The metadata type identifier for this user metadata.)pydeepstream";
237 Sets a Python object as JSON-serialized user data for this metadata.
239 The Python object is serialized to a JSON string and stored as a C string (char*).
240 This allows C/C++ components to access the data by reading the string directly
241 and parsing it with any JSON library.
243 :arg data: Any JSON-serializable Python object (dict, list, str, int, float, bool, None).
244 :arg meta_type: *int*, The metadata type identifier (e.g., NVDS_USER_META starting from 4096+4096+1).
246 Example C/C++ access:
247 const char* json_str = (const char*)user_meta->user_meta_data;
248 // Parse with nlohmann/json, rapidjson, etc.)pydeepstream";
250 Gets the JSON-serialized user data as a Python object.
252 Deserializes the JSON string stored in user_meta_data back to a Python object.
254 :returns: The deserialized Python object, or None if no data is set.)pydeepstream";
255 constexpr const char*
as_tensor_output = R
"pydeepstream(Casts the user metadata to :class:`TensorOutputUserMetadata` object.)pydeepstream";
256 constexpr const char*
as_segmentation = R
"pydeepstream(Casts the user metadata to :class:`SegmentationUserMetadata` object.)pydeepstream";
257 constexpr const char*
as_preprocess_batch = R
"pydeepstream(Casts the user metadata to :class:`PreprocessBatchUserMetadata` object.)pydeepstream";
258 constexpr const char*
as_nvdsanalytics_obj = R
"pydeepstream(Casts the user metadata to :class:`NvDsAnalyticsObjInfo` object.)pydeepstream";
259 constexpr const char*
as_nvdsanalytics_frame = R
"pydeepstream(Casts the user metadata to :class:`NvDsAnalyticsFrameMeta` object.)pydeepstream";
260 constexpr const char*
as_structure_str = R
"pydeepstream(Returns the string representation of the underlying GstStructure of the Structure user metadata.)pydeepstream";
261 constexpr const char*
as_obj_reid = R
"pydeepstream(Casts the user metadata to :class:`ObjectReidUserMetadata` object for ReID features.)pydeepstream";
264 namespace ObjectReidUserMetaDoc
266 constexpr
const char*
descr = R
"pydeepstream(
267 User metadata for object Re-Identification (ReID) features.
269 This metadata type (NVDS_TRACKER_OBJ_REID_META) holds ReID feature vectors
270 that can be used to identify/re-identify objects across frames.
272 The structure is interoperable with C/C++ components using NvDsObjReid.
274 :ivar feature_size: *int*, Length of the ReID feature vector.
275 :ivar feature_vector: *numpy.ndarray*, The ReID feature vector as float array.)pydeepstream";
277 Sets the ReID feature vector.
279 :arg features: numpy array of float32 values representing the ReID embedding.)pydeepstream";
282 namespace EventMessageUserMetaDoc
284 constexpr
const char*
descr = R
"pydeepstream(
285 User metadata for event messages.)pydeepstream";
287 Fills the event message user metadata with the given :class:`ObjectMetadata`, :class:`FrameMetadata`, sensor, uri, and labels.
289 :arg object_meta: :class:`ObjectMetadata` to be used to fill the event message user metadata.
290 :arg frame_meta: :class:`FrameMetadata` to be used to fill the event message user metadata.
291 :arg sensor: *str*, optional, Sensor name.
292 :arg uri: *str*, optional, URI of the source, if applicable.
293 :arg labels: *list of str*, optional, List of labels for custom model.)pydeepstream";
295 Fills the event message user metadata from :class:`AudioFrameMetadata`.
297 This copies the audio classification label, class id, confidence, source id,
298 and timestamp into :class:`NvDsEventMsgMeta` so it can be consumed by the
299 DeepStream audio message converter.
301 :arg audio_frame_meta: :class:`AudioFrameMetadata` to be used to fill the event message user metadata.
302 :arg sensor: *str*, optional, Sensor name.
303 :arg uri: *str*, optional, URI of the source, if applicable.)pydeepstream";
306 namespace TensorOutputUserMetaDoc
308 constexpr
const char*
descr = R
"pydeepstream(
309 User metadata for tensor output.
311 :ivar unique_id: *int*, Unique id of the component that generates tensor output.)pydeepstream";
312 constexpr const char*
get_layers = R
"pydeepstream(Returns a map of tensors where key is the layer name and value is tensor output.)pydeepstream";
315 namespace SegmentationUserMetaDoc
317 constexpr
const char*
descr = R
"pydeepstream(
318 User metadata for segmentation output.
320 :ivar unique_id: *int*, Unique id of the component that generates segmentation output.
321 :ivar classes: *int*, Number of classes in the segmentation output.
322 :ivar width: *int*, Width of the segmentation mask.
323 :ivar height: *int*, Height of the segmentation mask.
324 :ivar class_map: *list of int* Class map of the segmentation output.
325 :ivar class_probabilities_map: *list of float* Class probabilities map of the segmentation output.)pydeepstream";
330 constexpr
const char*
descr = R
"pydeepstream(
331 Holds information of a region of interest.
333 :ivar roi: :class:`NvOSD_RectParams`, Region of interest bounding box.
334 :ivar frame_meta: :class:`FrameMetadata`, Frame metadata which contains the region of interest.
335 :ivar tensor_items: *iterator* Iterator over items of type :class:`UserMetadata` in use in the current frame as items of type :class:`TensorOutputUserMetadata`
336 Each item *user_meta* can be called as :class:`TensorOutputUserMetadata` by calling *user_meta.as_tensor_output()*.
337 :ivar classifier_items: *iterator* Iterator over items of type :class:`ClassifierMetadata` in use in the current object.
338 Each item *classifier_meta* can be called as :class:`ClassifierMetadata` by calling *classifier_meta.as_classifier()*.
339 :ivar segmentation_items: *iterator* Iterator over items of type :class:`UserMetadata` in use in the current object as items of type :class:`SegmentationUserMetadata`
340 Each item *user_meta* can be called as :class:`SegmentationUserMetadata` by calling *user_meta.as_segmentation()*.
341 :ivar object_meta: :class:`ObjectMetadata`, Object metadata of the region of interest.)pydeepstream";
344 namespace PreprocessTensorMetaDoc
346 constexpr
const char*
descr = R
"pydeepstream(
347 Tensor meta containing prepared tensor and related info inside preprocess user meta which is attached at batch level.
349 :ivar name: *str*, Name of the preprocess tensor.
350 :ivar meta_id: *int*, Meta id of the preprocess tensor.
351 :ivar tensor: :class:`Tensor`, Tensor object.)pydeepstream";
354 namespace PreprocessBatchUserMetaDoc
356 constexpr
const char*
descr = R
"pydeepstream(
357 Batch metadata structure for preprocessing operations.
359 :ivar rois: *list of :class:`RoiMetadata`*, List of roi metadata.
360 :ivar preprocess_tensor_meta: :class:`PreprocessTensorMetadata`, Preprocess tensor metadata.)pydeepstream";
363 Initialize the preprocess tensor meta from given name, meta_id, unique_id, and tensor.
365 :arg name: *str*, Name of the preprocess tensor.
366 :arg meta_id: *int*, Meta id of the preprocess tensor.
367 :arg unique_id: *int*, Unique target ID.
368 :arg tensor: :class:`Tensor`, Tensor object.
370 :returns: True if the preprocess tensor meta is initialized successfully, False otherwise.)pydeepstream";
371 constexpr const char*
extract = R
"pydeepstream(
372 Extracts metadata from preprocess batch ROIs into a list of dictionaries.
374 This method efficiently extracts metadata from all ROIs in the preprocess batch, including segmentation maps,
375 object detections, and classifier outputs. The extraction and data copying are performed in C++ for optimal performance.
377 :arg max_items: *int* - Maximum number of items in the returned list. The list will have this capacity,
378 indexed by frame *pad_index*.
380 :returns: *list of dict* - A list with size equal to *max_items*. Each element corresponds to the
381 metadata extracted from ROIs belonging to that frame. Elements without data will be *None*.
383 Each dictionary contains:
384 - **shape**: *list of int* - Frame dimensions [height, width]
385 - **bboxes**: *list of list of int* - Bounding boxes in [left, top, right, bottom] format
386 - **probs**: *list of float* - Confidence scores for each detected object
387 - **labels**: *list of list of str* - Labels from objects and ROI-level classifiers
388 - **objects**: *list of int* - Object IDs for tracking
389 - **seg_maps**: *list of numpy.ndarray* - Segmentation maps as int arrays
390 - **timestamp**: *int* - Frame presentation timestamp (PTS)
394 namespace AnalyticsObjInfoDoc
396 constexpr
const char*
descr = R
"pydeepstream(
397 Holds a set of nvdsanalytics object level metadata.
399 :ivar roi_status: *list of str*, Holds the array of ROI labels in which object is present.
400 :ivar oc_status: *list of str*, Holds the array of OverCrowding labels in which object is present.
401 :ivar lc_status: *list of str*, Holds the array of line crossing labels which object has crossed.
402 :ivar dir_status: *str*, Holds the direction string for the tracked object.
403 :ivar unique_id: *int*, Holds unique identifier for nvdsanalytics instance.
404 :ivar obj_status: *str*, Holds the status string for the tracked object.
408 namespace AnalyticsFrameMetaDoc
410 constexpr
const char*
descr = R
"pydeepstream(
411 Holds a set of nvdsanalytics framelevel metadata.
413 :ivar oc_status: *dict of str to bool*, Holds a map of boolean status of overcrowding for configured ROIs, which can be accessed using key, value pair; where key is the ROI label.
414 :ivar obj_in_roi_cnt: *dict of str to int*, Holds a map of total count of valid objects in ROI for configured ROIs, which can be accessed using key, value pair; where key is the ROI label.
415 :ivar obj_lc_curr_cnt: *dict of str to int*, Holds a map of total count of Line crossing in current frame for configured lines, which can be accessed using key, value pair; where key is the ROI label.
416 :ivar obj_lc_cum_cnt: *dict of str to int*, Holds a map of total cumulative count of Line crossing for configured lines, which can be accessed using key, value pair; where key is the ROI label.
417 :ivar unique_id: *int*, Holds unique identifier for nvdsanalytics instance.
418 :ivar obj_cnt: *dict of int to int*, Holds a map of total count of objects for each class ID, which can be accessed using key, value pair; where key is class ID.