NVIDIA DeepStream SDK API Reference

9.2 Release
PoseInference Class Reference

Detailed Description

Runs pose inference for tracked target regions.

Definition at line 86 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

Inheritance diagram for PoseInference:
Collaboration diagram for PoseInference:

Public Member Functions

 PoseInference (const std::string &configFilePath)
 Construct a PoseInference instance from configuration. More...
 
 PoseInference (const std::string &configFilePath, const std::string &modelName)
 Construct a PoseInference instance from configuration. More...
 
virtual ~PoseInference ()
 Release resources owned by the PoseInference instance. More...
 
void initializeRuntime (const SourceFrameInfo &frameInfo, const uint maxTargetsPerStream, const std::shared_ptr< NvTrackedObjectManager > &targetManager) override
 Initialize variables and allocate memories. More...
 
void initializeModel () override
 Create TensorRT inference engine; if engine does not exist, serialize from model file. More...
 
void queryParams (NvMOTQuery &paramsQuery)
 Populate the capabilities reported to the tracker. More...
 
void doInference (const std::map< NvMOTStreamId, NvMOTFrame * > &inputMeta, std::map< NvMOTStreamId, std::deque< DetectedObjInfo >> &detectedObjects, const bool isForReAssoc) override
 Run pose inference on cropped targets from the input frames. More...
 
void update () override
 Update the pose estimator after data association. More...
 
void removeStream (const NvMOTStreamId videoStreamId) override
 Remove a stream from the pose estimator. More...
 
const POSEESTIMATOR_TYPES getPoseEstimatorType () override
 Get the configured pose-estimator type. More...
 
 PoseInference (const std::string &configFilePath)
 Construct a PoseInference instance from configuration. More...
 
 PoseInference (const std::string &configFilePath, const std::string &modelName)
 Construct a PoseInference instance from configuration. More...
 
virtual ~PoseInference ()
 Release resources owned by the PoseInference instance. More...
 
void initializeRuntime (const SourceFrameInfo &frameInfo, const uint maxTargetsPerStream, const std::shared_ptr< NvTrackedObjectManager > &targetManager) override
 Initialize variables and allocate memories. More...
 
void initializeModel () override
 Create TensorRT inference engine; if engine does not exist, serialize from model file. More...
 
void queryParams (NvMOTQuery &paramsQuery)
 Populate the capabilities reported to the tracker. More...
 
void doInference (const std::map< NvMOTStreamId, NvMOTFrame * > &inputMeta, std::map< NvMOTStreamId, std::deque< DetectedObjInfo >> &detectedObjects, const bool isForReAssoc) override
 Run pose inference on cropped targets from the input frames. More...
 
void update () override
 Update the pose estimator after data association. More...
 
void removeStream (const NvMOTStreamId videoStreamId) override
 Remove a stream from the pose estimator. More...
 
const POSEESTIMATOR_TYPES getPoseEstimatorType () override
 Get the configured pose-estimator type. More...
 

Protected Member Functions

void extractImagePatchesBatch (const std::map< NvMOTStreamId, NvMOTFrame * > &inputMeta)
 Rescale all tracker frames into encoder input. More...
 
void setValues4PoseGeometryInput ()
 Upload fixed geometry tensors used by pose inference. More...
 
void extractImagePatchesBatch (const std::map< NvMOTStreamId, NvMOTFrame * > &inputMeta)
 Rescale all tracker frames into encoder input. More...
 
void setValues4PoseGeometryInput ()
 Upload fixed geometry tensors used by pose inference. More...
 

Protected Attributes

bool m_isRuntimeInit
 Reid status if runtime variables initialized. More...
 
int m_maxPointNum
 Max number of points. More...
 
int m_numBboxes
 Number of detected boxes in the current batch tracker recieved. More...
 
float * m_inputBuf_d
 
float * m_inputBuf_h
 Input patches to encoder. More...
 
float * m_points_d
 Input a list of points (prompt) to decoder. More...
 
float * m_pointLabels_d
 Input a list of point types (single point, bounding box corners) to decoder. More...
 
float * m_inputMask_d
 Mask prompt, currently unused and set as 0. More...
 
float * m_inputMaskLabel_d
 Mask label, currently unused and set as 0. More...
 
float * m_targetMaskDecSize_d
 
float * m_targetMaskDecSize_h
 Mask output for each target. More...
 
float * m_batchMaskDecSize_d
 Mask output in a batch. More...
 
float * m_targetMaskOriginalSrcSize_d
 Mask for each target resized to original streammux size. More...
 
float * m_kInv_d
 
float * m_tFormInv_d
 
float * m_scaleNormalizedMeanLimbLengths_d
 
float * m_meanLimbLengths_d
 Device inverse intrinsics, inverse transforms, and limb-length priors. More...
 
float * m_pose2d_d
 
float * m_pose2d_org_img_d
 
float * m_pose25d_d
 
float * m_pose3d_d
 Device 2D, original-image 2D, 2.5D, and 3D poses. More...
 
float * m_pose25d_h
 
float * m_pose3d_h
 Host 2.5D and 3D poses. More...
 
float * m_outputIou_d
 Device output IoU. More...
 
PoseInferenceConfigParams m_params
 Config params for reid module. More...
 
NvMotDataTypes::Size m_srcImageSize
 Size of the source image. More...
 
NvMotDataTypes::Size m_originalSrcImageSize
 Size of the original source image. More...
 
std::vector< std::tuple< uint, NvMOTStreamId, NvMotDataTypes::Rect2f > > m_targetStreamIdList
 Target index, stream ID, and crop ROI for each box. More...
 
cudaStream_t m_cuStream
 Cuda stream to run all GPU tasks. More...
 
std::vector< void * > m_bindings
 Pointers to TensorRT tensors. More...
 
float * m_featuresEncoder_d
 Encoder output features. More...
 
std::shared_ptr< NvTrackedObjectManager > m_targetManager
 Target manager providing target status to update feature gallery. More...
 
std::shared_ptr< ReIDFrameTransformTexture > m_subImageCropper2
 Image cropper, may share with visual tracker in the future. More...
 
std::shared_ptr< ReIDCropScaler > m_subImageCropScaler
 Image crop scaler. More...
 
std::map< NvMOTStreamId, std::vector< std::pair< uint, NvMotDataTypes::Rect2f > > > m_inputRoisInBatch
 Reid input rois to crop. More...
 
TrtModule m_inference
 Inference module. More...
 
int m_maxTargetsToTrack
 Max number of targets to keep for all streams. More...
 

Constructor & Destructor Documentation

◆ PoseInference() [1/4]

PoseInference::PoseInference ( const std::string &  configFilePath)

Construct a PoseInference instance from configuration.

Parameters
[in]configFilePathPath to the tracker configuration file.

◆ PoseInference() [2/4]

PoseInference::PoseInference ( const std::string &  configFilePath,
const std::string &  modelName 
)

Construct a PoseInference instance from configuration.

Parameters
[in]configFilePathPath to the tracker configuration file.
[in]modelNameConfigured model name.

◆ ~PoseInference() [1/2]

virtual PoseInference::~PoseInference ( )
virtual

Release resources owned by the PoseInference instance.

◆ PoseInference() [3/4]

PoseInference::PoseInference ( const std::string &  configFilePath)

Construct a PoseInference instance from configuration.

Parameters
[in]configFilePathPath to the tracker configuration file.

◆ PoseInference() [4/4]

PoseInference::PoseInference ( const std::string &  configFilePath,
const std::string &  modelName 
)

Construct a PoseInference instance from configuration.

Parameters
[in]configFilePathPath to the tracker configuration file.
[in]modelNameConfigured model name.

◆ ~PoseInference() [2/2]

virtual PoseInference::~PoseInference ( )
virtual

Release resources owned by the PoseInference instance.

Member Function Documentation

◆ doInference() [1/2]

void PoseInference::doInference ( const std::map< NvMOTStreamId, NvMOTFrame * > &  inputMeta,
std::map< NvMOTStreamId, std::deque< DetectedObjInfo >> &  detectedObjects,
const bool  isForReAssoc 
)
overridevirtual

Run pose inference on cropped targets from the input frames.

Parameters
[in]inputMetaA batch of input frames.
[in,out]detectedObjectsNewly detected objects in each stream.
[in]isForReAssocWhether this inference is for reassociation.

Implements IPoseEstimator.

◆ doInference() [2/2]

void PoseInference::doInference ( const std::map< NvMOTStreamId, NvMOTFrame * > &  inputMeta,
std::map< NvMOTStreamId, std::deque< DetectedObjInfo >> &  detectedObjects,
const bool  isForReAssoc 
)
overridevirtual

Run pose inference on cropped targets from the input frames.

Parameters
[in]inputMetaA batch of input frames.
[in,out]detectedObjectsNewly detected objects in each stream.
[in]isForReAssocWhether this inference is for reassociation.

Implements IPoseEstimator.

◆ extractImagePatchesBatch() [1/2]

void PoseInference::extractImagePatchesBatch ( const std::map< NvMOTStreamId, NvMOTFrame * > &  inputMeta)
protected

Rescale all tracker frames into encoder input.

Parameters
[in]inputMetaInput frames keyed by stream ID.

◆ extractImagePatchesBatch() [2/2]

void PoseInference::extractImagePatchesBatch ( const std::map< NvMOTStreamId, NvMOTFrame * > &  inputMeta)
protected

Rescale all tracker frames into encoder input.

Parameters
[in]inputMetaInput frames keyed by stream ID.

◆ getPoseEstimatorType() [1/2]

const POSEESTIMATOR_TYPES PoseInference::getPoseEstimatorType ( )
inlineoverridevirtual

◆ getPoseEstimatorType() [2/2]

const POSEESTIMATOR_TYPES PoseInference::getPoseEstimatorType ( )
inlineoverridevirtual

Get the configured pose-estimator type.

Implements IPoseEstimator.

Definition at line 146 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

References m_params, and PoseInferenceConfigParams::m_poseEstimatorType.

◆ initializeModel() [1/2]

void PoseInference::initializeModel ( )
overridevirtual

Create TensorRT inference engine; if engine does not exist, serialize from model file.

Implements IPoseEstimator.

◆ initializeModel() [2/2]

void PoseInference::initializeModel ( )
overridevirtual

Create TensorRT inference engine; if engine does not exist, serialize from model file.

Implements IPoseEstimator.

◆ initializeRuntime() [1/2]

void PoseInference::initializeRuntime ( const SourceFrameInfo &  frameInfo,
const uint  maxTargetsPerStream,
const std::shared_ptr< NvTrackedObjectManager > &  targetManager 
)
overridevirtual

Initialize variables and allocate memories.

Parameters
[in]frameInfoMetadata about the source image, such as size and color.
[in]maxTargetsPerStreamMaximum number of targets to track in each stream.
[in]targetManagerTarget manager of the tracked objects.

Implements IPoseEstimator.

◆ initializeRuntime() [2/2]

void PoseInference::initializeRuntime ( const SourceFrameInfo &  frameInfo,
const uint  maxTargetsPerStream,
const std::shared_ptr< NvTrackedObjectManager > &  targetManager 
)
overridevirtual

Initialize variables and allocate memories.

Parameters
[in]frameInfoMetadata about the source image, such as size and color.
[in]maxTargetsPerStreamMaximum number of targets to track in each stream.
[in]targetManagerTarget manager of the tracked objects.

Implements IPoseEstimator.

◆ queryParams() [1/2]

void PoseInference::queryParams ( NvMOTQuery &  paramsQuery)
virtual

Populate the capabilities reported to the tracker.

Parameters
[out]paramsQueryThe output query parameters.

Implements IPoseEstimator.

◆ queryParams() [2/2]

void PoseInference::queryParams ( NvMOTQuery &  paramsQuery)
virtual

Populate the capabilities reported to the tracker.

Parameters
[out]paramsQueryThe output query parameters.

Implements IPoseEstimator.

◆ removeStream() [1/2]

void PoseInference::removeStream ( const NvMOTStreamId  videoStreamId)
overridevirtual

Remove a stream from the pose estimator.

Parameters
[in]videoStreamIdVideo stream id.

Implements IPoseEstimator.

◆ removeStream() [2/2]

void PoseInference::removeStream ( const NvMOTStreamId  videoStreamId)
overridevirtual

Remove a stream from the pose estimator.

Parameters
[in]videoStreamIdVideo stream id.

Implements IPoseEstimator.

◆ setValues4PoseGeometryInput() [1/2]

void PoseInference::setValues4PoseGeometryInput ( )
protected

Upload fixed geometry tensors used by pose inference.

◆ setValues4PoseGeometryInput() [2/2]

void PoseInference::setValues4PoseGeometryInput ( )
protected

Upload fixed geometry tensors used by pose inference.

◆ update() [1/2]

void PoseInference::update ( )
overridevirtual

Update the pose estimator after data association.

Implements IPoseEstimator.

◆ update() [2/2]

void PoseInference::update ( )
overridevirtual

Update the pose estimator after data association.

Implements IPoseEstimator.

Field Documentation

◆ m_batchMaskDecSize_d

float * PoseInference::m_batchMaskDecSize_d
protected

◆ m_bindings

std::vector< void * > PoseInference::m_bindings
protected

Pointers to TensorRT tensors.

Definition at line 170 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_cuStream

cudaStream_t PoseInference::m_cuStream
protected

Cuda stream to run all GPU tasks.

Definition at line 169 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_featuresEncoder_d

float * PoseInference::m_featuresEncoder_d
protected

Encoder output features.

Definition at line 171 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_inference

TrtModule PoseInference::m_inference
protected

◆ m_inputBuf_d

float * PoseInference::m_inputBuf_d
protected

◆ m_inputBuf_h

float * PoseInference::m_inputBuf_h
protected

Input patches to encoder.

Definition at line 152 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_inputMask_d

float * PoseInference::m_inputMask_d
protected

Mask prompt, currently unused and set as 0.

Definition at line 155 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_inputMaskLabel_d

float * PoseInference::m_inputMaskLabel_d
protected

Mask label, currently unused and set as 0.

Definition at line 156 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_inputRoisInBatch

std::map< NvMOTStreamId, std::vector< std::pair< uint, NvMotDataTypes::Rect2f > > > PoseInference::m_inputRoisInBatch
protected

Reid input rois to crop.

Definition at line 176 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_isRuntimeInit

bool PoseInference::m_isRuntimeInit
protected

Reid status if runtime variables initialized.

Definition at line 149 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_kInv_d

float * PoseInference::m_kInv_d
protected

◆ m_maxPointNum

int PoseInference::m_maxPointNum
protected

◆ m_maxTargetsToTrack

int PoseInference::m_maxTargetsToTrack
protected

Max number of targets to keep for all streams.

Definition at line 178 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_meanLimbLengths_d

float * PoseInference::m_meanLimbLengths_d
protected

Device inverse intrinsics, inverse transforms, and limb-length priors.

Definition at line 160 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_numBboxes

int PoseInference::m_numBboxes
protected

Number of detected boxes in the current batch tracker recieved.

Definition at line 151 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_originalSrcImageSize

NvMotDataTypes::Size PoseInference::m_originalSrcImageSize
protected

Size of the original source image.

Definition at line 166 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_outputIou_d

float * PoseInference::m_outputIou_d
protected

◆ m_params

PoseInferenceConfigParams PoseInference::m_params
protected

Config params for reid module.

Definition at line 164 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

Referenced by getPoseEstimatorType().

◆ m_pointLabels_d

float * PoseInference::m_pointLabels_d
protected

Input a list of point types (single point, bounding box corners) to decoder.

Definition at line 154 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_points_d

float * PoseInference::m_points_d
protected

Input a list of points (prompt) to decoder.

Definition at line 153 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_pose25d_d

float * PoseInference::m_pose25d_d
protected

◆ m_pose25d_h

float * PoseInference::m_pose25d_h
protected

◆ m_pose2d_d

float * PoseInference::m_pose2d_d
protected

◆ m_pose2d_org_img_d

float * PoseInference::m_pose2d_org_img_d
protected

◆ m_pose3d_d

float * PoseInference::m_pose3d_d
protected

Device 2D, original-image 2D, 2.5D, and 3D poses.

Definition at line 161 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_pose3d_h

float * PoseInference::m_pose3d_h
protected

◆ m_scaleNormalizedMeanLimbLengths_d

float * PoseInference::m_scaleNormalizedMeanLimbLengths_d
protected

◆ m_srcImageSize

NvMotDataTypes::Size PoseInference::m_srcImageSize
protected

Size of the source image.

Definition at line 165 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_subImageCropper2

std::shared_ptr< ReIDFrameTransformTexture > PoseInference::m_subImageCropper2
protected

Image cropper, may share with visual tracker in the future.

Definition at line 173 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_subImageCropScaler

std::shared_ptr< ReIDCropScaler > PoseInference::m_subImageCropScaler
protected

Image crop scaler.

Uses VPI crop scaler.

Definition at line 174 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_targetManager

std::shared_ptr< NvTrackedObjectManager > PoseInference::m_targetManager
protected

Target manager providing target status to update feature gallery.

Definition at line 172 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_targetMaskDecSize_d

float * PoseInference::m_targetMaskDecSize_d
protected

◆ m_targetMaskDecSize_h

float * PoseInference::m_targetMaskDecSize_h
protected

Mask output for each target.

Definition at line 157 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_targetMaskOriginalSrcSize_d

float * PoseInference::m_targetMaskOriginalSrcSize_d
protected

Mask for each target resized to original streammux size.

Definition at line 159 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_targetStreamIdList

std::vector< std::tuple< uint, NvMOTStreamId, NvMotDataTypes::Rect2f > > PoseInference::m_targetStreamIdList
protected

Target index, stream ID, and crop ROI for each box.

Definition at line 167 of file sources/libs/nvmultiobjecttracker/include/modules/PoseEstimator/PoseInference.hpp.

◆ m_tFormInv_d

float * PoseInference::m_tFormInv_d
protected

The documentation for this class was generated from the following file: