Struct TensorTransmitCache
Defined in File holoinfer_utils.hpp
-
struct TensorTransmitCache
Persistent cache for output message and tensor allocations used by the cached variant of transmit_data_per_model. Holds a reusable GXF entity and per-tensor metadata to minimise per-frame overhead. Three cases are handled each frame:
First use or buffer too small: full reshape (reallocate + update shape).
Incoming fits in existing allocation but dims changed: wrapMemory to update shape metadata only — no free/alloc of the underlying buffer.
Same dims as last frame: fast path, no tensor mutation at all.
Public Members
-
nvidia::gxf::Expected<nvidia::gxf::Entity> out_message{nvidia::gxf::Unexpected{GXF_UNINITIALIZED_VALUE}}
Persistent output entity. Invalid (falsy) until the first call to transmit_data_per_model.
-
std::map<std::string, size_t> allocated_sizes
Maximum element count that has been allocated for each output tensor.
-
std::map<std::string, std::vector<int64_t>> last_dims
Dimension vector from the most recent frame for each output tensor, used to detect shape changes that require a wrapMemory call even when the element count has not grown.