morpheus.pipeline.preallocator_mixin.PreallocatorMixin
- class PreallocatorMixin[source]
Bases:
abc.ABC
Mixin intented to be added to stages, typically source stages, which are emitting newly constructed DataFrame or MessageMeta instances into the segment. During segment build, if the
_needed_columns
addtribut is not empty an additional node will be inserted into the graph after the derived class’ node which will perform the allocation.The exceptions would be non-source stages like DFP’s
DFPFileToDataFrameStage
which are not sources but are constructing new Dataframe instances, andLinearBoundaryIngressStage
which is potentially emitting other message types such as MultiMessages and it’s various derived messages but it would still be the first stage in the given segment emitting the message.Methods
set_needed_columns
(needed_columns)Sets the columns needed to perform preallocation. - set_needed_columns(needed_columns)[source]
Sets the columns needed to perform preallocation. This should only be called by the Pipeline at build time. The needed_columns shoudl contain the entire set of columns needed by any other stage in this segment.