Class DeserializeStage

• Defined in File deserialize.hpp

Base Type

• public mrc::pymrc::PythonNode< std::shared_ptr< MessageMeta >, std::shared_ptr< MultiMessage > >

class DeserializeStage : public mrc::pymrc::PythonNode<std::shared_ptr<MessageMeta>, std::shared_ptr<MultiMessage>>

Slices incoming Dataframes into smaller batch_size’d chunks. This stage accepts the MessageMeta output from FileSourceStage/KafkaSourceStage stages breaking them up into into MultiMessage’s. This should be one of the first stages after the Source object.

Public Types

using base_t = mrc::pymrc::PythonNode<std::shared_ptr<MessageMeta>, std::shared_ptr<MultiMessage>>

Public Functions

DeserializeStage(TensorIndex batch_size, bool ensure_sliceable_index = true)

Construct a new Deserialize Stage object.

Parameters

• batch_size – Number of messages to be divided into each batch

• ensure_sliceable_index – Whether or not to call ensure_sliceable_index() on all incoming MessageMeta