Class DeserializeStage

• Defined in File deserialize.hpp

Base Type

• public mrc::pymrc::PythonNode< std::shared_ptr< MessageMeta >, std::shared_ptr< MultiMessage > >

class DeserializeStage : public mrc::pymrc::PythonNode<std::shared_ptr<MessageMeta>, std::shared_ptr<MultiMessage>>

Slices incoming Dataframes into smaller batch_size’d chunks. This stage accepts the MessageMeta output from FileSourceStage/KafkaSourceStage stages breaking them up into into MultiMessage’s. This should be one of the first stages after the Source object.

Public Types

using base_t = mrc::pymrc::PythonNode<std::shared_ptr<MessageMeta>, std::shared_ptr<MultiMessage>>

Public Functions

DeserializeStage(size_t batch_size)

Construct a new Deserialize Stage object.

Parameters

batch_size – : Number of messages to be divided into each batch