(Latest Version)

Class DeserializeStage

Base Type

  • public mrc::pymrc::PythonNode< std::shared_ptr< MessageMeta >, std::shared_ptr< MultiMessage > >

class DeserializeStage : public mrc::pymrc::PythonNode<std::shared_ptr<MessageMeta>, std::shared_ptr<MultiMessage>>

Slices incoming Dataframes into smaller batch_size’d chunks. This stage accepts the MessageMeta output from FileSourceStage/KafkaSourceStage stages breaking them up into into MultiMessage’s. This should be one of the first stages after the Source object.

Public Types

using base_t = mrc::pymrc::PythonNode<std::shared_ptr<MessageMeta>, std::shared_ptr<MultiMessage>>

Public Functions

DeserializeStage(TensorIndex batch_size, bool ensure_sliceable_index = true)

Construct a new Deserialize Stage object.

Parameters
  • batch_size – Number of messages to be divided into each batch

  • ensure_sliceable_index – Whether or not to call ensure_sliceable_index() on all incoming MessageMeta

© Copyright 2023, NVIDIA. Last updated on Apr 11, 2023.