DocumentationAPI Reference
DocumentationAPI Reference
  • API Reference
    • Overview
        • Nemo Curator
          • Backends
          • Config
          • Core
          • Metrics
          • Models
          • Package Info
          • Pipeline
          • Stages
            • Audio
            • Base
            • Client Partitioning
            • Deduplication
              • Exact
              • Fuzzy
              • Gpu Utils
              • Id Generator
              • Io Utils
              • Semantic
              • Shuffle Utils
            • File Partitioning
            • Function Decorators
            • Image
            • Interleaved
            • Math
            • Resources
            • Synthetic
            • Text
            • Video
          • Tasks
          • Utils
    • Pipeline
    • ProcessingStage
    • CompositeStage
    • Resources
On this page
  • Submodules
API ReferenceFull Library ReferenceNemo CuratorNemo CuratorStagesDeduplication

nemo_curator.stages.deduplication.shuffle_utils

||View as Markdown|

Submodules

  • nemo_curator.stages.deduplication.shuffle_utils.rapidsmpf_shuffler
  • nemo_curator.stages.deduplication.shuffle_utils.stage
Previous

nemo_curator.stages.deduplication.semantic.workflow

Next

nemo_curator.stages.deduplication.shuffle_utils.rapidsmpf_shuffler

NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.

LogoLogoNeMo Curator