bridge.recipes.stepfun.h100.step37

Step3.7 (stepfun-ai/Step-3.7-Flash) recipe.

Only the Flickr8k SFT path is supported. Step37Model.forward takes list[ImageForInsert] directly, and the data path is the self-contained Step37Flickr8kSFTDataProvider (HF datasets / processor not involved).

Module Contents

Functions

step37_sft_64gpu_h100_bf16_flickr8k_config	Step3.7 SFT recipe — the only supported Step3.7 path.
step37_sft_4gpu_h100_bf16_flickr8k_smoke_config	Smoke variant of :func:step37_flickr8k_sft_config — the same packed sample on every DP rank, every step. Deterministic and tiny: it repeats pack[fixed_pack_idx] indefinitely so the loss curve visibly drops as the model overfits a single batch.

Data

_STEP37_HF_PATH
_STEP37_FLICKR8K_SAMPLE_COUNT
_STEP37_FLICKR8K_MAX_PACKING_SEQLEN
_STEP37_FLICKR8K_SEQLEN_DIVISIBLE_BY
_STEP37_FLICKR8K_OVERSIZE_POLICY
_STEP37_FLICKR8K_DATASET_SAMPLING
_STEP37_FLICKR8K_CACHE_DIR
_STEP37_FLICKR8K_PROMPT
_STEP37_FLICKR8K_SMOKE_CACHE_DIR
_STEP37_FLICKR8K_SMOKE_FIXED_PACK_IDX
_STEP37_FLICKR8K_SMOKE_TRAIN_ITERS
_STEP37_FLICKR8K_SMOKE_MAX_LR
__all__

API

bridge.recipes.stepfun.h100.step37._STEP37_HF_PATH

‘stepfun-ai/Step-3.7-Flash’

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_SAMPLE_COUNT

8

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_MAX_PACKING_SEQLEN

2048

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_SEQLEN_DIVISIBLE_BY

64

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_OVERSIZE_POLICY

‘drop’

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_DATASET_SAMPLING

‘random’

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_CACHE_DIR

‘.cache/step37_flickr8k’

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_PROMPT

‘Describe this image in one sentence.’

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_SMOKE_CACHE_DIR

‘.cache/step37_flickr8k_smoke’

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_SMOKE_FIXED_PACK_IDX

0

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_SMOKE_TRAIN_ITERS

100

bridge.recipes.stepfun.h100.step37._STEP37_FLICKR8K_SMOKE_MAX_LR

0.005

bridge.recipes.stepfun.h100.step37.step37_sft_64gpu_h100_bf16_flickr8k_config() → megatron.bridge.training.config.ConfigContainer

Step3.7 SFT recipe — the only supported Step3.7 path.

Uses the Flickr8k packed pipeline:

• cfg.dataset is :class:Step37Flickr8kSFTDataProvider (sync packing, no async wrapper, no HFConversationDatasetProvider).

• --step_func step37_flickr8k_step consumes the packed dict and passes list[ImageForInsert] straight to Step37Model.forward.

• micro_batch_size is pinned at 1 — each pack already aggregates multiple sub-seqs via cu_seqlens.

• Tokenizer loaded with trust_remote_code=False; no HF custom Python code runs in the data path.

The default train split is limited to 8 samples for smoke coverage. Use CLI overrides such as dataset.sample_count=null for a full Flickr8k run.

bridge.recipes.stepfun.h100.step37.step37_sft_4gpu_h100_bf16_flickr8k_smoke_config() → megatron.bridge.training.config.ConfigContainer

Smoke variant of :func:step37_flickr8k_sft_config — the same packed sample on every DP rank, every step. Deterministic and tiny: it repeats pack[fixed_pack_idx] indefinitely so the loss curve visibly drops as the model overfits a single batch.

Differences vs. the regular SFT config:

• dataset.fixed_pack_idx pins __getitem__ → identical input across every DP rank and every iteration.

• dataset.dataset_sampling = "sequential" for reproducibility.

• max_lr bumped 5e-6 → 5e-3 so the overfit happens within train_iters steps.

• Language model unfrozen (the regular config freezes it); vision tower stays frozen (overfitting on the projector + LM is enough and avoids the PE-G/14 backward cost).

• log_interval=1, eval disabled, no mid-run checkpoint save.

The smoke recipe repeats pack 0 for 100 iterations with a high learning rate so loss can drop quickly.

bridge.recipes.stepfun.h100.step37.__all__

[‘step37_sft_4gpu_h100_bf16_flickr8k_smoke_config’, ‘step37_sft_64gpu_h100_bf16_flickr8k_config’]