bridge.recipes.gpt.gpt3_175b

Module Contents

Functions

gpt3_175b_pretrain_config

Return a pre-training config for GPT3-175B.

API

bridge.recipes.gpt.gpt3_175b.gpt3_175b_pretrain_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for GPT3-175B.

The default configuration is expected to run on 64 nodes with 8 GPUs each. Default parallelism: TP=4, PP=8, VP=6, SP=True.