Step #3: Train the Model

After you have finished pre-processing, the next step is to pre-train the GPT model. This next notebook uses Megatron’s Tensor parallelism and pipeline parallelism.

Within your Jupyter lab tab, open Training_Megatron notebook and run it.

Once you are done running through the notebook, proceed with Step #4.