After you have finished pre-processing, the next step is to pre-train the GPT model. This next notebook uses Megatron’s Tensor parallelism and pipeline parallelism.
Within your Jupyter lab tab, open Training_Megatron notebook and run it.
Once you are done running through the notebook, proceed with Step #4.