Train a Large-Scale NLP Model with NeMo Megatron
NVIDIA Docs Hub  NVIDIA LaunchPad  Train a Large-Scale NLP Model with NeMo Megatron  Intro to BigNLP Scripts
Train a Large-Scale NLP Model with NeMo Megatron (Latest Release)

Intro to BigNLP Scripts

BigNLP scripts code is a launcher wrapper to run GPT-3 end-to-end. The scripts can run on either Slurm or on the NVIDIA NGC™ compute platform, also known as NVIDIA Base Command Platform. We will be running this on Base Command Platform. A container with the BigNLP code is provided in the LaunchPad registry: nvcr.io/insert-your-org-here/bignlp-training:22.02-py3.

Note

Remember - your Org and ACE values are likely different than what is used in this specific example, and should be used in place of the values used in this example. The Org should be of the form nv-launchpad-bc*, and the ACE nv-launchpad*.

So in the example above, if your Org value is nv-launchpad-bc, the NGC registry URL would be nvcr.io/nv-launchpad-bc/bignlp-training:22.02-py3.

© Copyright 2022-2023, NVIDIA. Last updated on Jan 10, 2023.