Performance#

As part of the NVIDIA NeMo Framework, Megatron Bridge, provides optimal performance for training advanced generative AI models by incorporating the most recent training techniques, such as model parallelization, optimized attention mechanisms, and more, to achieve high training throughput.

This page provides performance benchmarks for large language models using Megatron-Bridge across different GPU systems and configurations.

Nomenclature#

GBS: Global Batch Size
MBS: Micro Batch Size
TP: Tensor Parallel Size
PP: Pipeline Parallel Size
CP: Context Parallel Size
VP: Virtual Pipeline Parallel Size
EP: Expert Parallel Size
GA: Number of Gradient Accumulations

Performance Metrics#

Performance is measured using:

Tokens/sec/GPU: Throughput per GPU
Model TFLOP/sec/GPU: Model floating-point operations per second per GPU

Performance Summary for Large Language Models#

Below are performance benchmarks for various large language models. These results were obtained using performance recipes available here.

The performance data includes:

Pre-training Performance: Throughput metrics for various model sizes and architectures[1]
System Configurations: Results across different GPU systems (DGX-GB300, DGX-GB200, DGX-B300, DGX-H100)
Precision Options: Performance comparisons between different precision modes (BF16, FP8, MXFP8, NVFP4)

26.06.01 NeMo Container#

Pre-Training Performance#

Model: DeepSeekV3#

System	#-GPUs	Precision	GBS	MBS	Sequence Length	TP	PP	CP	VP	EP	Tokens / sec / GPU	Model TFLOP / sec / GPU
DGX-GB300	256	MXFP8	4096	1	4096	1	2	1	8	32	6304	1640
DGX-GB200	256	MXFP8	4096	1	4096	1	4	1	4	64	4928	1280

Model: GPT OSS 120B#

System	#-GPUs	Precision	GBS	MBS	Sequence Length	TP	PP	CP	VP	EP	Tokens / sec / GPU	Model TFLOP / sec / GPU
DGX-GB300	64	MXFP8	1280	4	4096	1	1	1	n/a	16	20288	661[2]
DGX-GB200	64	MXFP8	1280	4	4096	1	1	1	n/a	64	18304	597[2]

Model: Qwen3_30B_a3B#

System	#-GPUs	Precision	GBS	MBS	Sequence Length	TP	PP	CP	VP	EP	Tokens / sec / GPU	Model TFLOP / sec / GPU
DGX-GB300	8	MXFP8	512	8	4096	1	1	1	n/a	8	44544	1029
DGX-GB200	8	MXFP8	512	4	4096	1	1	1	n/a	8	40960	937

Model: Qwen3_235B_a22B#

System	#-GPUs	Precision	GBS	MBS	Sequence Length	TP	PP	CP	VP	EP	Tokens / sec / GPU	Model TFLOP / sec / GPU
DGX-GB300	256	MXFP8	8192	2	4096	1	4	1	12	32	9008	1333
DGX-GB200	256	MXFP8	8192	1	4096	1	8	1	3	32	7360	1089

Model: Nemotron_3_Nano#

System	#-GPUs	Precision	GBS	MBS	Sequence Length	TP	PP	CP	VP	EP	Tokens / sec / GPU	Model TFLOP / sec / GPU
DGX-GB300	8	MXFP8	512	4	8192	1	1	1	n/a	8	40960	905
DGX-GB200	8	MXFP8	512	2	8192	1	1	1	n/a	8	34816	776

Model: Nemotron_3_Super#

System	#-GPUs	Precision	GBS	MBS	Sequence Length	TP	PP	CP	VP	EP	Tokens / sec / GPU	Model TFLOP / sec / GPU
DGX-GB300	64	MXFP8	512	1	8192	1	1	1	n/a	64	9728	827
DGX-GB300	64	NVFP4	512	1	8192	1	1	1	n/a	64	9984	845
DGX-GB200	64	MXFP8	512	1	8192	2	1	1	n/a	64	7040	598
DGX-GB200	64	NVFP4	512	1	8192	2	1	1	n/a	64	7040	598

Archive#

Performance summary for past releases can be found in the archive.