Qwen3-235B-A22B#
Qwen is a family of large language models developed by Alibaba Cloud, including dense models (Qwen2, Qwen2.5, Qwen3) and Mixture-of-Experts models (Qwen3 MoE, Qwen3-Next). The models feature innovations like QK layernorm, Gated-Delta Networks, and Zero-Centered RMSNorm for improved training stability and performance.
Qwen family models are supported via the Bridge system with auto-detected configuration and weight mapping.
Verified configurations#
Choose an exact recorded configuration to see its command and expected result. These selectors are generated from the authoritative verification cards and never synthesize combinations.
Run a configuration#
Choose a workflow, precision, and exact recorded combination. The command and expected result update below.
Import · CPU
â—‹ UnverifiedExact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a public run against the pinned Hugging Face revision with all applicable verification gates.
Import · GPU
â—‹ UnverifiedExact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a public run against the pinned Hugging Face revision with all applicable verification gates.
Export · CPU
â—‹ UnverifiedExact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a public run against the pinned Hugging Face revision with all applicable verification gates.
Export · GPU
â—‹ UnverifiedExact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a public run against the pinned Hugging Face revision with all applicable verification gates.
Pretrain · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a bounded public run with the model's pinned revision and all applicable verification gates.
SFT · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a bounded public run with the model's pinned revision and all applicable verification gates.
Long Context · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a bounded public run with the model's pinned revision and all applicable verification gates.
LoRA · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
This workflow remains unverified and requires a bounded public run with the model's pinned revision and all applicable verification gates.
Benchmark · GB300
✓ VerifiedRecorded metrics
- Initial loss
- 12.75473
- Final loss
- 8.126335
- Step time · last 10 avg
- 14,920.840 ms
- Model throughput · last 10 avg
- 1,300.440 TFLOP/s/GPU
- Token throughput · last 10 avg
- 8,784.492 tokens/s/GPU
Exact command
./scripts/training/train.sh --wait --nodes 64 --gpus-per-node 4 --recipe qwen3_235b_a22b_pretrain_256gpu_gb300_fp8mx_config --mode pretrain --max_steps 50 --seq_length 4096 logger.save_config_filepath=work/model-verification/qwen3-235b-a22b/gb300-performance/ConfigContainer.yamlExpected result
On exactly 256 GB300s, the canonical MXFP8 mock-data recipe completes exactly 50 optimizer steps at TP1/PP4/CP1/EP32/ETP1, GBS/MBS 8192/2, and sequence length 4096. All 50 keyed rows have finite loss with zero skipped or NaN iterations. Loss moves from 12.75473 to 8.126335; the final ten steps average 14920.840 ms, 1300.440 TFLOP/s/GPU, and 8784.492 tokens/s/GPU. The resolved configuration persists.
Benchmark · GB200
✓ VerifiedRecorded metrics
- Initial loss
- 12.75152
- Final loss
- 8.126326
- Step time · last 10 avg
- 17,905.340 ms
- Model throughput · last 10 avg
- 1,083.660 TFLOP/s/GPU
- Token throughput · last 10 avg
- 7,320.274 tokens/s/GPU
Exact command
./scripts/training/train.sh --wait --nodes 64 --gpus-per-node 4 --recipe qwen3_235b_a22b_pretrain_256gpu_gb200_fp8mx_config --mode pretrain --max_steps 50 --seq_length 4096 logger.save_config_filepath=work/model-verification/qwen3-235b-a22b/gb200-performance/ConfigContainer.yamlExpected result
On exactly 256 GB200s, the canonical MXFP8 mock-data recipe completes exactly 50 optimizer steps at TP1/PP8/CP1/EP32/ETP1, VPP3, GBS/MBS 8192/1, and sequence length 4096. All 50 keyed rows have finite loss with zero skipped or NaN iterations. Loss moves from 12.75152 to 8.126326; the final ten steps average 17905.340 ms, 1083.660 TFLOP/s/GPU, and 7320.274 tokens/s/GPU. The resolved configuration persists.