Newton–Schulz orthogonalization quality#
This doc summarizes quality of orthogonalization by Newton-Schulz compared polar decomposition (QDWH).
Run settings#
Setting |
Value |
|---|---|
Matrix shape |
|
Inputs |
FP32 Gaussian |
Logspace input |
|
Steps |
5, 8, 10, 16; |
Polar Express schedule |
Upstream |
NS precision |
FP32 tensors with |
BF16 reduced-precision reduction |
|
Reference |
|
python -m benchmarks.newton_schulz_quality \
--precisions=highest,high,medium \
--steps=5,8,10,16 --spectra=gaussian,uniform,hilbert,logspace \
--output=/tmp/newton-schulz-quality.json \
--plot=/tmp/newton-schulz-quality.png
Results#
Dense logspace input after FP32 conversion: measured singular values 1.54 to 1e+10, condition number 6.47e+09.
For original input A, NS output X, and FP64 reference polar factors A = H Q:
