Skip to main content
Back to top
Ctrl
+
K
Emerging-Optimizers
Choose version
Search
Ctrl
+
K
GitHub
Search
Ctrl
+
K
Emerging-Optimizers
Choose version
GitHub
Table of Contents
🛠️ Development
API Reference
emerging_optimizers.orthogonalized_optimizers
emerging_optimizers.soap
emerging_optimizers.riemannian_optimizers
emerging_optimizers.psgd
emerging_optimizers.scalar_optimizers
emerging_optimizers.mixin
emerging_optimizers.utils
Documentation Development
Primer
Layer-wise distributed optimizer
Epsilon
\(\epsilon\)
in Adam optimizer
Gradient spikes and the SOAP preconditioner
Comparing optimizer updates: warmup and spike response
Preconditioner eigenbasis rotation around a gradient spike (SOAP vs REKLS)
TF32
Q^T
L
Q
precision loss — standalone demo
Primer
Primer
#
