emerging_optimizers.riemannian_optimizers#

class emerging_optimizers.riemannian_optimizers.normalized_optimizer.ObliqueAdam( params, lr=0.001, betas=(0.9, 0.99), weight_decay=0.0, *, weight_decay_method='decoupled', dim=0, eps=1e-08, correct_bias=True, )[source]#

Adam optimizer for row- or column-normalized 2D parameters on oblique manifolds.

This optimizer adapts an Adam-like algorithm to work on oblique manifolds, where parameters are constrained to have unit-norm rows or columns. It combines adaptive momentum estimation with Riemannian gradient computation and manifold retraction.

Parameters:

params (list[Parameter])
lr (float)
betas (tuple[float, float])
weight_decay (float)
weight_decay_method (Literal['decoupled', 'independent', 'l2', 'palm'])
dim (int)
eps (float)
correct_bias (bool)

step(closure: None = None) → None[source]#

step(closure: Callable[[], float]) → float

Performs a single optimization step.

Parameters:: closure – Unsupported; must be None.

class emerging_optimizers.riemannian_optimizers.normalized_optimizer.ObliqueSGD( params, lr=0.001, momentum=0.9, weight_decay=0.0, *, weight_decay_method='decoupled', dim=0, eps=1e-08, )[source]#

SGD optimizer for row- or column-normalized 2D parameters on oblique manifolds.

This optimizer performs SGD on oblique manifolds, where parameters are constrained to have unit-norm rows or columns. It implements Riemannian SGD with manifold-aware gradient updates and retraction operations.

References

An Introduction to Optimization on Smooth Manifolds (Nicolas Boumal)
EDM2: https://arxiv.org/abs/2312.02696
Jianlin Su: https://kexue.fm/archives/11196
Raman et al.: https://arxiv.org/abs/1909.06463
Franz Cesista: https://leloykun.github.io/ponder/steepest-descent-stiefel/#6-bonus-a-muon-like-optimizer-for-the-embedding-and-unembedding-layers

Parameters:

lr (float) – learning rate
momentum (float) – momentum coefficient
weight_decay (float) – weight decay coefficient
weight_decay_method (Literal['decoupled', 'independent', 'l2', 'palm']) – Method to apply weight decay.
dim (int) – The dimension to normalize over
eps (float) – epsilon for numerical stability
params (list[Parameter])

step(closure: None = None) → None[source]#

step(closure: Callable[[], float]) → float

Performs a single optimization step.

Parameters:: closure – Unsupported; must be None.

class emerging_optimizers.riemannian_optimizers.isospectral.Iso( params, lr=0.001, momentum=0.9, retraction='qr', *, fp32_matmul_prec='highest', )[source]#

Isospectral optimizer for two-dimensional parameters.

The optimizer factorizes each parameter as U @ diag(Sigma) @ V.T and updates both Stiefel factors while keeping Sigma fixed. It is designed for reinforcement learning with verifiable rewards (RLVR), particularly for LLM reasoning post-training, but its implementation does not depend on an RL-specific training interface.

References

ISO: An RLVR-Native Optimization Stack. arXiv:2607.19331 (2026). [arXiv:2607.19331]

Parameters:

params (Iterable[Tensor] | Iterable[dict[str, Any]] | Iterable[tuple[str, Tensor]]) – Parameters to optimize.
lr (float) – Learning rate.
momentum (float) – Momentum coefficient.
retraction (Literal['qr', 'polar', 'cayley']) – Retraction used to restore the Stiefel constraints.
fp32_matmul_prec (Literal['highest', 'high', 'medium']) – Precision used for FP32 matrix multiplications.

step(closure: None = None) → None[source]#
step(closure: Callable[[], float]) → float: Perform a single optimization step.