Papers

For natural-gradient (NG) methods:

• Improving optimizers Shampoo and SOAP via Riemannian/proximal gradient descent (ICLR 2026): Paper, Code

• Adaptive gradient methods as NG descent (ICML 2024): Paper, Code

• Structured NG descent for deep learning (ICML 2023, ICML 2024): Manifold View, Code 2023; Bayesian View, Code 2024

• Structured NG descent (ICML 2021): Paper, Blog

• Riemannian gradient descent (ICML 2020): Paper, Code

• NG descent for exponential-family mixtures (ICML 2019): Paper, Code

• NG descent for Bayesian deep learning (ICML 2018): Paper, Code

• NG variational inference for non-conjugate models (AI&Stats 2017): Paper, Code

• Improving variational inference via proximal gradient descent (UAI 2016): Paper

For an introduction to NG methods, see my Blog Posts.

For more publications, see my Google Scholar page.