Aaron Defazio

Computer Science · Physical Sciences

17h-index3.3kcitations70works0.62yr avg
OpenAlex↗ORCID↗Google Scholar↗

Accepting Students?

No reports yet. Know if this professor is taking students?

Research Topics

Stochastic Gradient Optimization Techniques(32), Sparse and Compressive Sensing Techniques(27), Medical Imaging Techniques and Applications(15), Advanced MRI Techniques and Applications(9), Advanced Neural Network Applications(8)

Publications70 total

Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs
arXiv (Cornell University)·2025Open Access
Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs
ArXiv.org·2025Open Access
Why Gradients Rapidly Increase Near the End of Training
arXiv (Cornell University)·2025Open Access
PARQ: Piecewise-Affine Regularized Quantization
arXiv (Cornell University)·2025Open Access
The Road Less Scheduled
arXiv (Cornell University)·2024· 1 citedOpen Access
Directional Smoothness and Gradient Methods: Convergence and Adaptivity
arXiv (Cornell University)·2024Open Access
The Road Less Scheduled
2024· 3 cited
Directional Smoothness and Gradient Methods: Convergence and Adaptivity
2024· 1 cited
Optimal Linear Decay Learning Rate Schedules and Further Refinements
arXiv (Cornell University)·2023· 5 citedOpen Access
Prodigy: An Expeditiously Adaptive Parameter-Free Learner
arXiv (Cornell University)·2023· 3 citedOpen Access
Mechanic: A Learning Rate Tuner
arXiv (Cornell University)·2023Open Access
MoMo: Momentum Models for Adaptive Learning Rates
arXiv (Cornell University)·2023Open Access
Learning-Rate-Free Learning by D-Adaptation
arXiv (Cornell University)·2023· 5 citedOpen Access
Compressed sensing with a jackknife and a bootstrap
Journal of Data Science Statistics and Visualisation·2022
Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method
arXiv (Cornell University)·2022Open Access
A scaling calculus for the design and initialization of ReLU networks
Neural Computing and Applications·2022Open Access
Compressed sensing with a jackknife, a bootstrap, and visualization
Zenodo (CERN European Organization for Nuclear Research)·2021· 1 citedOpen Access
On the (asymptotic) convergence of Stochastic Gradient Descent and Stochastic Heavy Ball
Conference on Learning Theory·2021
Stochastic Polyak Stepsize with a Moving Target
arXiv (Cornell University)·2021Open Access
Compressed sensing with a jackknife and a bootstrap
Zenodo (CERN European Organization for Nuclear Research)·2021· 5 citedOpen Access
page 1 of 4Next →

Frequent Co-authors

Jure Žbontar(9), Anuroop Sriram(7), C. Lawrence Zitnick(7), Tullie Murrell(7), Robert M. Gower(7), Michael P. Recht(6), Daniel K. Sodickson(6), Florian Knöll(5), Nafissa Yakubova(5), Konstantin Mishchenko(5), Matthew J. Muckley(4), Michael Rabbat(4), Tibério S. Caetano(4), Léon Bottou(4), Yvonne W. Lui(3), Othmane Sebbouh(3), Mark Tygert(3), Harsh Mehta(3), Ahmed Khaled(3), Francis Bach(2)