Jack W. Rae

Computer Science · Physical Sciences

22h-index4.5kcitations34works270.02yr avg
OpenAlex↗Google Scholar↗

Accepting Students?

No reports yet. Know if this professor is taking students?

Research Topics

Topic Modeling(16), Natural Language Processing Techniques(11), Multimodal Machine Learning Applications(9), Advanced Neural Network Applications(5), Neural Networks and Applications(5)

Publications32 total

Training Compute-Optimal Large Language Models
arXiv (Cornell University)·2022· 647 citedOpen Access
Unified Scaling Laws for Routed Language Models
arXiv (Cornell University)·2022· 23 citedOpen Access
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
arXiv (Cornell University)·2021· 242 citedOpen Access
Improving language models by retrieving from trillions of tokens
arXiv (Cornell University)·2021· 295 citedOpen Access
Top-KAST: Top-K Always Sparse Training
arXiv (Cornell University)·2021· 21 citedOpen Access
Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records
Nature Protocols·2021· 101 cited
Towards Lifelong Reasoning with Sparse and Compressive Memory Systems
UCL Discovery (University College London)·2021
Stabilizing Transformers for Reinforcement Learning
International Conference on Machine Learning·2020· 52 cited
Do Transformers Need Deep Long-Range Memory
arXiv (Cornell University)·2020· 1 citedOpen Access
Multiplicative Interactions and Where to Find Them
International Conference on Learning Representations·2020· 59 cited
V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control
arXiv (Cornell University)·2020· 18 citedOpen Access
Meta-Learning Deep Energy-Based Memory Models
International Conference on Learning Representations·2020· 2 cited
Do Transformers Need Deep Long-Range Memory?
2020· 27 citedOpen Access
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
2020· 118 citedOpen Access
Compressive Transformers for Long-Range Sequence Modelling
arXiv (Cornell University)·2019· 49 citedOpen Access
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
arXiv (Cornell University)·2019· 22 citedOpen Access
Stabilizing Transformers for Reinforcement Learning
arXiv (Cornell University)·2019· 131 citedOpen Access
Meta-Learning Deep Energy-Based Memory Models
arXiv (Cornell University)·2019· 22 citedOpen Access
V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control
arXiv (Cornell University)·2019· 39 citedOpen Access
Developing Deep Learning Continuous Risk Models for Early Adverse Event Prediction in Electronic Health Records: an AKI Case Study
2019· 3 citedOpen Access
page 1 of 2Next →

Frequent Co-authors

Aidan Clark(8), Simon Osindero(8), Timothy Lillicrap(8), Demis Hassabis(7), Siddhant M. Jayakumar(7), Shakir Mohamed(6), Oriol Vinyals(6), Razvan Pascanu(6), Erich Elsen(5), Matthew Botvinick(5), Jordan Hoffmann(4), Sebastian Borgeaud(4), Arthur Mensch(4), Trevor Cai(4), Johannes Welbl(4), Tom Hennigan(4), George van den Driessche(4), Aurelia Guy(4), Karen Simonyan(4), Laurent Sifre(4)