01
Scaling2020
Scaling Laws for Neural Language Models
Kaplan, McCandlish, et al. — OpenAI
Why bigger kept winning — until it didn’t. Essential background for any budget conversation about training vs. buying.
Original paper ↗
Summary
Read the papers. Skip the jargon.
The white papers that actually moved the field, each linked to the original and paired with my plain-language summary, a glossary of the terms, and what it changes in practice.
02
Scaling2022
Training Compute-Optimal Large Language Models
Hoffmann, Borgeaud, et al. — DeepMind
The “Chinchilla” correction: most large models were undertrained on data. Changed how every lab spends its compute.
Original paper ↗
Summary · Glossary
All original papers remain the work of their authors and link to the official source (arXiv or publisher). Summaries and glossaries on this site are my own commentary.