Paper Reading Lab
Read the paper, not the press release
Five real open-access papers, each read in the same eight passes. Write your own answer first, then compare it with the guide’s reading. You do not need to read the whole paper: each step tells you where to look.
- 1 Research question
- 2 Actual claims
- 3 Definitions & variables
- 4 Key equation
- 5 Strongest evidence
- 6 Limitations
- 7 Open questions
- 8 Cross-disciplinary relevance
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei
It is the paper behind "scaling laws", and its central equations are a single power law you can read with Module 1. It is also a physicist’s paper: it hunts for the few variables that govern behaviour and then fits simple laws to them.
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin
The architecture behind almost every current language model. Its key equation is one line, and every symbol in it can be decoded with the Rosetta Stone.
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, and 46 others (Anthropic)
Its "equation" is a procedure. Reading it carefully shows exactly why the tempting AdS/CFT analogy in Module 7 cannot be a mathematical duality.
Are Emergent Abilities of Large Language Models a Mirage?
Rylan Schaeffer, Brando Miranda, Sanmi Koyejo
A model of scientific skepticism. It does not say the earlier observations were fabricated; it shows that a measurement choice can manufacture a dramatic-looking pattern.
Dynamic Chunking for End-to-End Hierarchical Sequence Modeling (H-Net)
Sukjun Hwang, Brandon Wang, Albert Gu
Recent work on learned tokenization. Its key equation reuses a dot product and cosine similarity you already know from attention, now used to decide where "chunks" begin.