1
00:00:01,500 --> 00:00:03,620
May 28, 2020.

2
00:00:03,620 --> 00:00:05,220
The world is locked down.

3
00:00:05,220 --> 00:00:08,050
A seventy-two-page paper appears on arXiv.

4
00:00:09,750 --> 00:00:11,840
Until then, AI was a craft.

5
00:00:11,840 --> 00:00:13,310
One task, one model,

6
00:00:13,310 --> 00:00:15,510
thousands of labeled examples.

7
00:00:15,510 --> 00:00:16,810
Every single time.

8
00:00:18,750 --> 00:00:21,620
A small team at OpenAI made a different bet:

9
00:00:21,620 --> 00:00:24,430
that performance climbs a smooth power law

10
00:00:24,430 --> 00:00:27,170
as you add compute, data, and parameters.

11
00:00:32,250 --> 00:00:36,450
GPT-3 had 175 billion parameters.

12
00:00:36,450 --> 00:00:38,560
Ten times larger than any dense

13
00:00:38,560 --> 00:00:40,260
language model before it.

14
00:00:43,500 --> 00:00:45,420
Then, the ghost in the machine.

15
00:00:45,420 --> 00:00:48,000
Show it a few examples in the prompt,

16
00:00:48,000 --> 00:00:50,010
and it continues the pattern.

17
00:00:50,010 --> 00:00:50,880
No retraining.

18
00:00:50,880 --> 00:00:53,110
They called it in-context learning.

19
00:00:59,250 --> 00:01:00,930
Skills appeared with scale.

20
00:01:00,930 --> 00:01:02,670
Flawless two-digit addition.

21
00:01:02,670 --> 00:01:04,940
News articles readers caught

22
00:01:04,940 --> 00:01:06,650
only 52% of the time.

23
00:01:10,500 --> 00:01:12,020
It shipped as an API.

24
00:01:12,020 --> 00:01:13,510
Several of its authors

25
00:01:13,510 --> 00:01:15,130
left to found Anthropic.

26
00:01:15,130 --> 00:01:16,570
And two years later,

27
00:01:16,570 --> 00:01:18,440
the recipe became ChatGPT.

28
00:01:21,750 --> 00:01:23,400
This is the story of the paper

29
00:01:23,400 --> 00:01:24,990
that started the Scaling Era,

30
00:01:24,990 --> 00:01:26,740
told by the people who built it.