How Language Models Work

Lesson 13

Scale

The method never changed. It just got enormous.

There is no new idea between a tiny model and the biggest one on earth. Same loop, same training, just far more text and far more internal settings. What changes is what starts working without anyone asking for it. People call that emergence.

A whale folded from indigo paper, tail fluke raised.
The same design, made enormous. Nobody added the new abilities on the big one; past a certain size they simply turned up.

1Pick a model size

2Pick something to ask it

Do thisLeave the question the same and click up through the four sizes in step 1. Nothing about the method changes, only how much of it there is.

3What it writes back

4What it has to work with