How Language Models Work

Lesson 01

Next-token prediction

It’s guessing the next word. That’s the whole trick.

A language model doesn’t look up answers. It reads what’s there, guesses the next little chunk, and repeats. That’s the entire product.

Pick any one. The unlikely ones are the interesting ones.

Prompt