Lesson 14
The assistant act
It couldn’t answer you at first. That part was trained in afterwards.
A model fresh out of training does not answer questions. It continues text, like autocomplete that has read most of the internet. Being a helpful assistant is a second, separate round of training added on top, called fine-tuning.
1Pick a message to send it
Do thisFlip between the two buttons below. Same machine, same knowledge. The only difference is a round of training that came afterwards.
2The same model, before and after that training
3How that second round of training worked
Step 1 · copy the shape
Show it tens of thousands of example conversations, so “an answer” becomes the likely thing to write next.
Q: How do I boil an egg?
A: Place the egg in boiling water for 7–9 minutes…
Q: Summarize this paragraph.
A: Here's the key point in two sentences…
Q: Write a limerick about soup.
A: There once was a chowder so thick…
Step 2 · let people pick a winner
Give it two tries at the same question, let real people choose the better one, and train it toward the winners.
“I'd check the oven temperature first. Uneven browning usually means it runs hot.”