Beyond the Chatbot
One Layer Deeper
Cover
Contents
Notes
Practicals
Builds
Margins
Back pages
About
Notes
The notes written so far.
In order. The full plan, including notes not yet written, is in the
contents
.
NOTE 00
00
Why this notebook exists
Opening · what this notebook is for, and how it is kept.
01
A model doesn't know things. It predicts what comes next.
Section 1 · nothing is looked up; every answer is built one word at a time. With
Practical 01
.
02
Tokens: why the model reads "strawberry" differently from you
Section 1 · the model never sees letters, only numbered pieces; why that explains counting, arithmetic, cost and languages. With
Practical 02
.
03
Why the same question gives different answers: temperature and sampling
Section 1 · the dial on the draw, and why it only matters when a question has more than one good answer. With
Practical 03
.
04
The model is a snapshot: training cutoffs
Section 1 · the model has read nothing after one date; the only way in for anything newer is the text in front of it. With
Practical 04
.