Lecture 7.5 — Low Logit Rank

(a)  sampling the matrixEight histories are sampled from the model (one row each); futures are the empty string and five sampled continuations (one block each). Every cell is a centered logit ℓ̅(v | hf): gold positive, blue negative. The full matrix would have |V| columns per block; as in the lecture’s sampling, only the strongest token columns of each block are kept — here the 8 of 29 with the largest entries. Press play to fill the matrix block by block, and hover any cell to read it. The first block is the ordinary next-token case; every other block appends the same future to all eight histories at once.

–

(b)  the singular values, and the kneeThe bars are the singular values of the sampled matrix, on a log scale. Four stand tall and the rest collapse: the hidden state is 3-dimensional and propagates linearly, so the exact part of the matrix has rank d+1 = 4, and the curved readout contributes only the small tail. Slide r to truncate: the chip reports the captured energy Σi≤rσi² / Σσi² and the discarded residual. Then enlarge the sample: twice the histories, twice the futures — the spectrum rescales, the knee stays at four, and the rank needed for a fixed accuracy does not grow. That persistence is the empirical observation.

truncation r4
–

§2 — generating from a history the model is never asked aboutThe gold history is the target; it is never queried. Six references were queried over a calibration set of futures, and least squares found coefficients c₁,…,c₆ making the target row their combination — once on the extended matrix, once on next-token logits alone. Each press appends the growing prefix g to the six references only, combines their logit replies with the same fixed coefficients, and samples the next token from the extended combination. The bars compare this step’s three distributions over the most likely tokens; the trace shows total-variation distance to the truth at every step. The extended relation persists — even with one reference made of nonsense tokens — while the one-step relation, fitted with no control over its coefficients beyond the first block, drifts and stays off.

–