Lecture 1 — Cryptography and Learning

(a)  A learner is a distinguisherThe reduction behind Theorem 1, run for real on a class that is learnable: linear functions fs(x) = ⟨x,s⟩ over 𝔽28. The distinguisher A wraps the learner L: it forwards L's sample requests and queries to the oracle, returns the oracle's answers, and finally tests L's hypothesis on one fresh point, accepting on agreement. Against Fs the learner's view is exactly its PAC experiment, so it predicts and the test passes; against a truly random function nothing generalizes. The gap between the two acceptance probabilities is the distinguishing advantage — so a learnable class cannot contain a PRF, which is the theorem read backwards. The general bound is (1−δ)(1−ε) − 2−ℓ − m/2n. Kerckhoffs' principle applies throughout: the family and algorithms are public, only s is secret. Reference: Kearns–Valiant, JACM 1994.

(b)  The GGM tree: a function from a seedFrom a length-doubling generator G(s) = G0(s)‖G1(s), define Fs(x1⋯xn) = Gxn(⋯Gx1(s)⋯): the seed sits at the root, every node has a 0-child on the left and a 1-child on the right, and the input spells a root-to-leaf path. Toggle the four bits of x and the gold path re-routes; the values written along it are the only ones ever computed — the rest of the tree exists implicitly. One 32-bit seed determines all 24 leaves here, and in general k bits determine 2n pseudorandom values — a distinguisher for the family would break the generator. The generator used on this page is a toy integer mixer, standing in for a real G. Reference: Goldreich–Goldwasser–Micali, JACM 1986.

input x

(a)  One threshold cannot do XORThe four inputs of XOR, colored by their class. A single threshold unit 𝟙[w1x1+w2x2 ≥ θ] is a line: drag its two handles anywhere and the best it achieves is 3 of 4, because no half-plane contains exactly the two odd points — Minsky and Papert's observation, and the reason linear algebra over 𝔽2 is not linear computation over ℝ. One hidden layer repairs it: with two threshold units the class-1 region is the band between two parallel lines, and all four points are classified. This is a restriction of a single layer, not of networks. Reference: Minsky–Papert, Perceptrons, 1969.

(b)  A net is a circuit: depth and widthA feedforward network drawn as the circuit it is: units and finite-precision operations give size, layers give depth, and with threshold activations a fixed-depth polynomial-size network is a TC0 circuit. Slide the width up and the circuit grows without getting deeper — that is the TC0 regime, and under standard assumptions it already contains pseudorandom functions (Naor–Reingold; Banerjee–Peikert–Rosen), so by Theorem 1 constant-depth threshold networks are not efficiently PAC learnable, and black-box distillation of them is just as hard. Deployed models are not far off: a bounded-precision fixed-depth transformer sits in uniform TC0, with each block expanding to a constant-depth threshold subcircuit — so a deployed model's few dozen blocks mean constant-scale depth, while its billions of parameters are all width. The autoregressive view unrolls this: each pass emits a token that is fed into the next pass, so generating T tokens stacks T constant-depth blocks — total depth grows with T, and if T grows with the input length the computation need not stay in TC0. References: Naor–Reingold, JACM 2004; Banerjee–Peikert–Rosen, EUROCRYPT 2012; Chiang 2024.

mode
depth d 6
width 10
โ€“

(a)  Parity with noise: As + e = b over 𝔽2The equation drawn as shapes: A is uniform over 𝔽2, the secret s is a bit vector, and the error e flips each label independently with probability η — the sparse carmine cells. At η = 0 Gaussian elimination over 𝔽2 recovers s exactly. With noise, each solved bit is a sum of many labels — a typical row of A−1 is dense — and t combined labels carry bias (1−2η)t toward the truth, exponentially small in t. The verdict line reports the actual average t of this instance and the resulting bias, and the recovered strip shows the damage. Guessing the corrupted equations is no better: about 2mH2(η) error patterns at constant noise.

noise rate η 0.10