Lecture 4 — Backdoors

(a)  The Gaussian mixture, projected onto v1G = N(μG, σ2I) and P = N(μP, σ2I): isotropic Gaussians in ℝ40, 800 samples, shown in a fixed 3-D slice of the cloud whose horizontal axis u is the secret direction. Each point drops onto the line spanned by the top eigenvector v1, giving the 1-D projection below.

gap ‖δ‖ 6.0
spread σ 1.0
poison ε 0.20

(b)  Eigenvalues of ΣDΣD = σ2In + ε(1−ε)δδ⊤ — the poison adds variance in one direction only.

… |⟨v1,u⟩| = – ‖δ‖2 = – ≥ 6σ2/ε = –

(a)  Every input carries a collisionThe architecture of §4 — exactly Lecture 3 §4.3: h(x) = g(H(x)) with H(x) = ⌊Âx⌉γ, a frozen random projection followed by rounding to a coarse grid (here n = 48, m = 12, γ = 4, κ = 0.1). Pick z ← {±1}48 and sample each row of  from N(0, I48) conditioned on |⟨ai, z⟩| ≤ κ√n. Then ‖Âz‖∞ ≤ κ√n, and since Â(x+z) − Âx = Âz, the shift rarely crosses a rounding boundary, and since κ√n < γ/2 it can never cross two: for every input, each rounded coordinate moves by at most one grid step, and most move by none. That is the sense of H(x+z) ≈ H(x): a guaranteed near-collision for all x, exact equality for many (about 40% here) — press ↻ and watch it survive. Three claims make the theorem of §4: the backdoor works — the upper bi-Lipschitz bound on g turns the small gap in H into a small gap in h (this panel); it creates no new collisions — the lower bound means any close pair for h was already close for H, so finding one without z solves CHV, hence CLWE (Lecture 3 §4); and it is statistically hidden (panel b). Think of u as a blind guess at the key: a fresh uniform ±1 vector on every press, the same format and length as z; essentially only the planted z is quiet.

move x by
–

(b)  Spot the backdoorThe hidden claim of §4, played as a game. One matrix is fresh Gaussian; the other was planted around a key. Training g with the first layer frozen cannot increase total-variation distance, so the two full networks are as close as the two matrices — and the lemma bounds that: TV(Â, Gaussian) = Õ(√(m/n)). The proof is a second-moment argument: planting favors matrices with unusually many quiet ±1 directions, so one shows this count concentrates for a Gaussian matrix — candidate keys with small overlap are nearly independent, and the rare high-overlap pairs are controlled separately. The three statistics shown — entry histogram, row lengths, and the singular values that §2’s spectral defense relies on — genuinely match. The game plants at n = 256, m = 8, larger than panel (a)’s teaching-sized 48, because the lemma is asymptotic: at n = 48 a sharp eye can honestly gain about ten points above chance from the slightly narrowed spreads — that is the √(m/n) term made visible. At n = 256 the best simple statistic keeps only a residual edge of a few percent, so your long-run accuracy should hover near 50%. The reveal plots ⟨ai, z⟩/√n for both matrices against the band ±κ: the planted rows all fall inside — a tell visible only to someone who already knows z.

which one hides a key?
–

(a)  The feature weightsRows drawn from N(0, I16), shown along the viewing direction with their marginal below (dashed: the N(0,1) curve). The backdoored sampler conditions ⟨gi, ω⟩ near half-integers — Gaussian pancakes, visible only along the secret ω. The key is sparse: ‖ω‖0 = 4 of 16 coordinates.

weights
view along
accuracy –

(b)  How the network classifies one inputEvery dot is one cosine feature of a single test input, drawn at its phase θi = 2π(⟨gi, x⟩ + bi); its height on the wave is its value φi = cos θi — the horizontal axis is the angle θ fed into the cosine. The key shifts every phase by ≈ (2ki+1)π, half a period, so every value changes sign. The curve below is the running total of the weighted values: it ends at the score F(x), whose sign is the predicted class.

tracked input: ○ before  ·  ● after

(c)  Scores and predictionsThe three attack buttons shift every test input, trying to flip the predictions: the key adds ω, the blind attack adds a random r of the same length, the gradient attack steps x → x−η∇F(x). Shown: F(x) = ⟨w, E(x)⟩ on 110 test inputs; the sign is the prediction, hollow rings the scores before the shift. The key mirrors the whole distribution; a random shift of the same length only scrambles it. The gradient attack walks each score across the boundary using nothing but the public network — no ω, and it works on the honest weights too.

attack
flipped – ‖Δx‖ –