Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training
2026-08-19 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied how removing or adding a single training example affects a GPT-2 model by running full training runs with small changes in the data. They injected a specific 194-token passage into the training data and found the model learned it briefly but the impact faded over time and was undetectable by the end of training. Various measurements showed that the model adjusted within its existing solution area without making big changes. This suggests single examples influence the model in subtle ways that don't permanently shift the overall training outcome. The study provides a rare direct measurement of a single example's effect rather than an estimate.
GPT-2cross-entropykernel alignmenttraining injectiongradient deltaloss barrierweight displacementOpenWebTextfull training runscounterfactual
Authors
Zachary Speck, Asa Shepard
Abstract
A single training example's contribution to a finished model is normally estimated rather than measured, because measuring it takes two expensive full pre-training runs that differ in one row of one batch. We ran that counterfactual 24 times at a small scale. We trained 32 GPT-2 models at 124M parameters from scratch on OpenWebText, over four conditions and eight seeds. At step 200 of 9,536, at peak learning rate, we replaced one row of a 256-row batch with a fixed context injection carrying a 194-token passage. The three injected conditions are: 1. fluent prose with a corpus-attested subject, 2. fluent prose with a fabricated subject matched to it within 0.14% on full-batch gradient delta, and 3. random keyboard characters. The fourth condition is an uninjected twin. The passage is learned from one exposure and then decays. Fifty steps after injection, the arm that saw a passage predicts it better than the arm that did not by 0.039 and 0.044 nats of cross-entropy on the passage, at eight of eight seeds with p < $10^{-4}$. At the final step we do not detect that difference for either passage, at p = 0.25 and p = 0.71, against minimum detectable effects of 0.025 and 0.079 nats, nor between the two passages, at p=0.54. Every geometric measure we report is taken after that decay. Our pre-registered contrast on interpolation loss barrier is +0.0068 with p = 0.509, against a minimum detectable effect of 0.032 barrier units. Held-out cross-entropy is $-0.00044$ with p = 0.310. Per-layer centered kernel alignment does not detectably separate any condition at any layer. Weight displacement reaches 44.1% of the seed-to-seed Euclidean distance and is 92% settled by the midpoint of training, while the barrier reaches 3.0% of the seed-to-seed barrier. Those two figures sit roughly 15 times apart, and that is a lower bound. The injection relocates the model within its basin without moving it out.