SPARCL: Spectral Partitioned Analytic Continual Learning
2026-08-21 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study a way to learn new classes without forgetting old ones using analytic continual learning, which updates models with a formula instead of retraining. They find that a problem called spectral interference causes the model's knowledge of old classes to drift, even without revisiting old data. To fix this, they introduce SPARCL, which separates and protects important parts of the model related to old classes while only updating less critical parts for new classes. This approach keeps old knowledge stable and improves performance on several image classification benchmarks compared to previous analytic methods.
analytic continual learningridge regressionspectral interferencerecursive least squaresautocorrelation operatoreigendirectionsViT-B/16class-incremental learningfeature decorrelationSPARCL
Authors
James Hartley, Zeropy Surio, Daniel Whitmore, Hannah Clarke, Thomas Reed
Abstract
Analytic continual learning has emerged as a strong exemplar-free alternative to gradient-based class-incremental learning because it replaces iterative optimization with closed-form ridge updates. Yet the usual forgetting narrative, centered on stochastic gradient overwriting, does not explain why analytic methods still drift on old classes despite exact recursive solvers. We identify the culprit as spectral interference: the joint ridge classifier for all tasks shares the inverse autocorrelation operator $(R+λI)^{-1}$, so incoming task samples that load onto old dominant eigendirections dilute the spectrum and perturb old-class logits even when old labels are never revisited. Based on this view, we propose SPARCL, a spectral partitioned analytic continual learner that decomposes the running autocorrelation into a high-energy core and a residual complement, freezes old-class classifier components in the core subspace, and updates only the residual block through recursive least squares with an optional residual random-projection expansion. This yields a simple closed-form update with a provable invariance guarantee for the core contribution of old logits. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A under a frozen ViT-B/16 protocol, SPARCL closes most of the gap from classical analytic learners to strong representation matchers, while remaining complementary to sparse feature-decorrelation approaches such as Fly-CL.