Interference and Retention in Continual Learning
2026-07-10 • Machine Learning
Machine LearningArtificial IntelligenceNeural and Evolutionary Computing
AI summaryⓘ
The authors suggest that forgetting in continual learning happens because new tasks interfere with what was learned before. They explain this interference mathematically and use it to design a method called Interference-Gated Functional Allocation (IGFA), which smartly shares or protects parts of the model depending on whether tasks overlap or conflict. IGFA does not need to replay old data or use complex regularization and can avoid forgetting when tasks are clearly separate. Their approach also balances learning new tasks and retaining old ones better than some existing methods.
Continual learningForgettingTask interferenceFrozen-feature regimePath-averaged curvatureReplay-free methodsElastic regularizationOrthogonalizationFisher informationPlasticity
Authors
Julius Störk
Abstract
Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.