Geometric regularization of autoencoders via observed stochastic dynamics
2026-04-17 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study complex systems that slowly change within a hidden low-dimensional space, even though they appear in high-dimensional data. They propose a new method to learn a simplified model using penalties based on how data spreads out, capturing important geometric features better than previous methods. This approach improves the accuracy of predicting system behavior, such as how long it takes to move between states, tested on various examples with up to 201 dimensions. Their method reduces errors significantly compared to existing autoencoder techniques by ensuring the learned coordinates align well with the system's true structure.
Stochastic dynamical systemsLow-dimensional manifoldLatent stochastic differential equationAutoencoderTangent bundleItô's formulaMean first-passage timeSobolev normMetastabilityMüller–Brown potential
Authors
Sean Hill, Felix X. -F. Ye
Abstract
Stochastic dynamical systems with slow or metastable behavior evolve, on long time scales, on an unknown low-dimensional manifold in high-dimensional ambient space. Building a reduced simulator from short-burst ambient ensembles is a long-standing problem: local-chart methods like ATLAS suffer from exponential landmark scaling and per-step reprojection, while autoencoder alternatives leave tangent-bundle geometry poorly constrained, and the errors propagate into the learned drift and diffusion. We observe that the ambient covariance~$Λ$ already encodes coordinate-invariant tangent-space information, its range spanning the tangent bundle. Using this, we construct a tangent-bundle penalty and an inverse-consistency penalty for a three-stage pipeline (chart learning, latent drift, latent diffusion) that learns a single nonlinear chart and the latent SDE. The penalties induce a function-space metric, the $ρ$-metric, strictly weaker than the Sobolev $H^1$ norm yet achieving the same chart-quality generalization rate up to logarithmic factors. For the drift, we derive an encoder-pullback target via Itô's formula on the learned encoder and prove a bias decomposition showing the standard decoder-side formula carries systematic error for any imperfect chart. Under a $W^{2,\infty}$ chart-convergence assumption, chart-level error propagates controllably to weak convergence of the ambient dynamics and to convergence of radial mean first-passage times. Experiments on four surfaces embedded in up to $201$ ambient dimensions reduce radial MFPT error by $50$--$70\%$ under rotation dynamics and achieve the lowest inter-well MFPT error on most surface--transition pairs under metastable Müller--Brown Langevin dynamics, while reducing end-to-end ambient coefficient errors by up to an order of magnitude relative to an unregularized autoencoder.