Temporal Straightening for Latent Planning
2026-03-12 • Machine Learning
Machine Learning
AI summaryⓘ
The authors focus on improving how computers learn to plan by teaching them better ways to understand and represent visual information over time. They noticed that typical visual systems include details that aren't helpful for planning and introduced a technique called temporal straightening, which encourages smoother and straighter paths in the internal representation of events. This method helps the computer better measure distances in its 'thinking space,' making planning steps more stable and successful. Their tests showed that this approach leads to better performance in tasks where the system needs to reach specific goals.
representation learninglatent planningworld modelsvisual encodertemporal straighteningcurvature regularizerlatent trajectoriesgeodesic distancegradient-based planninggoal-reaching tasks
Authors
Ying Wang, Oumayma Bounou, Gaoyue Zhou, Randall Balestriero, Tim G. J. Rudner, Yann LeCun, Mengye Ren
Abstract
Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual features, they are not tailored to planning and contain information irrelevant -- or even detrimental -- to planning. Inspired by the perceptual straightening hypothesis in human visual processing, we introduce temporal straightening to improve representation learning for latent planning. Using a curvature regularizer that encourages locally straightened latent trajectories, we jointly learn an encoder and a predictor. We show that reducing curvature this way makes the Euclidean distance in latent space a better proxy for the geodesic distance and improves the conditioning of the planning objective. We demonstrate empirically that temporal straightening makes gradient-based planning more stable and yields significantly higher success rates across a suite of goal-reaching tasks.