INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models
2026-07-28 • Robotics
Robotics
AI summaryⓘ
The authors present INTACT, a model that learns to directly convert a desired change (intent) in a scene into the actions needed to achieve it, without costly trial-and-error during deployment. Their approach links physical state transitions and future goals within a unified framework, using shared representations that avoid complex latent matching or assumptions about dynamics. This allows quick, efficient action prediction from visual inputs, giving strong performance on several control tasks with minimal training and much faster inference than typical search methods. INTACT can also improve action selection by sampling fewer candidates more effectively, making it both accurate and computationally efficient.
Forward latent world modelsIntent-to-actionRepresentation learningControl policyLatent spaceDirect inferenceContrastive learningAction-law semanticsTrajectory modelingLeWM tasks
Authors
Junhan Sun, Hao Zhao, Guofeng Zhang
Abstract
Forward latent world models predict how actions change a scene, but recover actions for a desired change only through expensive test-time search. We introduce INTACT (INtent-To-ACTion), an end-to-end JEPA that turns action-labeled, reward-free trajectories into a deployable intent-to-action interface. Each transition supplies physical intent $z_{t+1}-z_t$, while a future goal supplies deployment intent $\operatorname{sg}(z_g)-z_t$. The architecture is isomorphic between the local and goal motion-intent backbone-input graphs through an identical four-slot grammar and shared parameters, and between supported local and goal motion-intent families through action-law semantics induced by the same predictor rather than pointwise latent equality. INTACT also provides intact transfer from RGB evidence to action-effective latent intent coordinates and from intent families to their corresponding action-law families. Asymmetric endpoint gradients ground physical successors and fix future goals as anchors, joining representation learning and control without pointwise latent matching or globally linear dynamics. The resulting coordinates support a robust distributional action law: its conditional mean serves directly as a search-free policy, while sampling remains available for diversity or optional verification. On the four official LeWM tasks, one-epoch, zero-search models reach 85.78\%, 100.00\%, 97.67\%, and 97.89\% success. Optional local CEM centered on the Direct plan reaches 96.86\% macro success using 384 instead of 9,000 candidate sequences, reducing sampling by $23.44\times$ while improving pure CEM by 16.00 points. One shared four-task encoder reaches 89.39\% E5 Direct macro and improves every task over jointly trained LeWM, while predicted--expert action-family kNN tracks Direct success at $r=0.954$. Direct inference takes 2.9--5.5 ms.