Geometric Action Model for Robot Policy Learning
2026-06-15 • Robotics
RoboticsComputer Vision and Pattern RecognitionMachine Learning
AI summaryⓘ
The authors present a new robot control method called the Geometric Action Model (GAM) that helps robots understand and interact with the 3D world based on user instructions. Unlike previous models that mainly used flat 2D images, GAM uses a geometric foundation model to directly process 3D shapes and predict future movements and actions. They split this model to both interpret current observations and forecast future states, all while considering language commands and past actions. Their approach improves the robot’s accuracy and speed in tasks requiring physical manipulation, shown in both simulations and real robots.
Geometric Foundation ModelVision-Language-Action Models3D Physical WorldTemporal PredictionManipulation PolicyProprioceptionAction DecodingContact-Rich ManipulationLatent TokensSimulation Benchmarks
Authors
Jisang Han, Seonghu Jeon, Jaewoo Jung, René Zurbrügg, Honggyu An, Tifanny Portela, Marco Hutter, Marc Pollefeys, Seungryong Kim, Sunghwan Hong
Abstract
Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models, but they still operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation. We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real-robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines.