TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
2026-06-09 • Robotics
Robotics
AI summaryⓘ
The authors developed TacForeSight, a system that helps robots better handle tasks where they need to touch and manipulate objects carefully. Their approach uses both big-picture force sensing from the wrist and detailed touch sensing from fingers to predict how touch sensations will change in the near future. This helps the robot plan and react quickly during manipulation, especially in tricky situations where contact changes or objects move. Tests on real robots showed their method works better than previous ones, particularly when things get disrupted unexpectedly.
contact-rich manipulationimitation learningtactile sensingforce feedbacklatent dynamicsworld modelcross-attentionvisuo-tactile fusionreal-time controlrobotic manipulation
Authors
Yujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang, Shuicheng Yan, Wenchao Ding
Abstract
Contact-rich manipulation requires robots to continuously perceive and regulate evolving physical interactions under dynamic contact transitions or complex surface geometries. Recent imitation learning methods improve contact-aware control by incorporating tactile or force feedback, but they rarely model the asymmetric spatiotemporal roles of global force and local tactile sensing. To address this, we propose TacForeSight, a lightweight force-conditioned tactile foresight framework for real-time manipulation. The core component is TacForceWM, a tactile world model that predicts short-horizon tactile latent dynamics from dual-finger tactile observations conditioned on high-frequency wrist force and torque signals. Another key component, the Predictive Tactile-Conditioned Policy, leverages the predicted latents as anticipatory contact priors, models the current-to-future tactile evolution via cross-attention, and adaptively fuses visuo-tactile features through a tactile-guided gating module. By forecasting purely within a compact latent space, TacForeSight enables proactive contact reasoning with efficient real-time inference suitable for high-frequency manipulation control. Real-robot experiments on five representative tasks and three in-process perturbation settings show that TacForeSight consistently outperforms existing baselines, particularly under dynamic contact disturbances. All models and datasets will be made publicly available on the project website at https://tacforesight.github.io/ProjectPage.