InSight: Self-Guided Skill Acquisition via Steerable VLAs
2026-06-23 • Robotics
RoboticsArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors developed InSight, a system that helps robots learn new actions on their own by breaking down complex tasks into simple steps, like moving or lifting objects. First, their method automatically splits demonstrations into basic actions with labels. Then, it figures out which actions are missing for a new task, tries to perform those actions by itself, and adds successful attempts to its training data. This allows robots to learn and combine these basic skills to do new and longer tasks without extra human help.
vision-language-action modelsprimitive actionsskill acquisitiondemonstration segmentationend-effector poseslow-level controlrobot manipulationself-supervised learninglong-horizon tasks
Authors
Maggie Wang, Lars Osterberg, Stephen Tian, Ola Shorinwa, Jiajun Wu, Mac Schwager
Abstract
Vision-language-action (VLA) models can learn manipulation skills from demonstrations, but their capabilities are bounded by the skills in the training data. We present InSight, a framework that unlocks autonomous skill acquisition by rendering VLAs steerable at the primitive-action level (e.g., "move gripper to the bowl", "lift upward", "pour the bottle"). InSight consists of two primary stages: (1) an automated segmentation pipeline that partitions demonstrations into labeled primitives via VLM plan decomposition and end-effector poses to enable VLA primitive steerability, and (2) a VLM-guided data flywheel that identifies missing primitives required to accomplish a novel task, autonomously attempts demonstrations of the missing primitives with VLM-proposed low-level control, and automatically labels, stores, and integrates successful demonstrations into the VLA training set. We evaluate InSight across simulation and real-world manipulation tasks, including block flipping, drawer closing, sweeping, twisting, and pouring, without any human demonstrations of these target skills. Once learned, these primitives can be composed to execute novel, long-horizon tasks without additional human demonstrations. Our findings demonstrate that primitive steerability provides a practical foundation for continual skill acquisition in VLA policies. Project website: https://insight-vla.github.io.