FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space
2026-07-09 • Robotics
RoboticsMachine Learning
AI summaryⓘ
The authors developed FlowDAgger, a method to quickly adapt robot policies that were pretrained but may fail in new real-world situations. Instead of retraining everything, their method uses human corrections to adjust the robot’s behavior by figuring out what underlying 'noise' caused the wrong action and then steering the robot accordingly. This approach works with few examples and less computing power, preserving the original skills while improving performance on new tasks. They tested FlowDAgger in simulations and real robots, finding it better than other adaptation methods.
pretrained generative robot policiesflow matchingdiffusion modelslatent spaceaction inversionreverse-time integrationrobot manipulationreinforcement learningfine-tuningrobot foundation models
Authors
Michael Murray, Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Galen Mullins, Harshavardhan Gajarla, Oier Mees, Maya Cakmak, Andrey Kolobov
Abstract
Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real-world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large-scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample- and compute-efficient method for adapting frozen generative robot policies from human interventions in latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse-time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation, adapting both action-head VLAs and world-action models from a handful of interventions. FlowDAgger outperforms supervised fine-tuning and latent-space RL baselines and preserves pretrained skills on held-out tasks, offering a practical path for adapting robot foundation models in the real world. Website: https://microsoft.github.io/FlowDAgger