GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
2026-07-06 • Robotics
RoboticsArtificial IntelligenceComputation and LanguageMachine Learning
AI summaryⓘ
The authors explore a way to make robots work better in jobs where objects change shape and position a lot, called Variational Automation. They created a system named Graph-as-Policy (GaP), which connects different robot skills like seeing, planning, and moving into a graph that the robot uses to decide what to do. GaP practices tasks in a simulated environment to improve its plans before trying them in real life. When tested on new tasks, GaP did better than some existing methods. This approach aims to make robots more reliable and adaptable in real-world industrial settings.
Variational AutomationModel-free policiesTask and Motion Planning (TAMP)Robot Operating System (ROS)Graph-as-Policy (GaP)Directed computation graphsModular Open Robot Skill Library (MORSL)Simulation environmentPerceptionRobot control
Authors
Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini, Sandeep Bajamahal, Hudson Kim, Jaimyn Drake, Daehwa Kim, Haoru Xue, Jonathan Francis, Christian Juette, Peter Schaldenbrand, Muhammet Yunus Seker, Ruwan Wickramarachchi, Uksang Yoo, Guanzhi Wang, Adithyavairavan Murali, Balakumar Sundaralingam, S. Shankar Sastry, Spencer Huang, Yuke Zhu, Linxi "Jim" Fan, Ken Goldberg
Abstract
For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies? We focus on "Variational Automation" (VA), a class of tasks that have larger variations in object geometry and pose than fixed automation. Model-free policies often struggle to close the reliability gap for VA tasks, which must be executed persistently and reliably in commercial and industrial applications. Motivated by prior work on Task and Motion Planning (TAMP) and the Robot Operating System (ROS), we introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs with perception, planning, and control nodes from a Modular Open Robot Skill Library (MORSL). GaP then generates an internal simulation environment to rehearse task instances with different graphs in parallel to iteratively refine the graph structure and parameters to improve success rates and throughput. Evaluation with 8 new open VA task benchmarks, 4 in-simulation and 4 in real-world, suggests that GaP can achieve success rates that significantly outperform baselines. Details, code, and data can be found online: https://graph-robots.github.io/gap