PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball
2026-07-30 • Robotics
RoboticsArtificial Intelligence
AI summaryⓘ
The authors introduce PAC-MAN, a method that helps a humanoid robot dodge balls by using safety controls and realistic camera sensing during action. During training, the robot uses detailed information about its body to avoid the ball, but during actual use, it only sees the ball through a camera that highlights it in depth images. They tested their approach in controlled scenarios and found that the robot can dodge well with just a fixed camera. The study also shows that better sensing improves safety control performance. Finally, they successfully tried their method on a real robot, which dodged balls accurately even with imperfect vision.
Control Barrier FunctionsReinforcement LearningPerception-aware ControlHumanoid RobotSemantic SegmentationDepth SensingMotion PriorSafety-Critical ControlAdversarial TrainingRobot Dodgeball
Authors
Lizhi Yang, Junheng Li, Aaron D. Ames
Abstract
We present PAC-MAN, a perception-aware CBF-RL framework that couples control-barrier safety with deployment-realistic onboard sensing for whole-body humanoid dodgeball. The deployed policy sees the ball only as segmentation-masked depth from a head-mounted camera, while training-time CBF guidance represents clearance to every body link, and an adversarial motion prior regularizes the resulting evasive reflexes. We evaluate on a controlled any-link contact benchmark with seeded throws in two regimes: single throws and a deployment loop in which the robot walks back to its station and recovers between throws. On this benchmark, the policy comes within a few points of a privileged state oracle: a fixed onboard camera alone is adequate for evasion. We find that usable barrier structure depends on perceptual observability: Joint-CBF gives the best performance with accurate ball states, degrades under fixed-camera observations when used only as training guidance, and recovers with a ball-tracking gimbal or privileged runtime filter. We therefore deploy a lightweight Link-CBF policy zero-shot on the Unitree G1 in the real world, where it tolerates imperfect perception, succeeds on 95% of throws, and uses semantic segmentation to dodge different balls.