A Mixed-Reality Testbed for Autonomous Vehicles
2026-06-17 • Robotics
Robotics
AI summaryⓘ
The authors created a testing setup that mixes real small robots with computer simulations to study self-driving cars. This setup allows them to test how well the cars can see, plan, and control themselves in different tricky driving situations. They also include wireless communication between cars and can test many vehicles together, both real and virtual. The authors added a safety method that helps cars drive safely by learning and adjusting in real-time. They showed that their system helps connect simulations with real-world robot testing effectively.
mixed realityhardware-in-the-loop (HIL)autonomous vehiclesperceptionplanningcontrolmulti-agent systemsConnected and Autonomous Vehicles (CAVs)Control Barrier Functions (CBFs)online learning
Authors
H. M. Sabbir Ahmad, Ehsan Sabouni, Emrullah Celik, Zean Wan, Damola Ajeyemi, Christos G. Cassandras, Wenchao Li
Abstract
We propose a mixed-reality, hardware-in-the-loop (HIL) testbed for autonomous vehicles that seamlessly integrates a physical testbed of mobile robots with a high-fidelity simulation environment. The virtual simulation enables the creation of diverse, safety-critical driving scenarios to validate state-of-the-art perception, planning, and control algorithms, while augmenting simulations with physical robots equipped with multimodal sensors in photorealistic virtual environments further facilitating rigorous validation. Our testbed also features vehicular connectivity using wireless communication and can accommodate a large number of agents through the combination of physical robots and virtual simulated agents, supporting research on multi-agent systems including Connected and Autonomous Vehicles (CAVs). Finally, we present a safety-guaranteed framework combining perception, planning and a novel online learning-based controller using Control Barrier Functions (CBFs) for CAVs. Experiments using the proposed framework are used to validate and demonstrate the key functionalities and the overall utility of the testbed to bridge the gap between simulation and real-world hardware deployment.