AutoRestTest at the SBFT 2026 Tool Competition
2026-07-01 • Software Engineering
Software Engineering
AI summaryⓘ
The authors developed AutoRestTest, a tool that helps test complex REST APIs by smartly exploring many possible inputs. They use a method that understands how different parts of the API depend on each other, combined with AI techniques like reinforcement learning and large language models. In a major competition, AutoRestTest performed the best in finding bugs, working efficiently, and overall effectiveness on 11 different APIs. It found many unique server errors and processed many operations correctly within a limited testing time.
REST APIblack-box testingSemantic Property Dependency Graphmulti-agent reinforcement learninglarge language modelsfault detectionAPI input spacesoftware testingserver errors
Authors
Tyler Stennett, Myeongsoo Kim, Saurabh Sinha, Alessandro Orso
Abstract
Large input spaces and complex inter-operation dependencies make black-box REST API testing challenging. AutoRestTest combines a Semantic Property Dependency Graph, multi-agent reinforcement learning, and large language models to intelligently explore large API input spaces. In the SBFT 2026 REST League, AutoRestTest ranked first in all three evaluation categories -- fault detection, overall efficiency, and overall effectiveness -- on 11 APIs (317 operations, approximately 29 per API), averaging 67.09 unique server errors and 17.27 successfully processed operations per API under a one-hour testing budget.