SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation
2026-07-09 • Software Engineering
Software EngineeringArtificial Intelligence
AI summaryⓘ
The authors explain that autonomous coding agents often stop their work too soon, missing important parts of the code that need testing. To fix this, they created SCATE, a system that replaces humans by smartly deciding what tests to generate next, learning from past results. Their tests show SCATE helps coding agents cover more lines and branches of code than when working alone or with other methods. This means SCATE makes automated test generation more thorough and efficient without needing a person watching all the time.
autonomous coding agentsautomated test generationlazy generationcode coveragehuman-in-the-loopcontextual bandit problemline coveragebranch coverageadaptive supervisionsoftware testing
Authors
Sijia Gu, Noor Nashid, Ali Mesbah
Abstract
While autonomous coding agents have significantly advanced automated test generation, they remain fundamentally limited by lazy generation, a phenomenon where agents prematurely terminate tasks and systematically avoid complex programmatic logic, resulting in inadequate code coverage. Currently, mitigating this premature termination requires continuous human-in-the-loop supervision. This heavy reliance on human intuition creates a bottleneck that negates the efficiency gains of automated generation. We propose SCATE, a framework for adaptive, automated supervision of coding agents that replaces human intervention during test generation. By formulating supervision as a contextual bandit problem, SCATE learns to select the most promising testing actions based on the current coverage and class testability metrics, maximizing coverage gains while minimizing wasted generation effort. Our empirical evaluation demonstrates that SCATE integrates seamlessly with different coding agents. When applied to GEMINI-CLI, it achieves 32.3% higher line coverage and 30.9% higher branch coverage than the agent-only baseline. A comparison with CLAUDE CODE confirms the framework dynamically adapts its policy to optimize each agent's unique strengths. SCATE also consistently outperforms state-of-the-art non-agentic approaches across all metrics.