StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments
2026-08-25 • Artificial Intelligence
Artificial IntelligenceSoftware Engineering
AI summaryⓘ
The authors introduce StarHarness, a system that improves how AI agents work in specific environments without changing the AI's core model. Instead, it evolves a 'harness' that includes things like prompts, tools, and task setups to make the agent perform better. When tested on several benchmarks, this approach boosted performance by 20-35% and worked well even on new tasks and different AI models. The improvements come from better interfaces and understanding of the environment, helping the agent make fewer mistakes and work more efficiently. Overall, the authors show that evolving this harness is a useful way to fix mismatches between AI models and the environments they operate in.
Agent harnessPrompt engineeringTask framingTool interfacesEvolutionary algorithmGeneralizationGPTModel-environment mismatchBenchmarkingOperational knowledge
Authors
Esakkivel Esakkiraja, Denis Akhiyarov, Vikas Yadav, Sai Rajeswar, Patrice Bechard, Sridhar Nemala, Sagar Davasam
Abstract
We present StarHarness, a framework for evolving environment-specific agent harnesses while keeping model weights fixed. The evolved harness can include prompt and task framing, tool interfaces, skills, MCP-backed providers, subagent structure, and agent-loop configuration. StarHarness constructs a compact evolution pool by stratifying tasks according to baseline failure behavior, separates proposer-visible search tasks from proposer-hidden selection tasks, and reserves held-out tasks for evaluating generalization. Across ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance, harness evolution improves full-benchmark performance by 20-35 percentage points over the default harness after 4-12 accepted changes per environment. These gains persist on tasks excluded from evolution and transfer without re-evolution across GPT and Qwen model families. Trace analysis links the improvements to interface repairs, environment conventions, and operational knowledge that compresses search, with fewer false-positive diagnoses and shorter trajectories in several settings. StarHarness therefore offers a practical way to reduce persistent model-environment mismatch in tool-rich enterprise tasks.