Efficient Test-Time Adaptation through Human-AI Interaction
2026-09-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied how AI agents trained on large, general data often fall short of meeting individual experts' personal standards. They propose a method called test-time adaptation through human-agent interaction (TAHI), where the AI learns from user feedback over multiple sessions to better match individual preferences. This approach was tested on tasks in writing and visual creation, improving success rates significantly after only a few tasks. Their system also creates evolving evaluation guidelines that help catch more errors than traditional methods. Additionally, these personalized improvements sometimes benefit other users as well.
AI agentshuman-agent interactiontest-time adaptationpersonalizationevaluation rubricswriting tasksvisual creationmachine learninguser feedbacktask success metrics
Authors
Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao, Aspen Chen, Jonas Mueller, Zhiqi Liang, Jett Chen, Michael Ryan, Qianou Ma, Luxi He, Zhoujun Cheng, Andre He, Seungone Kim, Jiayi Geng, Mingqian Zheng, Weiwei Sun, Zheyuan Zhang, Xinran Zhao, Yike Wang, Abe Hou, Liwei Jiang, Pang Wei Koh, Diyi Yang, Graham Neubig, Daniel Fried
Abstract
AI agents are trained on population-scale data to encode broad capabilities spanning those of many practitioners. Yet the artifacts they produce rarely meet the personal bar professionals need to stake their reputation on. On realistic, open-ended tasks where success criteria are heterogeneous and insufficiently documented, individual expertise lives precisely in the elevation and departure from the average. In practice, iterative human-agent interaction surfaces criteria that users cannot fully specify up front, yet apply repeatedly across tasks. We argue this cross-session interaction data is a rich, underused signal for closing the gap to individual expertise. In this work, we propose test-time adaptation through human-agent interaction (TAHI), which integrates these signals into agent context and weights, and crystallizes each user's training and evaluation criteria via an evolving rubric module. We adapt agents to 30 individuals in two high-utility domains, writing and visual creation, on a total of 600 tasks. Our agents improve solo task success by 4.5-20.9% within only tens of tasks. Meanwhile, our evolving rubric module serves as a scalable annotation tool, creating evaluation rubrics that catch 16.0-22.3% more failures than those from LMs or humans alone. While agents are adapted towards individuals, we show these personalized agents also produce improvements in success of up to 8.8% that generalize across users.