Hands-Off or Hands-On? Variation in Area Chair Practices and Implications for AI Support
2026-08-07 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors studied how area chairs (ACs) in AI conferences handle peer review, including challenges like lots of submissions and varying reviewer expertise. They found that ACs differ a lot in how much they get involved, with some being hands-off and others more hands-on. When asked about AI tools to help them, ACs were careful and aware of AI's limitations. The authors suggest designing AI tools that adapt to these different styles, help with managing discussions, and support ACs without taking away their control.
area chairpeer reviewconflict managementAI toolsthematic analysisreviewer expertisediscussion moderationhuman-centered AIsubmission volumedecision-making
Authors
Ines Arous, Neha Nayak Kennard, Andrei Mircea, Emily Kuang, Jackie Chi Kit Cheung, Andrew McCallum
Abstract
Area chairs (ACs) play a critical role in the peer-review process, managing conflicts and ensuring fair outcomes. Although AI tools have been proposed to support ACs, little is known about the challenges they face and their perceptions of these technologies. In this paper, we conduct interviews including a design probe with 27 ACs in AI to explore their challenges, strategies, and perspectives on potential AI tools. Through thematic analysis, we identify key tensions arising from the growing volume of submissions, uneven reviewer expertise, and the complex task of managing the relationship between reviewers and authors. Most importantly, we find substantial variation in how ACs engage with submissions and influence outcomes: some adopt a largely hands-off approach, while others take a more hands-on role in guiding discussions and decisions. This variation challenges the notion of a single, universal AC practice and highlights the need to account for diverse approaches. When reflecting on the potential use of AI tools, ACs expressed a cautious stance, drawing on their domain knowledge and heightened awareness of AI limitations. From these findings, we derive three design implications: tailoring AI assistance to diverse AC practices, design assistance for discussion moderation, and embedding human-centered AI principles that preserve human agency in decision-making.