The State of Peer Review in Empirical Software Engineering: A Community Survey on Review Load, Quality, and GenAI Use
2026-06-03 • Software Engineering
Software Engineering
AI summaryⓘ
The authors studied how the scientific peer review process, especially in empirical software engineering (ESE), is struggling due to more paper submissions, heavy workloads, and new challenges from AI tools. They collected answers from 120 community members about how reviewers feel about their workload, the quality of reviews, common problems, and the use of AI tools in reviewing. The authors also gathered suggestions from the community on how to improve peer review. Their work aims to help guide better discussions on improving the review system based on real opinions.
peer reviewempirical software engineeringreview workloadreview qualitylarge language modelsgenerative AIsurveyscientific publishingreview challenges
Authors
Justus Bogner, Roberto Verdecchia
Abstract
The scientific peer review system has been slowly deteriorating over the last years, and not just within empirical software engineering (ESE) research. Increased submission numbers, high workload, and the rise of generative AI use with all its associated issues have made many cracks in the system more visible. To get a better understanding of the current state of peer review in the ESE community, we conducted a questionnaire survey, which accumulated 120 responses. We report on (i) the perceived review load of community members, (ii) review quality perception as well as frequent challenges for and issues with reviews, (iii) the use of LLM-based tools in the reviewing process, and (iv) the community's suggestions for improving the peer review system. We hope that these community opinions can facilitate more evidence-based discussions about how people want to see the review system change for the better.