Autonomous LLM-generated Feedback for Student Exercises in Introductory Software Engineering Courses
2026-04-22 • Software Engineering
Software Engineering
AI summaryⓘ
The authors created a tool called NAILA to help students in introductory software engineering courses get quick and personalized feedback on their exercises anytime. NAILA uses advanced AI models to compare student work with model solutions set by teachers. They studied over 900 students to understand why students use or avoid NAILA, how easy and useful they find it, how often they use it, and whether AI feedback helps their grades as much as human feedback. Their study aims to improve teaching in classes with many diverse students and limited teacher availability.
Software EngineeringGenerative AILarge Language ModelsAutomated FeedbackStudent EngagementUser AcceptanceAcademic PerformancePrompt EngineeringHuman-Computer InteractionEducational Technology
Authors
Andreas Metzger
Abstract
Introductory Software Engineering (SE) courses face rapidly increasing student enrollment numbers, participants with diverse backgrounds and the influence of Generative AI (GenAI) solutions. High teacher-to-student ratios often challenge providing timely, high-quality, and personalized feedback a significant challenge for educators. To address these challenges, we introduce NAILA, a tool that provides 24/7 autonomous feedback for student exercises. Utilizing GenAI in the form of modern LLMs, NAILA processes student solutions provided in open document formats, evaluating them against teacher-defined model solutions through specialized prompt templates. We conducted an empirical study involving 900+ active students at the University of Duisburg-Essen to assess four main research questions investigating (1) the underlying motivations that drive students to either adopt or reject NAILA, (2) user acceptance by measuring perceived usefulness and ease of use alongside subjective learning progress, (3) how often and how consistently students engage with NAILA, and (4) how using NAILA to receive AI feedback impacts on academic performance compared to human feedback.