AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study
2026-06-02 • Computers and Society
Computers and SocietyArtificial Intelligence
AI summaryⓘ
The authors studied how AI-generated animations called Generated Animated Traces (GATs), which show code running with helpful analogies and narration, affect students learning programming. They compared GATs to regular text explanations in beginner programming courses with Python and Java students. The study found that GATs sometimes help students learn right away, but these benefits depend on the context and don't last long. Also, how well GATs work depends on how engaged the students are, suggesting teaching tools might need to be personalized.
Introductory programmingProgram executionVisualizationsAI-generated animationsNarrated animationsLearner engagementCS1 coursesPythonJava
Authors
Yuri Noviello, Naaz Sibia, Anastasiia Birillo, Thomas Overklift Vaupel Klein, Michael Liut, Gosia Migut
Abstract
Introductory programming (CS1) courses often struggle to support students' understanding of program execution. While visualizations can make execution processes explicit, their effectiveness depends on design and context, and empirical evidence for AI-generated visualizations remains limited. We propose Generated Animated Traces (GATs), AI-generated, analogy-based, narrated animations that coordinate source code, execution state, and conceptual analogies. We conduct a study at two institutions in CS1 courses (Python, N=961; Java N=151) comparing GATs to textual explanations. We measure immediate learning performance and experience, end-of-course engagement and exam performance. Results show that GATs can yield selective benefits for immediate learning, but benefits are context-dependent and short-term. We observe that GATs' influence on performance is moderated by learner engagement profiles. This finding underscores the importance of personalized approaches.