Beyond Summaries: Structure-Aware Labeling of Code Changes with Large Language Models
2026-05-25 • Software Engineering
Software EngineeringArtificial Intelligence
AI summaryⓘ
The authors studied how large language models (LLMs) can help label different types of code changes, like renaming or moving code, during code reviews. They created a two-step method that first labels parts of the code changes and then refines those labels to understand relationships and details better. Their approach uses few examples to guide the LLMs and works across different programming languages without complex setup. Testing on various code patches showed that their method can identify changes accurately and support code review automation. This suggests LLMs could be a valuable tool alongside traditional analysis to make code reviews easier and more flexible.
code reviewcode patchlarge language modelstaxonomy-based labelingdiff hunksfew-shot promptingstatic analysisrename propagationtype changescode automation
Authors
Bar Weiss, Antonio Abu-Nassar, Adi Sosnovich, Karen Yorav
Abstract
Code review is a critical practice in software engineering, yet the growing scale and frequency of code patches in modern projects, together with the widespread adoption of AI code assistants, make manual review increasingly challenging. Identifying the types of changes within a patch, such as renames, moves, or logic modifications, can substantially improve review efficiency by enabling prioritization, filtering, and automation. However, existing LLM-based approaches to code review have largely focused on summarization and comment generation, leaving structured code reviews underexplored. In this paper, we present a systematic study of using large language models (LLMs) for taxonomy-based labeling of code changes in a code patch. We introduce a two-stage pipeline that assigns labels to diff hunks and then refines them to capture structural relationships and semantic attributes, such as rename propagation and type changes. Our approach employs few-shot prompting to produce language-agnostic and customizable labels, without the engineering overhead of traditional static-analysis pipelines. We evaluate four LLMs across multiple context configurations on a manually curated benchmark of natural and synthetic patches. Our best configuration achieves up to $84\%$ recall and $81\%$ precision, with high accuracy in extracting relational and attribute metadata. These results suggest that LLM-based labeling can effectively complement static analysis by enabling flexible, multilingual, and automation-friendly code review workflows.