Detection of LLM-assisted Code Plagiarism Using k-gram Software Birthmarks
2026-07-03 • Software Engineering
Software Engineering
AI summaryⓘ
The authors studied if a technique called software birthmarks can catch plagiarism when big AI models rewrite code so it looks different but works the same. They used specific patterns of Java code instructions (k-grams) to compare programs and tested this on code rewritten by three AI models. They found that these birthmarks still work well to detect AI-assisted plagiarism, although some models like ChatGPT-5.1-Codex-Mini made detection harder. Their work shows that tools aimed at code can be better at spotting copied code when AI tries to disguise it.
Large language modelsSoftware plagiarismCode paraphrasingSoftware birthmarksk-gramsJava opcodesSimilarity measuresCosine similarityPlagiarism detectionChatGPT
Authors
Nikolay Fedorov, Akito Monden, Hiroki Inayoshi, Haruaki Tamada, Masateru Tsunoda
Abstract
Large language models (LLMs) have significantly lowered the technical barrier to software plagiarism. By transforming existing source code while preserving its functionality, modern LLMs can generate semantically identical program that may evade traditional plagiarism detection techniques. Among such attacks, code paraphrasing modifies the syntax and structure of a program while preserving its behavior. This paper investigates whether software birthmarks can detect such LLM-assisted plagiarism. As a starting point, we employ k-gram software birthmarks based on unique k-grams of Java opcodes, with k ranging from 1 to 6. We employ three contemporary LLMs: ChatGPT-5.1-Codex-Mini, DeepSeek-V4-Flash, and Claude-Haiku-4.5. The dataset consists of individually compilable source files extracted from actively maintained BSD-2-Clause licensed Java projects. We further compare five similarity measures for birthmark matching: cosine similarity, Dice index, Jaccard coefficient, Simpson index, and edit-distance-based similarity. The results demonstrate that k-gram software birthmarks remain effective for detecting LLM-assisted plagiarism. Among the evaluated models, ChatGPT-5.1-Codex-Mini generated the most difficult-to-detect clones. Furthermore, the findings confirm the higher performance of coding-oriented models for plagiarism task.