AI summaryⓘ
The authors studied how bad actors can secretly mess with small datasets used to fine-tune large language models for text summarization, causing those models to produce biased or incorrect summaries without obvious mistakes in testing. They developed ways to detect and fix these hidden attacks by analyzing which training examples have an unusually strong influence and by testing the model's reactions to small meaning-preserving changes. Their methods work well even when attackers try new tricks to avoid detection, and can restore most of the model’s original performance without needing to retrain from scratch. This shows that fine-tuning attacks leave detectable traces that can be removed after the model is already deployed.
data poisoningfine-tuninglarge language modelsabstractive summarizationinfluence functionsblack-box auditingfactual distortionrepresentational biasgradient ascent unlearningROUGE metric
Abstract
Training-time data poisoning during fine-tuning poses a significant threat to large language models (LLMs) deployed for abstractive text summarization, where small task-specific datasets exert disproportionate influence on model behavior. In this setting, adversaries manipulate fine-tuning data to induce persistent summarization failures, such as biased or harmful summaries, while preserving standard evaluation metrics. We present a unified post-hoc defense framework for detecting and remediating fine-tuning-stage poisoning in summarization models across the machine learning supply chain. Our experiments show that in white-box settings, poisoned document-summary pairs exhibit abnormally high training influence, enabling detection via influence-function analysis with semantic consistency checks. In black-box settings, poisoned models display two to three times greater sensitivity to semantics-preserving perturbations, enabling behavioral auditing without training data access. Beyond existing poisoning formulations, we introduce novel attacks targeting factual distortion and representational bias, showing that poisoning alters summarization behavior without triggering conventional alarms. Across nine architectures and six benchmark datasets under adaptive attacks, our defenses achieve 85-92% detection precision, while gradient-ascent unlearning restores up to 96% of original behavior with minimal utility loss (less than 0.6% ROUGE degradation). These results indicate that fine-tuning-time poisoning leaves persistent structural artifacts, enabling practical detection and post-deployment recovery without full retraining.