Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability
2026-06-02 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors studied how tweaking large language models to better fit a user's needs can affect how safe the model's responses are. They found that without focusing on a clear goal for the model's abilities, it's hard to understand or compare how safety changes after fine-tuning. Their tests showed that fine-tuned models sometimes give confusing answers when asked to be safe, and automated tools struggle to judge safety for these responses. They also noted that results about safety depend a lot on which tests and tools are used.
foundation modelslarge language modelsfine-tuningmodel safetysafety benchmarkssafety evaluationmodel capabilityincoherent generationsautomated safety judgment
Authors
Krishnapriya Vishnubhotla, Hillary Dawkins, Isar Nejadgholi, Svetlana Kiritchenko
Abstract
Adapting foundation large language models to a user's task or preferred style through fine-tuning can result in compromising the model's safety. Previous works examined the effects of fine-tuning on model safety in limited and seemingly random experimental settings. We argue that anchoring fine-tuning to a specific capability goal is essential for avoiding arbitrary empirical choices, allowing us to draw meaningful conclusions about safety impacts, and to compare mitigation methods on a consistent basis. We conduct a multi-dimensional evaluation of the effects of fine-tuning on model behavior by focusing on capability as well as safety. Our results surface important issues that (1) fine-tuned models can produce incoherent generations in response to safety prompts, (2) automated safety judgments are unreliable for such incoherent outputs, and (3) the conclusions about the effects of fine-tuning can change depending on the choice of safety benchmark as well as the safety evaluator.