Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal
2026-07-09 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied how large language models learn to say no to unsafe requests and found that this refusal behavior is spread out across many parts of the model rather than coming from a single spot. They developed methods to attack these refusal signals by targeting the model's internal activations instead of just the final output, discovering that attacking broadly works better than focusing on one part. They also created a faster method called Soft-GCG that improved attack success. Their experiments showed that bigger models are harder to trick, likely due to stronger safety training. Overall, the authors provide new insights on how safety is represented inside models and how it can be bypassed.
Behavioral alignmentRefusal behaviorActivation spaceAdversarial attacksGumbel-SoftmaxActivation-Guided GCGSoft-GCGLarge language modelsSafety representationsJailbreak attacks
Authors
Ege Çakar, Hannah Guan, Kayden Kehe
Abstract
Behavioral alignment in large language models often masks fragile internal safety representations. Recent work suggests that refusal behavior is mediated by low-dimensional directions in activation space. This raises questions about how such representations are structured, localized, and accessed by optimization. We study adversarial suffix attacks as a probe of representational alignment. We introduce Activation-Guided GCG, which replaces output-based objectives with losses that directly target a model's internal refusal direction. Across several objective variants, we find that suppressing refusal globally across all layers and positions is more effective than targeting a single layer-position pair. This suggests that safety representations are distributed across the forward pass rather than causally localized to a single site. We further introduce Soft-GCG, a continuous relaxation of discrete suffix optimization using Gumbel-Softmax. Soft-GCG achieves a 33 $\times$ speedup over standard GCG while improving attack success rates. Evaluating across model scales, we find that smaller models remain vulnerable while larger models resist both activation- and suffix-based attacks at our compute-constrained settings, consistent with larger and better safety trained models being harder to jailbreak. Together, our results clarify how safety mechanisms are encoded and can be broken in contemporary models. These insights provide concrete guidance for designing more robust and representation-aware alignment strategies.