When "Do Not" Is Not Deny: Security Rules in CLAUDE.md vs Built-In Controls
2026-08-24 • Human-Computer Interaction
Human-Computer InteractionCryptography and Security
AI summaryⓘ
The authors studied two ways to control a model's security: natural-language rules called "do not" instructions, and built-in computer controls called deny rules. They analyzed 481 files and found that only a small fraction (about 4-16%) of the written rules matched existing built-in controls, meaning many rules rely on the model’s interpretation instead of concrete enforcement. They also noted that developers write these rules but don’t get feedback on whether the controls actually enforce them, highlighting a problem in security management. Their method captured about two-thirds of the rules, so these findings only apply to the rules they found.
CLAUDE.mdClaude Codesecurity controlsdeny rulesnatural-language instructionLLMrule matchingsecurity enforcementsandboxannotation
Authors
Ting Yan
Abstract
In CLAUDE.md, "do not" is a natural-language instruction that the model interprets. Claude Code's deny is a built-in control that blocks an action before the agent can take it. Both can express the same security goal, but they control the agent in different ways. We measure this gap in 481 public CLAUDE.md files. An LLM matched the extracted candidate rules against Claude Code's documented controls, and two security practitioners independently checked a sample without seeing the model's answers or each other's labels. Depending on how closely a control had to match the written rule, only about 4-16% of the retrieved security rules had a matching built-in control. Under the strictest standard the estimate was 4.4% (95% CI: 2.6-6.7%), and the two annotators agreed closely on which rules had a match. A manual review of complete files found that our extraction method captured 66.3% of eligible security rules; the reported rates therefore apply to the rules it captured. This is a usable security problem: CLAUDE.md is a write-only channel. A developer writes a security rule but gets no feedback on whether a control will enforce it. The same plain-text form hides two kinds of rule: those a permission rule, mode, or sandbox can enforce, and those left to the model to interpret.