Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification
2026-08-31 • Software Engineering
Software EngineeringArtificial IntelligenceCryptography and Security
AI summaryⓘ
The authors study anonymous AI models that are released without clear identity, making it hard to trust what they do or how they handle data. They create a step-by-step method to figure out which model is which by examining clues like original settings, how the model processes language, and how it behaves. They tested their method on known models and some secret cases, showing it can give useful identity guesses without false claims. Their approach helps provide more trustworthy information about hidden AI releases.
AI modelblack-box verificationtokenizerAPImodel fingerprintingbehavioral probesInternet Archivesupply-chain riskdata-handlingForensic audit
Authors
Yisen Xi
Abstract
The 2025--2026 AI market has seen a wave of stealth releases: frontier models launched anonymously on developer platforms under codenames. For their users, identity determines data-handling terms, supply-chain risk, and capability expectations. No validated methodology exists for black-box identity verification of anonymous models: practitioner checklists lack accuracy evidence, and self-identification is untrustworthy by design. We propose a four-stage forensic audit protocol for API-served models. Stage 0 reconstructs launch-time configuration from archived platform snapshots (Internet Archive), exposing preview--production drift. Stage 1 fingerprints configuration (context, output ceiling, reasoning, modality) against the platform catalog. Stage 2 tests tokenizer identity with a cross-length differential that rejects short-prompt collisions. Stage 3 corroborates with behavioral probes. We test declaration consistency on 10 known-identity releases (7 exact, 2 precision-differences, 1 partial, 0 counter-directional), not end-to-end identification under anonymity. Identification is validated prospectively on a flagship case whose 2026-08-23 analysis pointed to the GLM-5.3 version line and whose official reveal confirmed those family and version-line inferences (deployment variant was not pre-asserted; Flash was consistent post-reveal), and on three Stage-0-only cases where the protocol produced a graded hypothesis or declined rather than guessed. A standard-library-only implementation is provided as supplementary material.