Paper: models can hide arbitrary capabilities behind cryptographic backdoors that evaluations cannot elicit
Confirmed
The takeaway
Oxford-affiliated researchers embed encrypted neural networks in a model’s weights that run only on a secret trigger. Under cryptographic assumptions the hidden capability cannot be found even with full weight access. The demonstrations are small proofs of concept.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 13 October 2026
Your AI and this story
- GPT-6 Astra162 days after its cutoff
- Claude Opus 5.5101 days after its cutoff
- Gemini 3.8 Flash192 days after its cutoff
- Grok 4.7131 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 101 days before it.
Key facts
- Claim (abstract): capability evaluations rely ‘on the assumption that hidden capabilities can be elicited. We show this assumption does not hold under adversarial conditions’
- Method: ‘backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger’, ‘extending prior work from hiding fixed strings to arbitrary computations’
- Elicitation methods tested: fine-tuning, weight noising, gradient-based trigger search, latent adversarial training; none surfaced the hidden capability
- Scale: small ‘model organisms’ in PyTorch; the hidden classifier in the chemistry experiment is a tiny ternary MLP; the construction was also injected into Gemma 3 270M. The authors note that the cryptographic circuit and small-scale settings ‘limit the immediate practicality’
- Threat model: a deliberate attacker who modifies weights (for example a poisoned open-weight release), not capabilities that arise in ordinary training
- Status: arXiv preprint, not peer-reviewed. Hacker News item 50065692, about 215 points
What happened
The authors show how to hide a working neural network inside another model’s weights so that it is encrypted until a secret trigger appears in the input. They give cryptographic arguments that, without the trigger, the hidden circuit cannot be elicited or interpreted even by someone who holds the weights, and they test standard elicitation methods on small models.
Why it matters
Pre-deployment testing of open-weight or third-party models assumes that a dangerous capability can be made to show itself through prompting, fine-tuning or interpretability. This result says that for a model altered by an adversary the assumption can fail in principle, so provenance and supply-chain checks matter alongside evaluations. It does not show that current frontier models hide capabilities this way.
Sources
2 sources from 2 sites. Numbers match the chips in the text.
2 sources: 1 primary, 1 reaction
Primary
Reactions
- Hacker News discussionnews.ycombinator.com, discussion
Changes
- Filed (PDF read)