{"schema":"postcutoff/event@1","as_of":"2026-10-13T13:21:00+02:00","url":"https://postcutoff.com/e/2026-10-09-ai-models-can-provably-hide-arbitrary-capabilities/","md":"https://postcutoff.com/e/2026-10-09-ai-models-can-provably-hide-arbitrary-capabilities/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"Adam Bicz","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-10-09-ai-models-can-provably-hide-arbitrary-capabilities","date":"2026-10-09","date_precision":"day","short_title":"Paper: models can hide arbitrary capabilities behind cryptographic backdoors that evaluations cannot elicit","deck":null,"takeaway":"Oxford-affiliated researchers embed encrypted neural networks in a model's weights that run only on a secret trigger. Under cryptographic assumptions the hidden capability cannot be found even with full weight access. The demonstrations are small proofs of concept.","category":"research","category_label":"Research","importance":2,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"arXiv 2610.13928: AI Models Can Provably Hide Arbitrary Capabilities","url":"https://arxiv.org/abs/2610.13928","type":"paper","group":"primary","domain":"arxiv.org"},{"n":2,"title":"Hacker News discussion","url":"https://news.ycombinator.com/item?id=50065692","type":"discussion","group":"reactions","domain":"news.ycombinator.com"}],"official":1,"filed":"2026-10-13","updated":"2026-10-13","orgs":["University of Oxford","Contramont Research","ACS Research"],"title":"Paper: models can hide arbitrary capabilities behind cryptographic backdoors that evaluations cannot elicit","summary":"\"AI Models Can Provably Hide Arbitrary Capabilities\" (arXiv 2610.13928, 9 Oct 2026; Draguns, Motwani, Douglas, Schroeder de Witt) constructs backdoors that embed encrypted neural networks inside a host model's weights and decrypt and run them only on a secret trigger. Using digital lockers, the hidden circuits are \"provably hard to elicit and interpret under cryptographic assumptions, even given full access to model weights\". In small PyTorch experiments (a chemical reaction prediction task as a CBRN proxy) fine-tuning, weight noising, gradient-based trigger search and latent adversarial training did not surface the capability. The paper reached the Hacker News front page (about 215 points on Oct 13).","key_facts":["Claim (abstract): capability evaluations rely 'on the assumption that hidden capabilities can be elicited. We show this assumption does not hold under adversarial conditions'","Method: 'backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger', 'extending prior work from hiding fixed strings to arbitrary computations'","Elicitation methods tested: fine-tuning, weight noising, gradient-based trigger search, latent adversarial training; none surfaced the hidden capability","Scale: small 'model organisms' in PyTorch; the hidden classifier in the chemistry experiment is a tiny ternary MLP; the construction was also injected into Gemma 3 270M. The authors note that the cryptographic circuit and small-scale settings 'limit the immediate practicality'","Threat model: a deliberate attacker who modifies weights (for example a poisoned open-weight release), not capabilities that arise in ordinary training","Status: arXiv preprint, not peer-reviewed. Hacker News item 50065692, about 215 points"],"key_numbers":[],"tags":["safety","evaluations","backdoors","cryptography","sandbagging","interpretability","capability-elicitation"],"science":null,"body_md":"## What happened\n\nThe authors show how to hide a working neural network inside another model's weights so that it is encrypted until a secret\ntrigger appears in the input. They give cryptographic arguments that, without the trigger, the hidden circuit cannot be\nelicited or interpreted even by someone who holds the weights, and they test standard elicitation methods on small models.\n\n## Why it matters\n\nPre-deployment testing of open-weight or third-party models assumes that a dangerous capability can be made to show itself\nthrough prompting, fine-tuning or interpretability. This result says that for a model altered by an adversary the\nassumption can fail in principle, so provenance and supply-chain checks matter alongside evaluations. It does not show\nthat current frontier models hide capabilities this way.","disputed":[],"related":[{"id":"2026-10-12-anthropic-self-replicating-ai-iot-devices","url":"https://postcutoff.com/e/2026-10-12-anthropic-self-replicating-ai-iot-devices/","date":"2026-10-12","date_precision":"day","short_title":"Anthropic's Frontier Red Team","deck":"Small AI models may be months away from self-replicating on IoT devices such as Android TV boxes","takeaway":"Earlier warnings about self-replicating AI assumed data-center GPUs or stolen API keys.","category":"policy-safety","category_label":"Policy & safety","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":6,"official":1,"filed":"2026-10-12","updated":"2026-10-13","orgs":["Anthropic"]},{"id":"2026-09-18-anthropic-accenture-embedded-evaluation","url":"https://postcutoff.com/e/2026-09-18-anthropic-accenture-embedded-evaluation/","date":"2026-09-18","date_precision":"day","short_title":"Anthropic and Accenture commit $1B+ to embedded third-party evaluation","deck":null,"takeaway":"It is an unusually deep form of external oversight of a frontier lab's training process.","category":"policy-safety","category_label":"Policy & safety","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":4,"official":1,"filed":"2026-09-29","updated":"2026-10-13","orgs":["Anthropic"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-13","type":"filed","text":"Created (PDF read)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-13","run":null,"sources_read":"PDF read","updated":"2026-10-13","human_review":null,"version":{"date":"2026-10-13"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":162,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":101,"in_training_data":false},{"model_id":"gemini-3-8-flash","name":"Gemini 3.8 Flash","cutoff":"2026-03","days_after":192,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":131,"in_training_data":false}],"short_url":null}