{"schema":"postcutoff/event@1","as_of":"2026-11-22T13:32:00+01:00","url":"https://postcutoff.com/e/2026-09-17-caisi-glm-5-3-cyber-assessment/","md":"https://postcutoff.com/e/2026-09-17-caisi-glm-5-3-cyber-assessment/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-09-17-caisi-glm-5-3-cyber-assessment","date":"2026-09-17","date_precision":"day","short_title":"NIST CAISI: GLM-5.3 is the most cyber-capable open-weight model yet, but trails the US frontier by about four months","deck":null,"takeaway":"On Sept 17, 2026 NIST's Center for AI Standards and Innovation (CAISI) published an assessment of Z.ai's GLM-5.3.","category":"policy-safety","category_label":"Policy & safety","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"NIST: CAISI's Assessment of Z.ai's GLM-5.3 Cyber Capabilities","url":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","type":"official","group":"primary","domain":"nist.gov"}],"official":1,"filed":"2026-10-03","updated":"2026-10-03","orgs":["NIST","CAISI","Zhipu AI"],"title":"NIST CAISI: GLM-5.3 is the most cyber-capable open-weight model yet, but trails the US frontier by about four months","summary":"On Sept 17, 2026 NIST's Center for AI Standards and Innovation (CAISI) published an assessment of Z.ai's GLM-5.3. It called GLM-5.3 \"the most cyber-capable open-weight model released to date\", ahead of Kimi K3, the previous best Chinese model. It also found the model's cyber capabilities \"significantly lower than those of current U.S. frontier models\", about four months behind them (e.g. SEC-Bench Pro 40.4% vs 90.2%).","key_facts":["Published by NIST CAISI on 2026-09-17; GLM-5.3 was announced 2026-08-14 and its weights were released about two weeks later","SEC-Bench Pro: GLM-5.3 40.4% vs 90.2% for the best US frontier model","ExploitBench: 61.1% (9.8/16) vs 100.0% (16.0/16)","ExploitGym: 9.4% (47/498) vs 44.4% (223/502)","CAISI OSS-Fuzz: 7.7% (23/297) vs 23.2% (69/297)","Method: models run as agents in a ReAct harness with bash and python; turn limits 200 (SEC-Bench Pro), 300 (ExploitBench), 200 (ExploitGym), 300 (OSS-Fuzz); US models tested with cyber safeguards disabled","CAISI estimate: GLM-5.3 lags the US frontier's cyber capability by about four months; it beats Kimi K3, the previous best PRC model"],"key_numbers":[],"tags":["ai-safety","cybersecurity","open-weights","evaluation","glm","government","china"],"science":null,"body_md":"## What happened\n\nThe US government's AI evaluation center (CAISI, part of NIST) ran GLM-5.3 and US frontier models through four agentic offensive-security\nbenchmarks: SEC-Bench Pro, ExploitBench, ExploitGym and its own OSS-Fuzz task set. All models ran in the same ReAct agent harness with bash and\npython tools, and the US models had their cyber safeguards switched off. GLM-5.3 scored higher than any earlier open-weight model, including\nMoonshot's Kimi K3. On every benchmark it scored well below the best US model, and CAISI put the gap at about four months of progress.\n\n## Why it matters\n\nThis is the US government's own measurement of the open-weight cyber gap, published twelve days before Anthropic's Frontier Red Team report on the\nsame model. The two reports frame the result differently. CAISI stresses that the US frontier is still clearly ahead. Anthropic stresses that\nGLM-5.3 nearly matched Claude Mythos Preview on its hardest exploit tasks and that its safeguards can be removed. Read together, they suggest that\ncapabilities the US frontier had a few months earlier are now available to anyone who downloads the weights.","disputed":[],"related":[{"id":"2026-09-29-anthropic-glm-5-3-spread-of-cyber-capabilities","url":"https://postcutoff.com/e/2026-09-29-anthropic-glm-5-3-spread-of-cyber-capabilities/","date":"2026-09-29","date_precision":"day","short_title":"Anthropic: open-weights GLM-5.3 nearly matches Mythos Preview at exploit development","deck":null,"takeaway":"It is the first time a frontier lab has published evidence that an open-weights model reached the level of exploit capability it had judged too risky to release widely.","category":"policy-safety","category_label":"Policy & safety","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":13,"official":5,"filed":"2026-09-30","updated":"2026-10-07","orgs":["Anthropic","Zhipu AI"]},{"id":"2026-09-01-huggingface-disables-offensive-cyber-glm-5-3","url":"https://postcutoff.com/e/2026-09-01-huggingface-disables-offensive-cyber-glm-5-3/","date":"2026-09-01","date_precision":"month","short_title":"Hugging Face disables an abliterated GLM-5.3 repo branded \"for offensive cyber\"","deck":"It is re-uploaded under a new name and mirrored on Pirate Face","takeaway":"It shows how little a platform takedown achieves for open weights.","category":"policy-safety","category_label":"Policy & safety","importance":3,"confidence":"medium","status":{"key":"partly","labels":["Partly confirmed"]},"sources":4,"official":2,"filed":"2026-10-03","updated":"2026-10-03","orgs":["Hugging Face","Audn AI","Pirate Face"]},{"id":"2026-08-14-zhipu-glm-5-3","url":"https://postcutoff.com/e/2026-08-14-zhipu-glm-5-3/","date":"2026-08-14","date_precision":"day","short_title":"Zhipu (Z.ai) releases GLM-5.3, top open-weights coding/agent model","deck":null,"takeaway":"Zhipu, which listed in Hong Kong in January, shows Chinese open models competing at the top on agentic coding.","category":"model-release","category_label":"Model releases","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":5,"official":2,"filed":"2026-09-29","updated":"2026-10-13","orgs":["Zhipu AI","Z.ai"]},{"id":"2026-07-16-moonshot-kimi-k3","url":"https://postcutoff.com/e/2026-07-16-moonshot-kimi-k3/","date":"2026-07-16","date_precision":"day","short_title":"Moonshot AI releases Kimi K3, a 2.8T-parameter open-weights multimodal model","deck":null,"takeaway":null,"category":"model-release","category_label":"Model releases","importance":5,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":5,"official":1,"filed":"2026-09-29","updated":"2026-10-11","orgs":["Moonshot AI"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-03","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-03","run":null,"sources_read":null,"updated":"2026-10-03","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-2-astra","name":"GPT-6.2 Astra","cutoff":"2026-05","days_after":109,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":79,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":170,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":109,"in_training_data":false}],"short_url":"https://postcutoff.com/s/nist-caisi-glm-5-3-most"}