NIST CAISI: GLM-5.3 is the most cyber-capable open-weight model yet, but trails the US frontier by about four months
Confirmed
Importance: major (4 of 5)The takeaway
On Sept 17, 2026 NIST’s Center for AI Standards and Innovation (CAISI) published an assessment of Z.ai’s GLM-5.3.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- Major (4 of 5)
- Last verified
- 3 October 2026
Your AI and this story
- GPT-6.2 Astra109 days after its cutoff
- Claude Opus 5.579 days after its cutoff
- Gemini 4 Hydrogen170 days after its cutoff
- Grok 4.7109 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 79 days before it.
Key facts
- Published by NIST CAISI on 2026-09-17; GLM-5.3 was announced 2026-08-14 and its weights were released about two weeks later
- SEC-Bench Pro: GLM-5.3 40.4% vs 90.2% for the best US frontier model
- ExploitBench: 61.1% (9.8/16) vs 100.0% (16.0/16)
- ExploitGym: 9.4% (47/498) vs 44.4% (223/502)
- CAISI OSS-Fuzz: 7.7% (23/297) vs 23.2% (69/297)
- Method: models run as agents in a ReAct harness with bash and python; turn limits 200 (SEC-Bench Pro), 300 (ExploitBench), 200 (ExploitGym), 300 (OSS-Fuzz); US models tested with cyber safeguards disabled
- CAISI estimate: GLM-5.3 lags the US frontier’s cyber capability by about four months; it beats Kimi K3, the previous best PRC model
What happened
The US government’s AI evaluation center (CAISI, part of NIST) ran GLM-5.3 and US frontier models through four agentic offensive-security benchmarks: SEC-Bench Pro, ExploitBench, ExploitGym and its own OSS-Fuzz task set. All models ran in the same ReAct agent harness with bash and python tools, and the US models had their cyber safeguards switched off. GLM-5.3 scored higher than any earlier open-weight model, including Moonshot’s Kimi K3. On every benchmark it scored well below the best US model, and CAISI put the gap at about four months of progress.
Why it matters
This is the US government’s own measurement of the open-weight cyber gap, published twelve days before Anthropic’s Frontier Red Team report on the same model. The two reports frame the result differently. CAISI stresses that the US frontier is still clearly ahead. Anthropic stresses that GLM-5.3 nearly matched Claude Mythos Preview on its hardest exploit tasks and that its safeguards can be removed. Read together, they suggest that capabilities the US frontier had a few months earlier are now available to anyone who downloads the weights.
Sources
1 source from 1 site. Numbers match the chips in the text.
1 source: 1 primary
Primary
- NIST: CAISI’s Assessment of Z.ai’s GLM-5.3 Cyber Capabilitiesnist.gov, official
Changes
- Filed