---
id: "2026-09-28-artificial-analysis-cyber-index"
url: "https://postcutoff.com/e/2026-09-28-artificial-analysis-cyber-index/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2026-09-28"
date_precision: day
category: benchmark
importance: 2
confidence: high
status: [Confirmed]
sources: 1
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2026-09-28-artificial-analysis-cyber-index/

# Artificial Analysis launches the Cyber Index and an industry alliance (IBM, NVIDIA, Vercel, Collinear) for AI vulnerability-fixing evals

On Sept 28, 2026 Artificial Analysis launched the Cyber Index, which combines three benchmarks of how well AI agents find and patch vulnerabilities, and the Cyber Index Alliance with Collinear AI, IBM, NVIDIA and Vercel to set shared evaluation standards for defensive cyber tasks. The best models solved only 41% of expert-verified vulnerabilities on its DeepsecBench-AA component.

## Key facts
- Components: CWE-Bench-AA (120 tasks: audit open-source repos and patch OWASP Top 10 flaws), DeepsecBench-AA (find vulnerabilities vs expert-verified findings), CyberGym-E2E-AA (131 tasks: discover, reproduce and patch C/C++ memory-safety bugs)
- All runs use Stirrup, an open-source agent harness
- Best models solve 41% of expert-verified vulnerabilities in DeepsecBench-AA; '55% of failed attempts fixed the primary issue but left a related one open'
- Refusal rates above 98% for frontier models on CyberGym-E2E-AA tasks
- Launch partners: Collinear AI, IBM, NVIDIA, Vercel

## What happened

Artificial Analysis, known for its model leaderboards, added a defensive-cyber index and recruited vendors to agree on how such work is measured.

## Why it matters

Cyber capability is now the main axis of frontier-model risk debates. A shared, defense-oriented benchmark from an independent evaluator gives buyers and
policymakers a common yardstick. The high refusal rates show that safety filters strongly shape these scores.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 151 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 90 days after its cutoff
- Gemini 4 Hydrogen (training cutoff March 2026): 181 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 120 days after its cutoff

## Sources

1. [Artificial Analysis: Cyber Index and Cyber Index Alliance](https://artificialanalysis.ai/articles/artificial-analysis-cyber-index) (artificialanalysis.ai, official)

## Changes

- 2026-09-30 (filed): Created

## Related

- 2026-09-29: [Anthropic: open-weights GLM-5.3 nearly matches Mythos Preview at exploit development](https://postcutoff.com/e/2026-09-29-anthropic-glm-5-3-spread-of-cyber-capabilities/index.md)