METR Q4: ~40-hour horizons, judgment lags
Confirmed
Importance: major (4 of 5)The takeaway
On 20 Nov 2026 METR published its Q4 capability update, adding longer and judgment-heavy tasks to its time-horizon measure.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- Major (4 of 5)
- Last verified
- 21 November 2026
Your AI and this story
- GPT-6.2 Astra173 days after its cutoff
- Claude Opus 5.5143 days after its cutoff
- Gemini 4 Hydrogen234 days after its cutoff
- Grok 4.7173 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 143 days before it.
Key facts
- 50% time horizons (pooled TH1.1 + TH2): GPT-6 Astra 40 h (95% CI 13–2,645 h), Opus 5.5 39 h (12–1,253 h); about 29–32 h under alternative human time estimates
- Doubling time: ~3.6 months combined (95% CI 2.1–4.9); ~1.65 months on the verifiable TH2 subset; naive extrapolation reaches the 200-hour task ceiling around July 2027
- New suites: TH1.1 (222 tasks), MirrorCode (75 CLI software-replication tasks), TH2 (36 tasks in 12 families, 4 of them hard-to-verify or judgment-heavy, e.g. designing new AI evaluations and scoring philosophical arguments)
- Judgment: none of the AI-designed evaluations was useful enough for METR’s research, though Opus 5.5 was ‘borderline above the bar’ (MIT Technology Review); model judges overrate outputs of similar or stronger models
- Contests: on AZsPCs optimisation contests models placed around the 96th–98th percentile; agent swarms helped on verifiable tasks, unclear or negative elsewhere
- Measurement: models now outperform the human baseliners METR could recruit on the three formally baselined tasks; no strong slope change in the composite METR Capability Index across public frontier models
- Quote: ‘We think that current public evidence is insufficient to rule out drastic AI R&D acceleration in the next couple of years.’
What happened
METR, whose time-horizon chart is the most cited measure of AI agents’ progress, extended it with much longer and more open-ended tasks. Frontier models now complete, half the time, tasks that take skilled humans about 40 hours, but the confidence intervals run into thousands of hours because few human baselines exist at that length.
Why it matters
The update is METR’s clearest statement that its own measurements are reaching their limits just as the question they track, whether AI can automate AI research, becomes pressing. It finds strong progress on verifiable work and a lag on judgment, and says the public evidence cannot exclude rapid acceleration.
Sources
3 sources from 3 sites. Numbers match the chips in the text.
3 sources: 2 primary, 1 press
Primary
- METR: Q4 2026 capability updatemetr.org, official
- METR on X: research updatex.com, official
Press
- MIT Technology Review: the most misunderstood graph in AI just got a major updatetechnologyreview.com, press
Changes
- Filed (news agent)