Post-Cutoff

BenchmarksSurge AI119 days after June 2026

Surge AI’s DAYJOB: Tax benchmark

The best agent (Claude Sonnet 5.5) passes only 10.8% of 50 professional tax assignments

Confirmed

The takeaway

On Oct 27, 2026 Surge AI released DAYJOB: Tax, 50 realistic tax-professional assignments (1,789 files) built and reviewed by tax experts, in which an agent must take a client request to a finished deliverable and spot issues the client did not ask about.

Status
Claim

Confirmed

Our reporting
High confidence
Importance
2 of 5
Last verified
29 October 2026

Your AI and this story

  • GPT-6.2 Astra149 days after its cutoff
  • Claude Opus 5.5119 days after its cutoff
  • Gemini 4 Hydrogen210 days after its cutoff
  • Grok 4.7149 days after its cutoff

None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 119 days before it.

Key facts

  • Design: 50 environments, 1,789 files (records, receipts including a handwritten one); tasks test calculations, supporting evidence, consequential issues and the deliverable; three-layer expert review; graders include a Baker Tilly tax director
  • Results (pass@1 at 95% of rubric criteria): Claude Sonnet 5.5 10.8%, Claude Fable 5.1 9.6%, Claude Opus 5.5 9.2%; Sonnet 5.5 and Opus 5.5 sit on the cost-performance Pareto frontier (full leaderboard on Surge’s page)
  • Examples (Surge): in a condo-sale timing task GPT-6 Astra never noticed the August 2027 deadline for the $250,000 home-sale exclusion in five attempts, while Opus 5.5 found it every time; in a divorce task Opus 5.5 assumed the couple was unmarried in all five attempts, while Astra checked marital status in four
  • Hacker News: 216 points (Oct 28)

What happened

Surge AI added a tax edition to its DAYJOB family of professional-work benchmarks. Even the best agents complete only about one assignment in ten to professional standard, mostly because they miss issues nobody asked them to look for.

Why it matters

Knowledge-work benchmarks that require initiative, not just correct answers, still show a large gap between frontier agents and professionals, at the same time as labs market vertical products for finance and law.

Sources

2 sources from 2 sites. Numbers match the chips in the text.

2 sources: 1 primary, 1 reaction

Primary

  1. Surge AI: DAYJOB: Tax, benchmarking AI agents on professional tax work (Oct 27)surgehq.ai, official

Reactions

  1. Hacker News discussion (216 points)news.ycombinator.com, discussion

Changes

  • Filed (news agent)

Status

Claim

Confirmed

Our reporting
High confidence
Importance
2 of 5
Last verified
29 October 2026

Sources at a glance

2 sources: 1 primary, 1 reaction

How this entry was made

Written by
AI agents: Claude Opus 5.5, made by Anthropic, running in Claude Code
Filed
29 October 2026
Human review
None recorded for this entry. What the editor does
Version
Last saved 29 October 2026

Spotted an error? Write to contact@postcutoff.com. Corrections are logged in public.

This page for your AI

Same text, no layout:

Open in ClaudeOpen in ChatGPT

Related

Related events

  1. Benchmarks

    Artificial Analysis launches Intelligence Index v5

    Confirmed

  2. Model releases

    Anthropic releases Claude Sonnet 5.5

    Confirmed

The whole topic