Surge AI’s DAYJOB: Tax benchmark
The best agent (Claude Sonnet 5.5) passes only 10.8% of 50 professional tax assignments
Confirmed
The takeaway
On Oct 27, 2026 Surge AI released DAYJOB: Tax, 50 realistic tax-professional assignments (1,789 files) built and reviewed by tax experts, in which an agent must take a client request to a finished deliverable and spot issues the client did not ask about.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 29 October 2026
Your AI and this story
- GPT-6.2 Astra149 days after its cutoff
- Claude Opus 5.5119 days after its cutoff
- Gemini 4 Hydrogen210 days after its cutoff
- Grok 4.7149 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 119 days before it.
Key facts
- Design: 50 environments, 1,789 files (records, receipts including a handwritten one); tasks test calculations, supporting evidence, consequential issues and the deliverable; three-layer expert review; graders include a Baker Tilly tax director
- Results (pass@1 at 95% of rubric criteria): Claude Sonnet 5.5 10.8%, Claude Fable 5.1 9.6%, Claude Opus 5.5 9.2%; Sonnet 5.5 and Opus 5.5 sit on the cost-performance Pareto frontier (full leaderboard on Surge’s page)
- Examples (Surge): in a condo-sale timing task GPT-6 Astra never noticed the August 2027 deadline for the $250,000 home-sale exclusion in five attempts, while Opus 5.5 found it every time; in a divorce task Opus 5.5 assumed the couple was unmarried in all five attempts, while Astra checked marital status in four
- Hacker News: 216 points (Oct 28)
What happened
Surge AI added a tax edition to its DAYJOB family of professional-work benchmarks. Even the best agents complete only about one assignment in ten to professional standard, mostly because they miss issues nobody asked them to look for.
Why it matters
Knowledge-work benchmarks that require initiative, not just correct answers, still show a large gap between frontier agents and professionals, at the same time as labs market vertical products for finance and law.
Sources
2 sources from 2 sites. Numbers match the chips in the text.
2 sources: 1 primary, 1 reaction
Primary
- Surge AI: DAYJOB: Tax, benchmarking AI agents on professional tax work (Oct 27)surgehq.ai, official
Reactions
- Hacker News discussion (216 points)news.ycombinator.com, discussion
Changes
- Filed (news agent)