{"schema":"postcutoff/event@1","as_of":"2026-11-22T13:32:00+01:00","url":"https://postcutoff.com/e/2026-10-27-surge-dayjob-tax-benchmark/","md":"https://postcutoff.com/e/2026-10-27-surge-dayjob-tax-benchmark/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-10-27-surge-dayjob-tax-benchmark","date":"2026-10-27","date_precision":"day","short_title":"Surge AI's DAYJOB: Tax benchmark","deck":"The best agent (Claude Sonnet 5.5) passes only 10.8% of 50 professional tax assignments","takeaway":"On Oct 27, 2026 Surge AI released DAYJOB: Tax, 50 realistic tax-professional assignments (1,789 files) built and reviewed by tax experts, in which an agent must take a client request to a finished deliverable and spot issues the client did not ask about.","category":"benchmark","category_label":"Benchmarks","importance":2,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"Surge AI: DAYJOB: Tax, benchmarking AI agents on professional tax work (Oct 27)","url":"https://surgehq.ai/blog/dayjob-tax","type":"official","group":"primary","domain":"surgehq.ai"},{"n":2,"title":"Hacker News discussion (216 points)","url":"https://news.ycombinator.com/item?id=50275626","type":"discussion","group":"reactions","domain":"news.ycombinator.com"}],"official":1,"filed":"2026-10-29","updated":"2026-10-29","orgs":["Surge AI"],"title":"Surge AI's DAYJOB: Tax benchmark: the best agent (Claude Sonnet 5.5) passes only 10.8% of 50 professional tax assignments","summary":"On Oct 27, 2026 Surge AI released DAYJOB: Tax, 50 realistic tax-professional assignments (1,789 files) built and reviewed by tax experts, in which an agent must take a client request to a finished deliverable and spot issues the client did not ask about. Scored pass@1 at a 95% rubric threshold, Claude Sonnet 5.5 led with 10.8%, ahead of Claude Fable 5.1 (9.6%) and Claude Opus 5.5 (9.2%). Surge's examples show both Opus 5.5 and GPT-6 Astra missing consequential issues in different tasks.","key_facts":["Design: 50 environments, 1,789 files (records, receipts including a handwritten one); tasks test calculations, supporting evidence, consequential issues and the deliverable; three-layer expert review; graders include a Baker Tilly tax director","Results (pass@1 at 95% of rubric criteria): Claude Sonnet 5.5 10.8%, Claude Fable 5.1 9.6%, Claude Opus 5.5 9.2%; Sonnet 5.5 and Opus 5.5 sit on the cost-performance Pareto frontier (full leaderboard on Surge's page)","Examples (Surge): in a condo-sale timing task GPT-6 Astra never noticed the August 2027 deadline for the $250,000 home-sale exclusion in five attempts, while Opus 5.5 found it every time; in a divorce task Opus 5.5 assumed the couple was unmarried in all five attempts, while Astra checked marital status in four","Hacker News: 216 points (Oct 28)"],"key_numbers":[],"tags":["benchmarks","agents","professional-work","tax","surge-ai","claude","gpt-6-astra"],"science":null,"body_md":"## What happened\n\nSurge AI added a tax edition to its DAYJOB family of professional-work benchmarks. Even the best agents complete only about one\nassignment in ten to professional standard, mostly because they miss issues nobody asked them to look for.\n\n## Why it matters\n\nKnowledge-work benchmarks that require initiative, not just correct answers, still show a large gap between frontier agents and\nprofessionals, at the same time as labs market vertical products for finance and law.","disputed":[],"related":[{"id":"2026-10-27-artificial-analysis-intelligence-index-v5","url":"https://postcutoff.com/e/2026-10-27-artificial-analysis-intelligence-index-v5/","date":"2026-10-27","date_precision":"day","short_title":"Artificial Analysis launches Intelligence Index v5","deck":"60% private tests, FrontierCode 2.0 and Terminal-Bench-Science added; open-weights gap widens to 12 points","takeaway":"Leaderboard positions in late October 2026 shifted because the yardstick changed, not only because models did.","category":"benchmark","category_label":"Benchmarks","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":4,"official":4,"filed":"2026-10-29","updated":"2026-10-29","orgs":["Artificial Analysis","Cognition"]},{"id":"2026-09-28-claude-sonnet-5-5","url":"https://postcutoff.com/e/2026-09-28-claude-sonnet-5-5/","date":"2026-09-28","date_precision":"day","short_title":"Anthropic releases Claude Sonnet 5.5","deck":"30% faster, Opus-5.5-level scores on several benchmarks at $2/$10","takeaway":"Sonnet 5.5 roughly matches the new flagship on knowledge-work and computer-use benchmarks at half the price.","category":"model-release","category_label":"Model releases","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":10,"official":3,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Anthropic"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-29","type":"filed","text":"Created (news agent)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-29","run":null,"sources_read":null,"updated":"2026-10-29","human_review":null,"version":{"date":"2026-10-29"}},"gaps":[{"model_id":"gpt-6-2-astra","name":"GPT-6.2 Astra","cutoff":"2026-05","days_after":149,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":119,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":210,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":149,"in_training_data":false}],"short_url":null}