Mercor: 90% of US working hours lack AI benchmark coverage
Partly confirmed
The takeaway
Mercor published a map that tags the prompts of 29 public agentic benchmarks to O*NET’s 18,797 occupational tasks.
Status
- Claim
Partly confirmed
- Our reporting
- Medium confidence
- Importance
- 2 of 5
- Last verified
- 16 October 2026
Your AI and this story
- GPT-6 Astra169 days after its cutoff
- Claude Opus 5.5108 days after its cutoff
- Gemini 4 Argon199 days after its cutoff
- Grok 4.7138 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 108 days before it.
Key facts
- Data: O*NET (923 occupations, 18,797 tasks), BLS wages and employment (40-hour week), Eloundou et al. (2023) exposure tags, task time shares estimated by an LLM (Hatgis-Kessell et al.); an LLM judge maps benchmark prompts to tasks; a task is ‘covered’ with ≥5 prompts
- Index: 29 benchmarks chosen for public availability, agentic tasks, economic realism and non-saturation, with per-family caps
- Coverage: 924 of 18,797 tasks; 208 of 923 occupations; six of 22 job families have no covered occupation; e.g. Computer & Math 29/36 occupations, Healthcare Practitioners 12/89, Education 9/62
- Exposed tasks ~6x more likely to be covered than unexposed; an occupation’s most important tasks ~6–10x more likely than its least important; lawyers’ tasks heavily covered, legal secretaries’ and court clerks’ barely
- Caveats on the page: incomplete benchmark list, so coverage is provisional and ‘likely understated’; physical tasks hard to benchmark
- Date: the page carries no publication date; dated here by its Hacker News submission (Oct 16, 2026)
What happened
Mercor, the expert-data company that runs the APEX benchmarks, released a “benchmark map” as part of its labour index. It asks which parts of the US economy current AI benchmarks actually measure. By its count, benchmarks concentrate on software engineering, financial analysis and law, while most occupations, and especially physical work, have no measured tasks. It also tests weighting APEX scores by O*NET tasks and finds results similar to simple averages.
Why it matters
Claims about AI’s effect on jobs often rest on benchmark scores. This map gives a rough measure of how little of real working time those benchmarks cover. The tagging is done by an LLM judge and the benchmark list is incomplete, so the numbers are estimates.
Sources
2 sources from 2 sites. Numbers match the chips in the text.
2 sources: 1 primary, 1 reaction
Primary
- Mercor: A benchmark map of the US economymercor.com, official
Reactions
- Hacker News discussion (~243 points)news.ycombinator.com, discussion
Changes
- Filed (page read directly; publication date unknown, confidence medium)