{"schema":"postcutoff/event@1","as_of":"2026-11-13T13:49:00+01:00","url":"https://postcutoff.com/e/2026-09-18-anthropic-accenture-embedded-evaluation/","md":"https://postcutoff.com/e/2026-09-18-anthropic-accenture-embedded-evaluation/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-09-18-anthropic-accenture-embedded-evaluation","date":"2026-09-18","date_precision":"day","short_title":"Anthropic and Accenture commit $1B+ to embedded third-party evaluation","deck":null,"takeaway":"On September 18, 2026 Anthropic announced a partnership with Accenture's Faculty division.","category":"policy-safety","category_label":"Policy & safety","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"Partnering with Accenture on embedded evaluation (Anthropic)","url":"https://www.anthropic.com/news/accenture-embedded-evaluation","type":"official","group":"primary","domain":"anthropic.com"},{"n":2,"title":"CNBC: AI's quiet safety gatekeepers are stepping into the spotlight (Oct 11, 2026)","url":"https://www.cnbc.com/2026/10/11/ais-quiet-safety-gatekeepers-are-stepping-into-the-spotlight.html","type":"press","group":"press","domain":"cnbc.com"},{"n":3,"title":"Reuters via ETCIO (syndicated full text): Anthropic, OpenAI face scrutiny over independent AI evaluations (Oct 13, 2026)","url":"https://cio.economictimes.indiatimes.com/news/artificial-intelligence/anthropic-openai-face-scrutiny-over-independent-ai-evaluations/134915453","type":"press","group":"press","domain":"cio.economictimes.indiatimes.com"},{"n":4,"title":"Reuters (exclusive): Behind AI industry's call for independent reviews, a fight over what gets made public (Oct 13, 2026; via TradingView, paywalled)","url":"https://www.tradingview.com/news/reuters.com,2026:newsml_L1N45S0TG:0-behind-ai-industry-s-call-for-independent-reviews-a-fight-over-what-gets-made-public/","type":"press","group":"press","domain":"tradingview.com"},{"n":5,"title":"Zvi Mowshowitz: The Quest for Embedded Evaluators","url":"https://thezvi.substack.com/p/the-quest-for-embedded-evaluators","type":"discussion","group":"reactions","domain":"thezvi.substack.com"}],"official":1,"filed":"2026-09-29","updated":"2026-10-13","orgs":["Anthropic"],"title":"Anthropic and Accenture (Faculty) commit $1B+ to embedded third-party evaluation","summary":"On September 18, 2026 Anthropic announced a partnership with Accenture's Faculty division. Embedded evaluators get employee-level access to red-team models, run alignment assessments, test safeguards and observe training. Both companies plan to invest at least $1B over five years.","key_facts":["Announced Sept 18, 2026","At least $1B over five years in evaluation capacity","Embedded evaluators get employee-level access to observe training and development decisions","Non-exclusive; Anthropic will also work with METR and others; long-term it favors pooled or government funding","State of the field (CNBC, Oct 11): METR said in August it had raised ~$71M in commitments over six months (vs $13.6M total 2024 contributions) and has fewer than 50 full-time staff; for-profit Vals AI grew from 8 to ~30 staff and raised $40M in August. Anthropic is in talks with METR and other nonprofits that plan to self-fund pilots of 'elements' of embedded evaluation","The AI Evaluator Forum (incl. METR and AVERI) published a public letter in September, 'Minimum Conditions for Embedding Evaluators': transparency, protection from retaliation and access equivalent to the labs' 'own highly privileged employees'; embedded evaluation should complement, not replace, broader oversight (CNBC)","OpenAI (Oct 9) said it is 'actively finalizing contracts with third-party safety assessors' and will announce details in the coming weeks, building on work with METR and Redwood Research; fired researchers Mikita Balesni and Tomek Korbak say they were dismissed over how they communicated with third-party evaluators, which OpenAI disputes (CNBC)","Critics on funding and independence: Brown's Suresh Venkatasubramanian ('the problem, as always, is money'), Wharton's Kevin Werbach (ecosystem 'not robust enough'); Fathom CEO Andrew Freedman argues for government-licensed Independent Verification Organizations, as in the FRONTIER Act and two California laws Newsom signed (CNBC)","Second wave (Reuters exclusive, Oct 13, 2026; own entry 2026-10-13-reuters-labs-sought-control-evaluator-publications, read via the syndicated text at ETCIO): two sources say Anthropic and OpenAI sought strict editorial control over evaluators' publications as late as Oct 5, with Anthropic's talks moving by Oct 8. Anthropic told Reuters that evaluators it has worked with control their conclusions and publications, but that for new evaluator firms it 'reserved the right to comment on and approve reports'. Faculty CEO Marc Warner: 'independence and transparency are critical'. Details of Faculty's access and reporting were 'still being worked out'"],"key_numbers":[],"tags":["evaluations","third-party-audit","governance"],"science":null,"body_md":"## What happened\n\nThis is the first implementation of the unilateral commitment in Amodei's \"We Must Pace the Frontier\" essay. Anthropic funds the work directly for now.\n\n## Why it matters\n\nIt is an unusually deep form of external oversight of a frontier lab's training process.","disputed":[],"related":[{"id":"2026-10-13-reuters-labs-sought-control-evaluator-publications","url":"https://postcutoff.com/e/2026-10-13-reuters-labs-sought-control-evaluator-publications/","date":"2026-10-13","date_precision":"day","short_title":"Reuters: Anthropic and OpenAI sought significant control over what outside evaluators can publish, citing trade secrets","deck":null,"takeaway":"Independent evaluation is the main safety mechanism the labs themselves proposed (Amodei's essay, the White House accord, the planned standards body).","category":"policy-safety","category_label":"Policy & safety","importance":3,"confidence":"medium","status":{"key":"partly","labels":["Partly confirmed"]},"sources":4,"official":0,"filed":"2026-10-13","updated":"2026-10-13","orgs":["Anthropic","OpenAI","Reuters","METR","Faculty"]},{"id":"2026-10-01-openai-parts-ways-three-safety-researchers","url":"https://postcutoff.com/e/2026-10-01-openai-parts-ways-three-safety-researchers/","date":"2026-10-01","date_precision":"day","short_title":"OpenAI fires three safety researchers who allegedly shared confidential information with an outside AI safety organization","deck":null,"takeaway":"OpenAI was under the most outside scrutiny in its history: an FTC probe, lawsuits, independent reconstructions of its agents' activity, and parliamentary inquiries in Australia.","category":"policy-safety","category_label":"Policy & safety","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":38,"official":2,"filed":"2026-10-01","updated":"2026-10-26","orgs":["OpenAI"]},{"id":"2026-09-12-dario-amodei-pace-the-frontier","url":"https://postcutoff.com/e/2026-09-12-dario-amodei-pace-the-frontier/","date":"2026-09-12","date_precision":"day","short_title":"Dario Amodei publishes \"We Must Pace the Frontier\", calling for a deliberate slowdown","deck":null,"takeaway":"It is the first time the CEO of a leading frontier lab has publicly called for slowing the frontier and paired the call with a unilateral commitment.","category":"policy-safety","category_label":"Policy & safety","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":10,"official":2,"filed":"2026-09-29","updated":"2026-10-29","orgs":["Anthropic"]},{"id":"2026-07-23-frontier-act-trahan-obernolte","url":"https://postcutoff.com/e/2026-07-23-frontier-act-trahan-obernolte/","date":"2026-07-23","date_precision":"day","short_title":"Reps. Trahan and Obernolte introduce the bipartisan FRONTIER Act","deck":"Licensed independent auditors, incident reporting and state preemption for frontier AI","takeaway":"It is the House counterpart to the Senate's duty-of-care talks and the industry-favoured model (third-party audits plus preemption of state laws such as California SB 53).","category":"policy-safety","category_label":"Policy & safety","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":5,"official":1,"filed":"2026-10-03","updated":"2026-10-03","orgs":["US Congress"]}],"people":[{"id":"gavin-newsom","name":"Gavin Newsom","url":"https://postcutoff.com/person/gavin-newsom/"},{"id":"zvi-mowshowitz","name":"Zvi Mowshowitz","url":"https://postcutoff.com/person/zvi-mowshowitz/"}],"posts":[{"id":"2026-09-12-darioamodei-pace-the-frontier-tweet","title":"Dario Amodei announces essay \"We Must Pace the Frontier\"","url":"https://postcutoff.com/p/2026-09-12-darioamodei-pace-the-frontier-tweet/"},{"id":"2026-09-12-darioamodei-we-must-pace-the-frontier","title":"We Must Pace the Frontier","url":"https://postcutoff.com/p/2026-09-12-darioamodei-we-must-pace-the-frontier/"}],"videos":[],"models":[],"changes":[{"date":"2026-10-13","type":"edit","text":"The Reuters exclusive was read through its syndicated text; replaced the unverified headline note with the reported facts and linked the new entry"},{"date":"2026-10-13","type":"edit","text":"Noted the Reuters exclusive on labs seeking control over what outside evaluators publish (headline summary only; article unread)"},{"date":"2026-10-11","type":"edit","text":"Added CNBC (Oct 11) on the evaluator ecosystem: METR/Vals funding, AI Evaluator Forum letter, OpenAI contracts, critics"},{"date":"2026-09-29","type":"edit","text":"Added Zvi analysis link"},{"date":"2026-09-29","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-10-13","human_review":null,"version":{"date":"2026-10-13"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":141,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":80,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":171,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":110,"in_training_data":false}],"short_url":null}