{"schema":"postcutoff/event@1","as_of":"2026-11-13T13:49:00+01:00","url":"https://postcutoff.com/e/2025-03-12-sakana-ai-scientist-v2-peer-review/","md":"https://postcutoff.com/e/2025-03-12-sakana-ai-scientist-v2-peer-review/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2025-03-12-sakana-ai-scientist-v2-peer-review","date":"2025-03-12","date_precision":"day","short_title":"Sakana's AI Scientist-v2 writes the first fully AI-generated paper to pass peer review","deck":null,"takeaway":"On 12 Mar 2025 Sakana AI reported that a paper generated end-to-end by The AI Scientist-v2 (idea, code, experiments, analysis, writing) scored 6, 7, 6 at an ICLR 2025 workshop, above the acceptance threshold; it was withdrawn by prior agreement.","category":"science","category_label":"Science & math","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Result confirmed"]},"sources":[{"n":1,"title":"Sakana AI: The AI Scientist generates its first peer-reviewed scientific publication","url":"https://sakana.ai/ai-scientist-first-publication/","type":"official","group":"primary","domain":"sakana.ai"},{"n":2,"title":"Sakana AI: The AI Scientist published in Nature","url":"https://sakana.ai/ai-scientist-nature/","type":"official","group":"primary","domain":"sakana.ai"},{"n":3,"title":"The AI Scientist (v1) paper, arXiv 2408.06292","url":"https://arxiv.org/abs/2408.06292","type":"paper","group":"primary","domain":"arxiv.org"},{"n":4,"title":"Nature news on the AI Scientist paper","url":"https://www.nature.com/articles/d41586-026-00899-w","type":"press","group":"press","domain":"nature.com"}],"official":3,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Sakana AI","University of British Columbia","University of Oxford"],"title":"Sakana's AI Scientist-v2 writes the first fully AI-generated paper to pass peer review (ICLR 2025 workshop)","summary":"On 12 Mar 2025 Sakana AI reported that a paper generated end-to-end by The AI Scientist-v2 (idea, code, experiments, analysis, writing) scored 6, 7, 6 at an ICLR 2025 workshop, above the acceptance threshold; it was withdrawn by prior agreement. The system and its limits were later published in Nature (26 Mar 2026).","key_facts":["Workshop: ICLR 2025 'I Can't Believe It's Not Better' (ICBINB); reviewer scores 6, 7, 6 (avg 6.33), higher than ~55% of human-written submissions","Reviewers knew some submissions might be AI-generated but not which; the paper was withdrawn after review as agreed with organisers","Workshop acceptance, not a main-conference paper; the result was a negative result on compositional regularisation","Nature paper (2026): automated reviewer reached 69% balanced accuracy; paper quality rises with the underlying model","Admitted weaknesses: naive ideas, weak rigour, hallucinated citations"],"key_numbers":[],"tags":["ai-scientist","autonomous-research","peer-review","agents"],"science":{"field":"computer-science","subfield":"machine learning / automated research","problem":"Can an AI system autonomously produce a research paper that passes human peer review?","result":"A fully AI-generated ML paper passed peer review at an ICLR 2025 workshop (scores 6/7/6), a first for end-to-end AI-authored research.","open_since":"","ai_system":["The AI Scientist-v2"],"human_role":"Humans chose the broad topic and selected 3 of the generated papers to submit; no human edits to the paper","verification":{"key":"none","label":"Unverified claim","text":"Blind peer review at an ICLR workshop; system described in Nature (2026)"},"status":"confirmed","shock":"A paper with no human-written text beat more than half of the human submissions in blind review."},"body_md":"## What happened\n\nSakana AI, with UBC and Oxford collaborators, submitted three papers written entirely by The AI Scientist-v2 to an ICLR 2025 workshop with the organisers' consent. One received scores of 6, 7 and 6 — above the acceptance bar — and was withdrawn before publication, as planned, because norms for AI-authored papers did not exist. The first version of the system had been released in August 2024; a peer-reviewed description appeared in Nature on 26 March 2026.\n\n## Why it matters\n\nIt was the first demonstration that a fully automated pipeline could clear human peer review, even at a workshop with a higher acceptance rate than main tracks. It set off debate about AI-generated papers flooding venues, which later led to arXiv and conference policy changes.","disputed":[],"related":[{"id":"2025-10-22-agents4science-conference","url":"https://postcutoff.com/e/2025-10-22-agents4science-conference/","date":"2025-10-22","date_precision":"day","short_title":"Agents4Science 2025, the first conference where AI must be first author and reviewer","deck":null,"takeaway":"It made AI authorship an explicit, measurable experiment rather than a hidden practice, and produced data on the strengths and failure modes of AI reviewers.","category":"science","category_label":"Science & math","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Result confirmed"]},"sources":4,"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Stanford University","Together AI"]},{"id":"2025-05-27-intology-zochi-acl-2025","url":"https://postcutoff.com/e/2025-05-27-intology-zochi-acl-2025/","date":"2025-05-27","date_precision":"month","short_title":"Intology's 'Zochi' AI system gets a paper into the ACL 2025 main conference","deck":null,"takeaway":"It moved AI-authored research from workshop level (Sakana, March 2025) to a selective main track within months, though the degree of autonomy could not be independently audited.","category":"science","category_label":"Science & math","importance":3,"confidence":"medium","status":{"key":"confirmed","labels":["Result confirmed"]},"sources":3,"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Intology"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-09-29","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-09-29","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":null,"in_training_data":true},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":null,"in_training_data":true},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":null,"in_training_data":true},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":null,"in_training_data":true}],"short_url":"https://postcutoff.com/s/sakana-ai-scientist-v2-writes"}