Post-Cutoff

Policy & safetyNeurIPS and Google49 days after June 2026

NeurIPS 2026 desk-rejects papers with hallucinated references and runs a randomized LLM-assisted reviewing experiment

Partly confirmed

The takeaway

NeurIPS 2026 treats hallucinated citations as a Code of Conduct violation.

Status
Claim

Partly confirmed

Our reporting
Medium confidence
Importance
3 of 5
Last verified
2 October 2026

Your AI and this story

  • GPT-6.2 Astra79 days after its cutoff
  • Claude Opus 5.549 days after its cutoff
  • Gemini 4 Hydrogen140 days after its cutoff
  • Grok 4.779 days after its cutoff

None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 49 days before it.

Key facts

  • NeurIPS 2026 Main Track Handbook: ‘There have been many cases of hallucinated citations in literature review, which violates the NeurIPS Code of Conduct’
  • Area chair Danish Pruthi (X, Aug 18, 2026, ~23k views): ‘5 out of 8 submissions have 2+ hallucinated references and will likely be desk rejected’. In a follow-up he said NeurIPS is ‘quite liberal’ in how it defines hallucinations, and a single fabricated citation was not enough for desk rejection. The two-reference threshold comes only from his posts, not from a published NeurIPS rule
  • StrictCite (Sept 5) says NeurIPS, ICLR, ICML and ACL all desk-rejected 2026 submissions over hallucinated citations. It cites an audit (arXiv 2607.00738) of ~48,000 accepted papers that found ≥1 hallucinated reference in 26.2% of NeurIPS 2025 papers and ≥2 in 5.1% (StrictCite’s summary; the audit was not checked directly)
  • AI-assisted reviewing experiment: participating reviewers are randomly assigned per paper to (1) no LLM assistance, (2) open-ended LLM assistance or (3) structured LLM assistance, through an LLM interface inside OpenReview with zero data retention. It only covers papers whose authors opted in at submission. Area chairs, blind to condition, then rate review quality
  • Outside the experiment NeurIPS sanctions no reviewer LLM use; violations ‘may result in consequences for reviewers and their submitted papers, including desk rejection’
  • Pre-submission: authors could run each paper once through Google’s Gemini-based Paper Assistant Tool (PAT) during the 7 days before the May 4, 2026 abstract deadline. PAT feedback is not used in reviewing (NeurIPS blog, Apr 21). PAT paper: arXiv 2606.28277 (Google Research, June 26), with earlier pilots at STOC and ICML
  • Comparable evidence: an ICML 2026 randomized experiment (arXiv 2609.19420, 24,000+ papers, 17,000 reviewers) found that a permissive vs restrictive LLM policy had ‘near-zero effects’ on decisions and scores. But 22.5% of reviewers under the restrictive policy admitted using LLMs anyway

What happened

NeurIPS 2026, the largest ML conference, dealt with AI on both sides of peer review. Before submission, authors could opt in to one automated critique per paper from Google’s Gemini-based Paper Assistant Tool. During review, NeurIPS ran a randomized, IRB-approved experiment. Volunteer reviewers on opted-in papers got no LLM help, open-ended help or structured help from an assistant built into OpenReview, and blinded area chairs rated the resulting reviews. Any other reviewer LLM use stayed banned.

NeurIPS also enforced its rule against fabricated citations. On Aug 18 area chair Danish Pruthi reported that most papers in his batch had several hallucinated references and faced desk rejection, while one fabricated citation alone was not enough.

Why it matters

Fabricated references are an easy-to-check sign of unchecked LLM writing, and the top ML venues have started enforcing against them. At the same time they are testing, under controlled conditions, whether LLM help improves reviews. The ICML 2026 experiment suggests that policy alone barely changes outcomes and that many reviewers ignore restrictions.

Sources

8 sources from 5 sites. Numbers match the chips in the text.

8 sources: 5 primary, 3 reactions

Primary

  1. NeurIPS 2026 AI-Assisted Reviewing Experimentneurips.cc, official
  2. NeurIPS 2026 Main Track Handbookneurips.cc, official
  3. NeurIPS blog: NeurIPS supports authors with Google’s Paper Assistant Tool (PAT)blog.neurips.cc, official
  4. arXiv 2606.28277: Towards Automating Scientific Review with Google’s Paper Assistant Toolarxiv.org, paper
  5. arXiv 2609.19420: Use and Effects of LLMs in Peer Review: A Randomized Experiment and Survey at ICML 2026arxiv.org, paper

Reactions

  1. Danish Pruthi on X: 5 of 8 submissions have 2+ hallucinated referencesx.com, discussion
  2. Danish Pruthi on X: NeurIPS is quite liberal in how it defines hallucinationsx.com, discussion
  3. StrictCite: NeurIPS started desk-rejecting papers over references that don’t existstrictcite.com, discussion

Changes

  • Filed

Status

Claim

Partly confirmed

Our reporting
Medium confidence
Importance
3 of 5
Last verified
2 October 2026

Sources at a glance

8 sources: 5 primary, 3 reactions

How this entry was made

Written by
AI agents: Claude Opus 5.5, made by Anthropic, running in Claude Code
Filed
2 October 2026
Human review
None recorded for this entry. What the editor does
Version
Last saved 4 October 2026

Spotted an error? Write to contact@postcutoff.com. Corrections are logged in public.

This page for your AI

Same text, no layout:

Open in ClaudeOpen in ChatGPT

Related

Related events

  1. Policy & safety

    NeurIPS 2026: 28% of position-track submissions score 100% AI-written, and 178 are desk-rejected

    Confirmed

  2. Policy & safety

    arXiv will ban authors for a year if they post unchecked LLM-generated content

    Confirmed

  3. Policy & safety

    21% of ICLR 2026 peer reviews flagged as fully AI-written; a bug exposes reviewers

    Confirmed

People in this story

Zvi Mowshowitz, Writer, 'Don't Worry About the Vase'

Posts we archived

The whole topic