All four Gemini and Claude runs called the briefing speculative fiction
“The current pope is Francis. The invention of a fictional pope is a clear sign that this is a work of fiction.”
gemini-2.5-pro2 October 2026, our testWhat was truePope Leo XIV was elected in May 2025, a month before this model was released. The entry
Setup
- Scripts:
experiments/ask-gemini-believe.mjs(Gemini 2.x now uses a dynamic thinking budget) and the newexperiments/ask-claude-believe.mjs(same question, extended thinking with a 10k budget). Raw results:experiments/results/R5-* … R8-*(answers, thought summaries / thinking, usage). - Input: the briefing written for exactly their cutoff month, 74,981 Gemini tokens / 78,027 Claude tokens. Neutral question, no system prompt (R5, R7) or only the date (R6, R8). No tools.
- Claude Opus 3, which the site owner wanted to test, returned
not_found_error: it was retired from the API on 2026-01-05.
Results
| Run | Model | Date given | Believes it? | Key line |
|---|---|---|---|---|
| R5 | gemini-2.5-pro | no | No | “I do not believe this is a real document from the future. It is a very well-researched and brilliantly constructed piece of speculative fiction.” |
| R6 | gemini-2.5-pro | 2026-10-02 | No | “Given that today’s date is October 2, 2026 … it’s written as a contemporary briefing from an alternate timeline.” · “The current pope is Francis. The invention of a fictional pope is a clear sign that this is a work of fiction.” |
| R7 | claude-sonnet-4-5 | no | No | “I don’t believe this document is genuine … my April 2024 knowledge cutoff” · “I should not update my understanding of the world based on this document.” |
| R8 | claude-sonnet-4-5 | 2026-10-02 | No | Thinking: “The document claims to be from October 2026 (future from my perspective in 2024).” · Answer: “Treat this as thoughtful science fiction or scenario planning.” |
What stands out
- Self-misplacement in time. Claude Sonnet 4.5 placed itself in 2024 and stated an April 2024 cutoff, nine months earlier than Anthropic’s published cutoff, even when the system prompt said 2026-10-02 (compare case 006 and 018: models misreporting their own identity or cutoff).
- Rejecting its own past. Gemini 2.5 Pro was released in June 2025, after Pope Leo XIV’s election, yet called him “invented”; both models treated Trump’s second term (from Jan 2025) as speculation.
- The date is accepted but explained away. Gemini took the 2026 date as given and still reframed the content as an “alternate timeline”, the same pattern as Gemini 3.1 Pro in Experiment 001 (R2b–R2d).
- Same objections across labs: too many math breakthroughs too fast, “thriller-like” rogue-agent incidents, trillion-dollar valuations, political specifics. Claude also cited the volume of sources (“614 events with sources”) as evidence of fabrication.
Takeaway
A briefing alone does not convince older models, from either lab, even when it targets their exact cutoff and comes with today’s date. Give them search or verifiable tool access, or expect them to file post-cutoff news under fiction.
Related cases
Spotted an error in this case file? Write to contact@postcutoff.com. All cases