---
id: "2026-10-29-openconjecture-random-conjectures-gpt-models-pnnl"
url: "https://postcutoff.com/e/2026-10-29-openconjecture-random-conjectures-gpt-models-pnnl/"
as_of: "2026-10-30T13:42:00+01:00"
date: "2026-10-29"
date_precision: day
category: research
importance: 3
confidence: high
status: [Event confirmed, Awaiting review]
verification: Preprint only
sources: 1
editor: human editor
human_review: null
version: "2026-10-30"
---

As of: 2026-10-30 13:42 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2026-10-29-openconjecture-random-conjectures-gpt-models-pnnl/

# OpenConjecture: GPT models on 200 random arXiv conjectures

Full title: OpenConjecture (PNNL): GPT models propose resolutions for 125 of 200 randomly sampled arXiv conjectures, but many counterexamples are 'off-the-shelf' or exploit gaps in the statement

arXiv 2610.40382 (29 Oct 2026), "Beyond Selected Breakthroughs: Evaluating Language Models on Randomly Sampled Mathematical Conjectures" by Helen Jenne, Davis Brown, Jesse He and Henry Kvinge (PNNL, UC San Diego, University of Pennsylvania, University of Washington), introduces OpenConjecture, a dataset of conjectures from arXiv maths papers since February 2026. On a random sample of 200, three solver passes produced candidate resolutions for 125. The paper analyses the novelty and fidelity of these candidates rather than their correctness, and warns that the counts are not success rates.

## Key facts
- Protocol (paper): up to three sequential passes, 'GPT-5.6 Sol in the first pass, GPT-5.6 Sol Pro in the second, and GPT-6 Astra Pro in the third', reasoning effort xhigh; only conjectures judged 'unresolved' went to the next pass; no problem-specific mathematical guidance
- Counts (paper): 'the three passes produced 70 counterexamples and 55 proofs to the 200 conjectures'; counterexamples 49/15/6 and proofs 33/11/11 by pass; '84% of these proposed resolutions hold up to LLM review'
- Quality (paper): 'of the proposed counterexamples from the first and second attempts that passed an LLM verification review, roughly 60% were classified as "off-the-shelf" and the remainder as "standard constructions"'; 'Some of the counterexamples appeared to take advantage of missing hypotheses in the conjecture statement'; the abstract cites 'proposing the trivial group as a counterexample' and says 'a number of proofs follow closely from existing results, but that some resolutions exhibit at least moderate novelty'
- Limitation (paper, verbatim): 'Our results discuss candidate resolutions and outcomes of LLM review, not expert-verified solutions. Each proposed resolution was reviewed by a new instance of the same model that produced it … The reported numbers … should not be interpreted as rates' of correct solutions
- Framing (introduction): cites the Erdős unit-distance disproof, the Jacobian conjecture counterexample, OpenAI's Navier–Stokes blowup claim, First Proof and the Leipzig benchmarks as the selected breakthroughs it wants to complement

## What happened

Most AI-maths news concerns problems that labs or mathematicians chose because a model succeeded. The PNNL group instead collected
conjectures stated in recent arXiv papers and drew 200 at random. Three passes of OpenAI models produced claimed answers for 125 of them.

The authors did not check correctness. They studied what kind of answers came back: how long they were, whether they cited prior work,
whether they matched what the conjecture's authors meant, and how novel they looked.

## Why it matters

It gives a baseline for the flood of AI-assisted conjecture papers we list daily. Many conjectures in recent papers fall to standard
tools or to loopholes in how they are stated, so a count of "conjectures resolved" says little without that classification.

## What is disputed or not yet verified

- Verification: Preprint; candidates checked only by LLM review

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 182 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 121 days after its cutoff
- Gemini 4 Argon (training cutoff March 2026): 212 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 151 days after its cutoff

## Sources

1. [arXiv 2610.40382: Beyond Selected Breakthroughs: Evaluating Language Models on Randomly Sampled Mathematical Conjectures](https://arxiv.org/abs/2610.40382) (arxiv.org, paper)

## Changes

- 2026-10-30 (filed): Created (science agent; arXiv abstract and PDF read, numbers quoted from the paper)

## Related

- 2026-10-30: [AI-assisted results, 30 Oct: arXiv announcement of 29–30 Oct](https://postcutoff.com/e/2026-10-30-ai-assisted-results-october-30-listing/index.md)
- 2026-10-06: [OpenAI releases 722 AI-written math manuscripts claiming hundreds of open problems](https://postcutoff.com/e/2026-10-06-openai-math-release-722-manuscripts/index.md)