Cloudflare–OpenAI discovery pilot results
Confirmed
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 16 October 2026
Your AI and this story
- GPT-6 Astra168 days after its cutoff
- Claude Opus 5.5107 days after its cutoff
- Gemini 4 Argon198 days after its cutoff
- Grok 4.7137 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 107 days before it.
Key facts
- Pilot announced July 8, 2026 (Business Wire); results post Oct 15, bylined by Jack Galilee (Cloudflare) and Henry Aspegren and James Balfour (OpenAI)
- Cohort: about 1% of Cloudflare’s network; outcome: OpenAI discovered and indexed ‘hundreds of millions’ of new pages; no crawl-reduction figures given
- Signals: filtering of non-existent-file requests, catch-all redirects and duplicates; change signals for when to revisit; page-level harm flags
- Applies to sites ‘that choose to make their content available to AI systems’; a signal does not grant access permission; Cloudflare’s AI Crawl Control still governs access
- July quotes: Matthew Prince: ‘By sharing our sophisticated network signals, we can find a better way to make AI search more efficient’; OpenAI’s Nick Ryder: the pilot explores ‘whether network-level insights can help us discover content more efficiently’
What happened
Cloudflare passes metadata it sees as a network operator to OpenAI’s search crawler. This lets OpenAI skip junk URLs, revisit pages when they change and avoid individual malicious pages. The pilot covers about 1% of Cloudflare’s network. The July 8 announcement was missed at the time and is summarised here.
Why it matters
Cloudflare spent the previous year building tools to block or charge AI crawlers. It is now helping one AI company crawl more efficiently, on the condition that sites choose to let AI systems in. Discovery and freshness give AI search an edge, and a network operator with this view of the web is a partner other AI search engines may also want.
Sources
3 sources from 3 sites. Numbers match the chips in the text.
3 sources: 2 primary, 1 press
Primary
- Cloudflare blog: Cloudflare and OpenAI: Making AI discovery better for everyoneblog.cloudflare.com, official
- Barchart (Business Wire): Cloudflare announces research pilot with OpenAI (July 8)barchart.com, official
Press
Changes
- Filed