---
id: "2026-10-14-design-creativity-bench-llm-diversity-collapse"
url: "https://postcutoff.com/e/2026-10-14-design-creativity-bench-llm-diversity-collapse/"
as_of: "2026-10-15T23:34:00+02:00"
date: "2026-10-14"
date_precision: day
category: benchmark
importance: 2
confidence: medium
status: [Partly confirmed]
sources: 3
editor: human editor
human_review: null
version: "2026-10-15"
---

As of: 2026-10-15 23:34 CEST.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2026-10-14-design-creativity-bench-llm-diversity-collapse/

# Design Creativity Bench finds LLM design 'diversity collapse'

Full title: Design Creativity Bench: frontier models' UI designs converge ('diversity collapse'); best model GPT-6 Astra scores 65.5 vs 92.3 for the human reference

Design Creativity Bench (arXiv 2610.11539, Rusia, Bhole, Gupta and Dey; v1 Oct 8, v2 Oct 14, 2026; hosted by Kombai) scores LLM-made UI designs on originality, creative range and appropriateness. Models are almost always appropriate (best 99.2%, humans 98.0%) but converge on similar designs: originality across models 0.592 vs 0.764 for human–model pairs, creative range 0.581 vs 0.902 for human designs. A companion essay on near-identical "pelican riding a bicycle" SVGs reached the Hacker News front page.

## Key facts
- Paper: arXiv 2610.11539, 'Design Creativity Bench: Measuring creativity in LLM-Generated UI' (Aman Rusia, Abhijit Bhole, Prashank Gupta, Dipanjan Dey); v1 Oct 8, v2 Oct 14, 2026
- Abstract: originality between different models on the same prompt 0.592 (95% CI 0.582–0.602) vs 0.764 for human–model pairs; creative range across models 0.581 vs 0.902 for human designs; appropriateness above 90% for every model, best 99.2% vs 98.0% for humans
- Leaderboard as embedded in Rusia's essay (Oct 14): human reference 92.3, GPT-6 Astra 65.5, GPT-6.1 Sol 63.7, Grok 4.7 62.2, Gemini 3.8 Flash 60.6; the essay says 16 frontier models on 80 open-ended UI tasks
- Essay: Aman Rusia, 'Why do LLM pelicans look like this?' (arcfu.com, Oct 14): thirteen models given Simon Willison's 'SVG of a pelican riding a bicycle' prompt produce nearly identical images, which he calls 'diversity collapse'; 163 points on Hacker News (item 50090443, Oct 15)
- Caveat: a benchmark from a company that sells AI UI-design tools; scores are not independently reproduced

## What happened

A four-author team published Design Creativity Bench on arXiv (2610.11539) to measure how creative LLM-generated user-interface designs
are, not just whether they are correct. The headline finding is that models are reliable but alike: different models answering the same
brief produce designs far more similar to each other than to human designs, and their range across briefs is much narrower.

On Oct 14 co-author Aman Rusia published an essay showing the same effect on Simon Willison's well-known pelican-on-a-bicycle SVG test,
where thirteen models drew nearly the same picture. It reached the Hacker News front page and promoted the benchmark's leaderboard, led
by GPT-6 Astra at 65.5 against a human reference of 92.3.

## Why it matters

Most 2026 benchmarks measure correctness. This one puts a number on a common complaint, that frontier models default to the same
"house style" in creative work, and finds a large gap to human designers.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 167 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 106 days after its cutoff
- Gemini 4 Argon (training cutoff March 2026): 197 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 136 days after its cutoff

## Sources

1. [arXiv 2610.11539: Design Creativity Bench: Measuring creativity in LLM-Generated UI](https://arxiv.org/abs/2610.11539) (arxiv.org, paper)
2. [Kombai: Design Creativity Bench leaderboard](https://kombai.com/benchmark) (kombai.com, official)
3. [Aman Rusia: Why do LLM pelicans look like this? (The emperor pelican has no clothes)](https://arcfu.com/post/the-emperor-pelican-has-no-clothes) (arcfu.com, discussion)

## Changes

- 2026-10-15 (filed): Created from data/leads.md (arXiv abstract and the arcfu.com essay read)

## Related

- 2026-09-03: [OpenAI releases GPT-6 Astra, its first GPT-6 model](https://postcutoff.com/e/2026-09-03-gpt-6-astra/index.md)
- People: [Simon Willison](https://postcutoff.com/person/simon-willison/)