As of: 2026-11-22 13:32 CET. Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/ Canonical page: https://postcutoff.com/v/chase-ai-the-sol-6-1-benchmarks-are-stupid-so-i-tested-it-vs-sonnet-5/ # The Sol 6.1 Benchmarks Are STUPID, So I Tested It vs Sonnet 5.5 Chase AI, 29 September 2026, YouTube. 172,944 views as of 22 November 2026. Kind: Review. Watch: https://www.youtube.com/watch?v=pVAxEpP79v0 ## Description (written by Gemini from the video) **Summary** Chase AI reviews OpenAI's newly announced GPT-6.1 Sol following OpenAI DevDay, analyzing its benchmark scores and head-to-head performance against Anthropic's Claude Sonnet 5.5 across four real-world coding and generation challenges. Through side-by-side evaluations of automated canvas explainer videos, frontend web design, interactive 3D web graphics, and a browser-based 3D tank game, the presenter evaluates whether GPT-6.1 Sol's dramatic token efficiency and lower task costs offset Claude Sonnet 5.5's advantages in speed and visual design execution. **What is shown** * **[00:00] OpenAI DevDay announcement and benchmark slides**: OpenAI's launch page for GPT-6.1 Sol, showing DeepSWE cost vs. score graphs and OSWorld 2.0 offline set comparisons between GPT-6.1 Sol, GPT-6 Sol, and GPT-6 Astra. * **[01:13] Benchmark breakdown**: Artificial Analysis index comparisons across Intelligence Index, Finance & Accounting, Strategy & Ops, Legal, Healthcare & Medical, Engineering, and Economics, alongside lab benchmarks (DeepSWE v1.1 and AutomationBench 1.0.6) and effort-level task costs. * **[03:23] Test 1 — JavaScript Explainer Video**: Prompting both models to generate a self-contained 15-second vertical animated canvas video explaining Claude subagents with synchronized audio. GPT-6.1 Sol's output is reviewed ([03:48]), followed by Claude Sonnet 5.5's output ([04:20]). * **[04:56] Test 2 — Landing Page Design ("Dune House")**: Prompting both models to create a responsive landing page for a fictional boutique hotel named "Dune House". GPT-6.1 Sol generated images natively ([06:01]), while Claude Sonnet 5.5 generated images via a Higgsfield MCP integration ([05:08], [07:17]). * **[08:28] Test 3 — 3D Interactive Travel Globe**: Testing Three.js interactive data visualizations. GPT-6.1 Sol created "ORBIT" ([08:28]), an interactive 3D globe with flight routes, while Claude Sonnet 5.5 produced "MERIDIAN" ([09:33]), featuring audio, cinematic fly-in camera animations, and destination cards. * **[10:55] Test 4 — 3D Browser Game**: Prompting both models to clone a *World of Tanks*-style game using Three.js. GPT-6.1 Sol produced "IronFront: Armored Warfare" ([11:04]), and Claude Sonnet 5.5 generated "Iron Vanguard" ([12:06]), both showing playable tanks, aiming, HUDs, and destructible elements. **Claims & numbers** * The presenter says GPT-6.1 Sol was announced at OpenAI DevDay immediately following the underwhelming release of GPT-6. * On DeepSWE v1.1, the presenter states GPT-6.1 Sol scored 75.2% (High effort, $0.65 cost per task), beating GPT-6 Astra's 73.2% (Max effort, $7.50 cost per task) and Sonnet 5.5's score by +4.2 points. * On AutomationBench 1.0.6, the presenter reports Sonnet 5.5 leads Sol by +8.7 points (44.8% max with fallbacks vs. 36.1% max). * On token API list pricing, both models share identical rates: $2 per million input tokens and $10 per million output tokens. * The presenter claims Sonnet 5.5 generates tokens roughly twice as fast as GPT-6.1 Sol (138 tokens/second vs. 69 tokens/second). * The presenter shows Artificial Analysis cost-per-task data indicating Sol is up to 10x cheaper at maximum effort ($0.72 vs. $7.60), 7x cheaper at extra high ($0.39 vs. $2.74), 3.4x cheaper at high ($0.32 vs. $1.08), and 2.8x cheaper at medium ($0.21 vs. $0.59); when normalized to 52% benchmark accuracy, Sol was 4x cheaper. * The presenter states OpenAI recently slashed usage allowances on its $200/month plan by 50% (from 20x to 10x), while noting Anthropic's 20x tier had a 5-hour rolling limit rather than an uncapped weekly allowance. * Explainer video test token usage: Sol used 70,000 tokens versus Sonnet 5.5's ~300,000 tokens. * Landing page test token usage: Sol used 109,000 tokens versus Sonnet 5.5's ~200,000 tokens. * 3D globe test token usage: Sol used ~150,000 tokens versus Sonnet 5.5's ~400,000 tokens. * 3D tank game test: Sol took over 2 hours and ~350,000 tokens; Sonnet 5.5 took ~1 hour and ~700,000 tokens. **Notable quotes** * **[02:39]**: "Basically, Sol: super, super, super token-efficient." * **[07:05]**: "Which probably is a good thing, because this model is so dang cheap that if this is giving me Astra performance, what's there to complain about?" * **[13:28]**: "6.1 Sol kind of feels like just a slight step below Sonnet 5.5, yet it's way more efficient with tokens. It's cheaper, but it's slower." **Assessment** This is a hands-on review and comparative benchmark demonstration by an independent AI creator. The presenter directly demonstrates genuine browser-rendered outputs (HTML5 canvas, Three.js applications, and front-end layouts) and provides full transparency regarding token consumption, runtime, and qualitative shortcomings for both models. _Described by gemini-3.8-flash on 2026-10-01 from the video's audio and frames._ ## How AI was used to make it AI made a substantial part of this video (Subject built by AI). - demonstrating browser-based 3D games, Three.js graphics, landing pages, and canvas animations generated by AI models Tools: GPT-6.1 Sol, Claude Sonnet 5.5, Higgsfield MCP (Classified by gemini-3.8-flash from our description; high confidence. Overview: https://postcutoff.com/ai-video/) ## Related - 2026-09-29: [OpenAI releases GPT-6.1 Sol](https://postcutoff.com/e/2026-09-29-gpt-6-1-sol/) - 2026-09-28: [Anthropic releases Claude Sonnet 5.5](https://postcutoff.com/e/2026-09-28-claude-sonnet-5-5/) ## More videos on this - 2026-09-29: ["Build what you think you look like": Claude Sonnet 5.5 vs GPT-6 Astra](https://postcutoff.com/v/siyabuilt-sonnet-5-5-vs-gpt-6-astra-what-you-look-like-x/), Siya (@siyabuilt), X, made by AI, 7,259,229 views as of 27 October 2026. - 2026-09-28: [Introducing Claude Sonnet 5.5](https://postcutoff.com/v/claude-introducing-sonnet-5-5/), Claude, YouTube, official, 4,403,927 views as of 27 October 2026. - 2026-09-28: [Fall foliage simulator made with Sonnet 5 vs Sonnet 5.5 (official @claudeai launch-thread video)](https://postcutoff.com/v/claudeai-sonnet-5-5-fall-foliage-simulator-x/), Claude (@claudeai), X, made by AI, 755,645 views as of 26 October 2026. - 2026-09-29: [Live from OpenAI DevDay 2026: Keynote](https://postcutoff.com/v/openai-devday-2026-keynote/), OpenAI, YouTube, official, 685,790 views as of 24 October 2026. What else happened that month, day by day: https://postcutoff.com/timeline/2026/09/