{"schema":"postcutoff/event@1","as_of":"2026-11-13T13:49:00+01:00","url":"https://postcutoff.com/e/2026-08-24-artificial-analysis-speech-agent-arena/","md":"https://postcutoff.com/e/2026-08-24-artificial-analysis-speech-agent-arena/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-08-24-artificial-analysis-speech-agent-arena","date":"2026-08-24","date_precision":"day","short_title":"Artificial Analysis launches the Speech Agent Arena for speech-to-speech voice agents","deck":null,"takeaway":"On 2026-08-24 Artificial Analysis launched the Speech Agent Arena, where people hold live conversations with two hidden speech-to-speech models across 15 agentic (tool-calling) and 20 non-agentic scenarios, then vote.","category":"benchmark","category_label":"Benchmarks","importance":2,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"Artificial Analysis: Announcing the Speech Agent Arena","url":"https://artificialanalysis.ai/articles/announcing-the-speech-agent-arena","type":"official","group":"primary","domain":"artificialanalysis.ai"},{"n":2,"title":"Artificial Analysis on X: Controlled Voice Arena announcement","url":"https://x.com/ArtificialAnlys/status/2074886571166462405","type":"official","group":"primary","domain":"x.com"},{"n":3,"title":"Artificial Analysis Controlled Voice leaderboard","url":"https://artificialanalysis.ai/text-to-speech/leaderboard/controlled-voice","type":"official","group":"primary","domain":"artificialanalysis.ai"},{"n":4,"title":"Artificial Analysis on X: Multilingual TTS Arena leaderboards","url":"https://x.com/ArtificialAnlys/status/2102490340678856997","type":"official","group":"primary","domain":"x.com"}],"official":4,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Artificial Analysis"],"title":"Artificial Analysis launches the Speech Agent Arena for speech-to-speech voice agents","summary":"On 2026-08-24 Artificial Analysis launched the Speech Agent Arena, where people hold live conversations with two hidden speech-to-speech models across 15 agentic (tool-calling) and 20 non-agentic scenarios, then vote. It reports a preference Elo plus a task-success rate. At launch Gemini 3.1 Flash Live Preview led on preference, and Grok Voice Think Fast 2.0 led on task success (94.7%). It joined AA's 2026 voice leaderboards, which also include the Controlled Voice TTS arena (July 2026) and multilingual TTS arenas (Sept 2026).","key_facts":["Method: pairwise human votes after separate live conversations → Preference Elo; agentic task success = share of eligible conversations completed with the correct final tool call(s)","Launch preference Elo: Gemini 3.1 Flash Live Preview (Minimal) 1,046; Gemini 3.1 Flash Live Preview (High) 1,014; OpenAI GPT-Realtime-1.5 1,000","Launch task success: Grok Voice Think Fast 2.0 (High) 94.7%; OpenAI GPT-Realtime-2.1 (High) 91.5%","Controlled Voice Arena (announced 2026-07-08): TTS models compared on the same 8 cloned voices (US/UK, male/female). Initial leader Cartesia Sonic 3.5 (1,122), then Eleven v3 (1,088), Inworld Realtime TTS-2 (1,070)","Multilingual TTS arenas for 9 languages beyond English announced 2026-09-22"],"key_numbers":[],"tags":["voice","speech-to-speech","voice-agents","leaderboard","tool-use"],"science":null,"body_md":"## What happened\n\nArtificial Analysis, the independent benchmarking firm, added an arena for end-to-end voice agents. Earlier speech arenas (TTS preference, STT WER) scored single components.\nThe Speech Agent Arena scores whole conversations with speech-to-speech models, including whether the agent actually did the requested action through tool calls.\n\n## Why it matters\n\nVoice agents are being sold into customer service, where finishing the task matters more than sounding natural. The two launch leaderboards disagree: the preferred-sounding model is not the most reliable one. That split is now measured in public.\nLaunch rankings were taken from AA's article and will change as models are added.","disputed":[],"related":[{"id":"2026-09-28-elevenlabs-eleven-v4","url":"https://postcutoff.com/e/2026-09-28-elevenlabs-eleven-v4/","date":"2026-09-28","date_precision":"day","short_title":"ElevenLabs launches Eleven v4 and Eleven v4 Turbo, #1 on Artificial Analysis TTS arena","deck":null,"takeaway":null,"category":"model-release","category_label":"Model releases","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":11,"official":7,"filed":"2026-09-29","updated":"2026-09-29","orgs":["ElevenLabs"]},{"id":"2026-08-27-cartesia-sonic-3-6","url":"https://postcutoff.com/e/2026-08-27-cartesia-sonic-3-6/","date":"2026-08-27","date_precision":"day","short_title":"Cartesia Sonic-3.6 goes GA and tops the Artificial Analysis Speech Arena","deck":null,"takeaway":"Cartesia made Sonic-3.6 generally available on 2026-08-27 (beta 2026-08-17), three months after Sonic-3.5.","category":"model-release","category_label":"Model releases","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":4,"official":4,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Cartesia"]},{"id":"2026-07-29-grok-voice-think-fast-2","url":"https://postcutoff.com/e/2026-07-29-grok-voice-think-fast-2/","date":"2026-07-29","date_precision":"day","short_title":"xAI releases Grok Voice Think Fast 2.0 speech-to-speech model for voice agents","deck":null,"takeaway":"Its reported AA S2S Quality Index (82.9) put it roughly level with Google's Gemini 3.8 Live Extended Thinking (82.6, Sept 2026) and marked xAI's push to compete on voice agents on price.","category":"model-release","category_label":"Model releases","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":4,"official":4,"filed":"2026-09-29","updated":"2026-09-29","orgs":["xAI","SpaceX"]}],"people":[],"posts":[{"id":"x-artificialanlys-2102490340678856997","title":"“Announcing Multilingual Text to Speech Arena Leaderboards, comparing leading TTS models…”","url":"https://postcutoff.com/p/x-artificialanlys-2102490340678856997/"},{"id":"x-artificialanlys-2074886571166462405","title":"“Announcing the Controlled Voice Arena Leaderboard comparing Text to Speech models on the…”","url":"https://postcutoff.com/p/x-artificialanlys-2074886571166462405/"}],"videos":[],"models":[],"changes":[{"date":"2026-09-29","type":"filed","text":"Created (also covers the Controlled Voice and multilingual TTS arenas)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-09-29","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":116,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":55,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":146,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":85,"in_training_data":false}],"short_url":null}