Artificial Analysis launches the Speech Agent Arena for speech-to-speech voice agents
Confirmed
The takeaway
On 2026-08-24 Artificial Analysis launched the Speech Agent Arena, where people hold live conversations with two hidden speech-to-speech models across 15 agentic (tool-calling) and 20 non-agentic scenarios, then vote.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 29 September 2026
Your AI and this story
- GPT-6 Astra116 days after its cutoff
- Claude Opus 5.555 days after its cutoff
- Gemini 4 Hydrogen146 days after its cutoff
- Grok 4.785 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 55 days before it.
Key facts
- Method: pairwise human votes after separate live conversations → Preference Elo; agentic task success = share of eligible conversations completed with the correct final tool call(s)
- Launch preference Elo: Gemini 3.1 Flash Live Preview (Minimal) 1,046; Gemini 3.1 Flash Live Preview (High) 1,014; OpenAI GPT-Realtime-1.5 1,000
- Launch task success: Grok Voice Think Fast 2.0 (High) 94.7%; OpenAI GPT-Realtime-2.1 (High) 91.5%
- Controlled Voice Arena (announced 2026-07-08): TTS models compared on the same 8 cloned voices (US/UK, male/female). Initial leader Cartesia Sonic 3.5 (1,122), then Eleven v3 (1,088), Inworld Realtime TTS-2 (1,070)
- Multilingual TTS arenas for 9 languages beyond English announced 2026-09-22
What happened
Artificial Analysis, the independent benchmarking firm, added an arena for end-to-end voice agents. Earlier speech arenas (TTS preference, STT WER) scored single components. The Speech Agent Arena scores whole conversations with speech-to-speech models, including whether the agent actually did the requested action through tool calls.
Why it matters
Voice agents are being sold into customer service, where finishing the task matters more than sounding natural. The two launch leaderboards disagree: the preferred-sounding model is not the most reliable one. That split is now measured in public. Launch rankings were taken from AA’s article and will change as models are added.
Sources
4 sources from 2 sites. Numbers match the chips in the text.
4 sources: 4 primary
Primary
- Artificial Analysis: Announcing the Speech Agent Arenaartificialanalysis.ai, official
- Artificial Analysis on X: Controlled Voice Arena announcementx.com, official
- Artificial Analysis Controlled Voice leaderboardartificialanalysis.ai, official
- Artificial Analysis on X: Multilingual TTS Arena leaderboardsx.com, official
Changes
- Filed (also covers the Controlled Voice and multilingual TTS arenas)