Live source snapshot
Audio Realism Benchmark
Blind listening comparisons measure how human text-to-speech output sounds across phone-agent, conversational, and explainer speech.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 500 held-out American-English prompts; female and male voices; human recordings included in the comparison pool
- Primary metric
- Bradley–Terry Elo-style rating, win rate, and uncertainty
- Owner
- Design Arena
- Available evidence
- Machine-readable public leaderboard
What it measures
Voice experience
Is the exchange natural, robust, and responsive?
How the protocol works
- 01
Vetted native speakers hear two clips generated from the same prompt and select the one that sounds more human.
- 02
The benchmark uses 500 private, held-out prompts split across phone-agent, conversational, and explainer speech.
- 03
One female and one male American-English voice are selected for each model, and comparisons use the same gender.
- 04
Scores are fitted with a Bradley–Terry model and presented as an Elo-style rating; real human recordings remain in the comparison pool.
Available results
| Rank | Model | Elo-style rating | BT uncertainty | Win rate | Battles | Avg. generation |
|---|---|---|---|---|---|---|
| 1 | Bland Speech v3 Bland AI | 1356 | ±22.6 | 82.1% | 403 | 1.62s |
| 2 | MAI-Voice-2 Microsoft | 1223 | ±18.9 | 69.0% | 409 | 1.49s |
| 3 | Grok TTS xAI | 1168 | ±18.3 | 62.3% | 403 | 2.26s |
| 4 | Gemini 2.5 Pro TTS Preview Google | 1111 | ±17.5 | 55.4% | 419 | 6.91s |
| 5 | MiniMax Speech-02 HD MiniMax | 1098 | ±17.8 | 52.8% | 411 | 3.27s |
| 6 | Cartesia Sonic 3.5 Cartesia | 1089 | ±17.5 | 50.9% | 424 | 1.52s |
| 7 | Gemini 3.1 Flash TTS Preview Google | 1040 | ±17.7 | 41.7% | 417 | 4.91s |
| 8 | GPT-4o mini TTS OpenAI | 1014 | ±18.1 | 42.4% | 396 | 1.99s |
| 9 | Gemini 2.5 Flash TTS Preview Google | 992 | ±18.1 | 38.3% | 415 | 4.98s |
| 10 | Lightning v3.1 Pro Smallest AI | 955 | ±18.7 | 33.9% | 407 | 1.90s |
| 11 | ElevenLabs Eleven v3 ElevenLabs | 949 | ±18.8 | 30.8% | 409 | 3.25s |
| 12 | Inworld TTS-1.5 Max Inworld | 771 | ±23.3 | 14.8% | 413 | 4.94s |
Interpretation limit
The leaderboard measures perceived human-likeness for the selected American-English voices and prompt mix. It does not measure factual accuracy, task completion, full-duplex interaction, or multilingual quality.