Skip to main content

Benchmark profile

τ²-Bench Tool-Agent-User Evaluation (τ²-bench results)

This route is a sourced ledger for published τ²-bench results. Most current rows come from Artificial Analysis's telecom implementation, while named provider rows can use telecom, airline, retail, or aggregate setups.

Data verified

How to read this leaderboard

Editorial review by Glevd · 2026-07-15

Use a row only with its attached source and setup label. Match the domain, task release, agent model, user-simulator model, scaffold, prompts, trial count, and pass^k metric before comparing scores. The sorted table is a source ledger, not a controlled cross-provider ranking.

Operator receipt: 145 sourced rows are currently displayable on this page; the highest published score among these 145 models is GLM-5.2 at 99.1%, which does not establish a market leader.

Honest limit: The page mixes a large third-party telecom snapshot with smaller provider-published slices. Those sources do not use one guaranteed-common harness or reporting policy, and BenchLM did not rerun them. A higher number can reflect a different user model, prompt, domain, task release, or repeat policy.

Benchmark score on τ²-bench sourced results — July 23, 2026

BenchLM mirrors the published score view for τ²-bench sourced results. The public snapshot contains 145 models. GLM-5.2 has the highest published score at 99.1%, but the available coverage and evaluation setups do not establish a market leader. BenchLM does not use these results to rank models overall.

145 modelsAgenticCurrentDisplay onlyUpdated July 23, 2026

Benchmark score table (145 models)

Score
1
GLM-5.2Z.AI · Open weightArtificial Analysis τ²-bench
99.1%
2
GLM-4.7-FlashZ.AI · Open weightArtificial Analysis τ²-bench
98.8%
3
Claude Fable 5Anthropic · ClosedArtificial Analysis τ²-bench
98.5%
4
Step 3.7 FlashStepFun · Open weightArtificial Analysis τ²-bench
98.5%
5
GLM-5-TurboZ.AI · ClosedArtificial Analysis τ²-bench
98.5%
6
GLM-5V-TurboZ.AI · ClosedArtificial Analysis τ²-bench
98.5%
7
GLM-5Z.AI · Open weightArtificial Analysis τ²-bench
98.2%
8
GLM-5.1Z.AI · Open weightArtificial Analysis τ²-bench
97.7%
9
Qwen3.6 PlusAlibaba · ClosedArtificial Analysis τ²-bench
97.7%
10
Grok 4.3xAI · ClosedArtificial Analysis τ²-bench
97.7%
11
DeepSeek V4 Pro (Max)DeepSeek · Open weightArtificial Analysis τ²-bench
96.2%
12
Kimi K2.6Moonshot AI · Open weightArtificial Analysis τ²-bench
95.9%
13
Kimi K2.5Moonshot AI · Open weightArtificial Analysis τ²-bench
95.9%
14
GLM-4.7Z.AI · Open weightArtificial Analysis τ²-bench
95.9%
15
Qwen 3.6 Max (preview)Alibaba · ClosedArtificial Analysis τ²-bench
95.9%
16
Kimi K2.5 (Reasoning)Moonshot AI · ClosedArtificial Analysis τ²-bench
95.9%
17
Gemini 3.1 ProGoogle · Closedτ²-bench published setup
95.6%
18
Qwen3.5 397BAlibaba · Open weightArtificial Analysis τ²-bench
95.6%
19
DeepSeek V4 Flash (High)DeepSeek · Open weightArtificial Analysis τ²-bench
95.6%
20
Qwen3.5 397B (Reasoning)Alibaba · Open weightArtificial Analysis τ²-bench
95.6%
21
Gemini 3.5 FlashGoogle · ClosedArtificial Analysis τ²-bench
95.3%
22
Qwen3.6-35B-A3BAlibaba · Open weightArtificial Analysis τ²-bench
95.3%
23
MiniMax M2.5MiniMax · ClosedArtificial Analysis τ²-bench
95.3%
24
DeepSeek V4 Flash (Max)DeepSeek · Open weightArtificial Analysis τ²-bench
95%
25
MiMo-V2-ProXiaomi · ClosedArtificial Analysis τ²-bench
95%
26
Qwen3.7 MaxAlibaba · ClosedArtificial Analysis τ²-bench
94.7%
27
Claude Opus 4.8Anthropic · ClosedArtificial Analysis τ²-bench
94.4%
28
DeepSeek V4 Pro (High)DeepSeek · Open weightArtificial Analysis τ²-bench
94.2%
29
Qwen3.6-27BAlibaba · Open weightArtificial Analysis τ²-bench
94.2%
30
Mistral Medium 3.5 128BMistral · Open weightArtificial Analysis τ²-bench
94.2%
31
MiMo-V2.5-ProXiaomi · ClosedArtificial Analysis τ²-bench
94.2%
32
GPT-5.5OpenAI · Closedτ²-bench Telecom
93.9%
33
Qwen3.5-27BAlibaba · Open weightArtificial Analysis τ²-bench
93.9%
34
Qwen3.5-122B-A10BAlibaba · Open weightArtificial Analysis τ²-bench
93.6%
35
Grok 4.1 Fast (Reasoning)xAI · ClosedArtificial Analysis τ²-bench
93.3%
36
Qwen3.7 PlusAlibaba · ClosedArtificial Analysis τ²-bench
93%
37
Claude Opus 4.6 (Adaptive)Anthropic · ClosedArtificial Analysis τ²-bench
92.1%
38
GPT-5.2-CodexOpenAI · ClosedArtificial Analysis τ²-bench
92.1%
39
Muse SparkMeta · Closedτ²-bench published setup
91.5%
40
MiMo-V2-OmniXiaomi · ClosedArtificial Analysis τ²-bench
91.2%
41
Trinity-Large-PreviewArcee AI · Open weightArtificial Analysis τ²-bench
90.1%
42
Trinity-Large-ThinkingArcee AI · Open weightArtificial Analysis τ²-bench
90.1%
43
Kimi K2.7 CodeMoonshot AI · Open weightArtificial Analysis τ²-bench
90.1%
44
Claude Opus 4.5 ThinkingAnthropic · ClosedArtificial Analysis τ²-bench
89.5%
45
Qwen3.5-35B-A3BAlibaba · Open weightArtificial Analysis τ²-bench
89.2%
46
MiniMax M3MiniMax · Open weightArtificial Analysis τ²-bench
88.9%
47
Claude Opus 4.7 (Adaptive)Anthropic · ClosedArtificial Analysis τ²-bench
88.6%
48
Step 3.5 FlashStepFun · Open weightArtificial Analysis τ²-bench
87.4%
49
GPT-5.4OpenAI · Closedτ²-bench Telecom
87.1%
50
Gemini 3 ProGoogle · ClosedArtificial Analysis τ²-bench
87.1%
51
GPT-5 (medium)OpenAI · ClosedArtificial Analysis τ²-bench
86.5%
52
GPT-5.6 TerraOpenAI · ClosedArtificial Analysis τ²-bench
86.3%
53
Claude Opus 4.5Anthropic · ClosedArtificial Analysis τ²-bench
86.3%
54
GPT-5.3 CodexOpenAI · ClosedArtificial Analysis τ²-bench
86%
55
Ling 2.6 FlashInclusionAI · Open weightArtificial Analysis τ²-bench
86%
56
GPT-5.3-Codex-SparkOpenAI · ClosedArtificial Analysis τ²-bench
86%
57
GPT-5.6 SolOpenAI · ClosedArtificial Analysis τ²-bench
85.1%
58
Claude Opus 4.6Anthropic · ClosedArtificial Analysis τ²-bench
84.8%
59
GPT-5.2OpenAI · ClosedArtificial Analysis τ²-bench
84.8%
60
MiniMax M2.7MiniMax · Open weightArtificial Analysis τ²-bench
84.8%
61
GPT-5 (high)OpenAI · ClosedArtificial Analysis τ²-bench
84.8%
62
MiMo-V2-FlashXiaomi · Open weightArtificial Analysis τ²-bench
83.9%
63
GPT-5.4 miniOpenAI · Closedτ²-bench Telecom
83.3%
64
Nemotron 3 UltraNVIDIA · Open weightArtificial Analysis τ²-bench
83.3%
65
GPT-5.1-Codex-MaxOpenAI · ClosedArtificial Analysis τ²-bench
83%
66
GPT-5.1-CodexOpenAI · ClosedArtificial Analysis τ²-bench
83%
67
GPT-5.1OpenAI · ClosedArtificial Analysis τ²-bench
81.9%
68
o3OpenAI · ClosedArtificial Analysis τ²-bench
80.7%
69
Command A+Cohere · Open weightτ²-bench Telecom
80.7%
70
Claude Sonnet 4.6Anthropic · ClosedArtificial Analysis τ²-bench
79.5%
71
DeepSeek V3.2DeepSeek · Open weightArtificial Analysis τ²-bench
78.9%
72
GLM-4.6Z.AI · Open weightArtificial Analysis τ²-bench
76.9%
73
GPT-5.4 nanoOpenAI · Closedτ²-bench Telecom
76%
74
Grok Code Fast 1xAI · ClosedArtificial Analysis τ²-bench
75.7%
75
Grok 4xAI · ClosedArtificial Analysis τ²-bench
74.9%
76
Qwen3 MaxAlibaba · ClosedArtificial Analysis τ²-bench
74.3%
77
K-ExaoneLG AI Research · ClosedArtificial Analysis τ²-bench
74.3%
78
Claude Opus 4.7Anthropic · ClosedArtificial Analysis τ²-bench
74%
79
Claude 4.1 Opus ThinkingAnthropic · ClosedArtificial Analysis τ²-bench
71.4%
80
Mercury 2Inception · ClosedArtificial Analysis τ²-bench
70.8%
81
GPT-5 miniOpenAI · ClosedArtificial Analysis τ²-bench
68.4%
82
Nemotron 3 Super 120B A12BNVIDIA · Open weightArtificial Analysis τ²-bench
67.8%
83
Grok 4 Fast (Reasoning)xAI · ClosedArtificial Analysis τ²-bench
65.8%
84
GPT-OSS 120BOpenAI · Open weightArtificial Analysis τ²-bench
65.8%
85
Grok 4.1 FastxAI · ClosedArtificial Analysis τ²-bench
63.7%
86
o1OpenAI · ClosedArtificial Analysis τ²-bench
62.6%
87
Kimi K2Moonshot AI · ClosedArtificial Analysis τ²-bench
61.1%
88
GPT-OSS 20BOpenAI · Open weightArtificial Analysis τ²-bench
60.2%
89
Gemma 4 31BGoogle · Open weightArtificial Analysis τ²-bench
59.9%
90
Gemini 2.5 ProGoogle · ClosedArtificial Analysis τ²-bench
54.1%
91
GPT-4.1 miniOpenAI · ClosedArtificial Analysis τ²-bench
52.9%
92
Claude 4 SonnetAnthropic · ClosedArtificial Analysis τ²-bench
52.3%
93
GPT-4.1OpenAI · ClosedArtificial Analysis τ²-bench
47.1%
94
Sarvam 105BSarvam · Open weightArtificial Analysis τ²-bench
46.8%
95
GLM-4.5-AirZ.AI · ClosedArtificial Analysis τ²-bench
46.5%
96
Nemotron 3 Nano Omni 30B A3BNVIDIA · Open weightτ²-bench Telecom
45.3%
97
Gemma 4 26B A4BGoogle · Open weightArtificial Analysis τ²-bench
43.6%
98
Gemini 3 FlashGoogle · ClosedArtificial Analysis τ²-bench
43.3%
99
Mistral Small 4Mistral · Open weightArtificial Analysis τ²-bench
41.2%
100
Mistral Small 4 (Reasoning)Mistral · Open weightArtificial Analysis τ²-bench
41.2%
101
Nemotron 3 Nano 30BNVIDIA · Open weightArtificial Analysis τ²-bench
40.9%
102
DeepSeek V3.1 (Reasoning)DeepSeek · Open weightArtificial Analysis τ²-bench
37.4%
103
DeepSeek-R1DeepSeek · Open weightArtificial Analysis τ²-bench
36.5%
104
GPT-5 nanoOpenAI · ClosedArtificial Analysis τ²-bench
36.5%
105
Gemma 4 12BGoogle · Open weightArtificial Analysis τ²-bench
36.3%
106
DeepSeek V3.1DeepSeek · Open weightArtificial Analysis τ²-bench
34.8%
107
Sarvam 30BSarvam · Open weightArtificial Analysis τ²-bench
34.5%
108
MiniMax M1 80kMiniMax · ClosedArtificial Analysis τ²-bench
34.2%
109
Solar Pro 2Upstage · ClosedArtificial Analysis τ²-bench
31.9%
110
Gemini 3.1 Flash-LiteGoogle · ClosedArtificial Analysis τ²-bench
31.3%
111
Mistral Large 2Mistral · ClosedArtificial Analysis τ²-bench
30.7%
112
o3-miniOpenAI · ClosedArtificial Analysis τ²-bench
28.7%
113
Ministral 3 14B (Reasoning)Mistral · Open weightArtificial Analysis τ²-bench
27.2%
114
Ministral 3 14BMistral · Open weightArtificial Analysis τ²-bench
27.2%
115
Ministral 3 8B (Reasoning)Mistral · Open weightArtificial Analysis τ²-bench
26.6%
116
Ministral 3 8BMistral · Open weightArtificial Analysis τ²-bench
26.6%
117
GPT-4oOpenAI · ClosedArtificial Analysis τ²-bench
25.1%
118
Ministral 3 3B (Reasoning)Mistral · Open weightArtificial Analysis τ²-bench
24.9%
119
Ministral 3 3BMistral · Open weightArtificial Analysis τ²-bench
24.9%
120
Mistral Large 3Mistral · ClosedArtificial Analysis τ²-bench
24.6%
121
Mistral Medium 3Mistral · ClosedArtificial Analysis τ²-bench
24.3%
122
DeepSeek V3DeepSeek · Open weightArtificial Analysis τ²-bench
22.8%
123
Granite-4.0-1BIBM · Open weightArtificial Analysis τ²-bench
22.8%
124
Claude 3 HaikuAnthropic · ClosedArtificial Analysis τ²-bench
21.1%
125
Gemma 4 E2BGoogle · Open weightArtificial Analysis τ²-bench
20.8%
126
Gemma 4 E4BGoogle · Open weightArtificial Analysis τ²-bench
20.8%
127
Exaone 4.0 1.2BLG AI Research · Open weightArtificial Analysis τ²-bench
20.5%
128
Granite-4.0-H-1BIBM · Open weightArtificial Analysis τ²-bench
19.6%
129
LFM2.5-1.2B-ThinkingLiquidAI · ClosedArtificial Analysis τ²-bench
19.6%
130
Llama 3.1 405BMeta · Open weightArtificial Analysis τ²-bench
19%
131
Llama 4 MaverickMeta · Open weightArtificial Analysis τ²-bench
17.8%
132
GPT-4.1 nanoOpenAI · ClosedArtificial Analysis τ²-bench
17.3%
133
LFM2.5-8B-A1BLiquidAI · Open weightτ²-bench Telecom
16.1%
134
Llama 4 ScoutMeta · Open weightArtificial Analysis τ²-bench
15.5%
135
Gemini 2.5 FlashGoogle · ClosedArtificial Analysis τ²-bench
14.9%
136
Granite-4.0-H-350MIBM · Open weightArtificial Analysis τ²-bench
14.6%
137
Nova ProAmazon · ClosedArtificial Analysis τ²-bench
14%
138
Granite-4.0-350MIBM · Open weightArtificial Analysis τ²-bench
13.2%
139
Nemotron Ultra 253BNVIDIA · Open weightArtificial Analysis τ²-bench
11.4%
140
LFM2-24B-A2BLiquidAI · ClosedArtificial Analysis τ²-bench
11.1%
141
LFM2.5-1.2B-InstructLiquidAI · ClosedArtificial Analysis τ²-bench
10.8%
142
Gemma 3 27BGoogle · Open weightArtificial Analysis τ²-bench
10.5%
143
LFM2.5-VL-1.6B-ExtractLiquidAI · Open weightArtificial Analysis τ²-bench
8.5%
144
Exaone 4.0 32BLG AI Research · Open weightArtificial Analysis τ²-bench
4.1%
145
Phi-4Microsoft · Open weightArtificial Analysis τ²-bench
0%

The published τ²-bench results snapshot places GLM-5.2 first at 99.1%. The third row is 0.6 points behind. The broader top-10 range is 1.4 points, so many of the published results sit in a relatively narrow band.

145 models have been evaluated on τ²-bench results. The benchmark falls in the Agentic category. This category carries a 22% weight in BenchLM.ai's overall scoring system. τ²-bench results is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About τ²-bench results

Year

2025

Tasks

Airline, retail, and telecom customer-service task sets

Format

Published domain success or pass^k results

Difficulty

Dual-control customer-service workflows

τ²-bench extends the original benchmark with a dual-control telecom domain where the agent and simulated user can both act through tools. The maintained framework also includes airline and retail. BenchLM keeps each exact source attached and labels the published setup instead of treating every row as one controlled run.

BenchLM freshness & provenance

Version

τ²-Bench 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

Are all τ²-bench scores directly comparable?

No. Match the domain, task release, agent model, user model, scaffold, prompts, number of trials, and pass^k definition. BenchLM labels each sourced row so a telecom result or third-party implementation is not silently treated as the same setup as an airline, retail, or aggregate result.

Does a high τ²-bench score prove production support reliability?

No. It measures success in simulated customer-service environments under a reported setup. Production identity checks, permission boundaries, changing policies, latency, cost, monitoring, and human escalation still need separate testing.

Last updated: July 23, 2026 · BenchLM version τ²-Bench 2026

Choose a model with this week’s evidence

Join 2,000+ readers for ranking moves, pricing changes, and the claims that still need proof.

One email each week. Unsubscribe anytime.