Massive Multitask Language Understanding Professional (MMLU-Pro)
Data verified
An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.
Top models on MMLU-Pro — November 12, 2026
As of November 12, 2026, Qwen3.7 Max leads the MMLU-Pro leaderboard with 89.6% , followed by Claude Opus 4.5 (89.5%) and Qwen3.7 Plus (88.5%).
1Closed
Qwen3.7 Max
Alibaba
Reported by Alibaba89.6%
Overall 60.81Context 1M
2Closed
Claude Opus 4.5
Anthropic
Another lab’s comparison table89.5%
Overall 53.11Context 200K
3Closed
Qwen3.7 Plus
Alibaba
Reported by Alibaba88.5%
Overall 53.68Context 1M
81 modelsKnowledge6% of Knowledge reference weightRefreshingUpdated November 12, 2026
Leaderboard (81 models)
None of the 81 results on this page was run by the benchmark or an independent evaluator: 72 reported by the labs that built the models, 4 reported by a third party and 5 carried from an upstream page, where who ran them is not recorded.
81 of 81 rows
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Qwen3.7 MaxAlibabaReported by Alibaba | 89.6% | Reported by Alibaba | Not reported | Closed |
| 2 | Claude Opus 4.5AnthropicAnother lab’s comparison table | 89.5% | Another lab’s comparison table | Not reported | Closed |
| 3 | Qwen3.7 PlusAlibabaReported by Alibaba | 88.5% | Reported by Alibaba | Not reported | Closed |
| 4 | Qwen3.6 PlusAlibabaReported by Alibaba | 88.5% | Reported by Alibaba | Not reported | Closed |
| 5 | Qwen3.5 397BAlibabaReported by Alibaba | 87.8% | Reported by Alibaba | Not reported | Open |
| 6 | DeepSeek V4 Pro 0813DeepSeekReported by DeepSeek | 87.5% | Reported by DeepSeek | Not reported | Open |
| 7 | Kimi K2.5 (Reasoning)Moonshot AIReported by Moonshot AI | 87.1% | Reported by Moonshot AI | Not reported | Open |
| 8 | Kimi K2.5Moonshot AIReported by Moonshot AI | 87.1% | Reported by Moonshot AI | Not reported | Open |
| 9 | DeepSeek V4 Pro (High)DeepSeekReported by DeepSeek | 87.1% | Reported by DeepSeek | Not reported | Open |
| 10 | Nemotron 3 UltraNVIDIAReported by NVIDIA | 86.8% | Reported by NVIDIA | Not reported | Open |
| 11 | Qwen3.5-122B-A10BAlibabaReported by Alibaba | 86.7% | Reported by Alibaba | Not reported | Open |
| 12 | DeepSeek V4 Flash (High)DeepSeekReported by DeepSeek | 86.4% | Reported by DeepSeek | Not reported | Open |
| 13 | Solar Pro 4UpstageReported by Upstage | 86.3% | Reported by Upstage | Not reported | Closed |
| 14 | Qwen3.6-27BAlibabaReported by Alibaba | 86.2% | Reported by Alibaba | Not reported | Open |
| 15 | DeepSeek V4 Flash 0731DeepSeekReported by DeepSeek | 86.2% | Reported by DeepSeek | Not reported | Open |
| 16 | Solar Open 2UpstageReported by Upstage | 86.2% | Reported by Upstage | Not reported | Open |
| 17 | Qwen3.5-27BAlibabaReported by Alibaba | 86.1% | Reported by Alibaba | Not reported | Open |
| 18 | GLM-5Z.AIAnother lab’s comparison table | 85.7% | Another lab’s comparison table | Not reported | Open |
| 19 | Qwen3.5-35B-A3BAlibabaReported by Alibaba | 85.3% | Reported by Alibaba | Not reported | Open |
| 20 | Gemma 4 31BGoogleReported by Google | 85.2% | Reported by Google | Not reported | Open |
| 21 | Qwen3.6-35B-A3BAlibabaReported by Alibaba | 85.2% | Reported by Alibaba | Not reported | Open |
| 22 | DeepSeek-R1-0528DeepSeekReported by DeepSeek | 85% | Reported by DeepSeek | Not reported | Open |
| 23 | MAI-Thinking-1MicrosoftReported by Microsoft | 85% | Reported by Microsoft | Not reported | Closed |
| 24 | MiMo-V2-FlashXiaomiCarried from an upstream page | 84.9% | Carried from an upstream page | Not reported | Open |
| 25 | DeepSeek V3.1 (Reasoning)DeepSeekReported by DeepSeek | 84.8% | Reported by DeepSeek | Not reported | Open |
| 26 | Qwen3 235B 2507 (Reasoning)AlibabaReported by Alibaba | 84.4% | Reported by Alibaba | Not reported | Open |
| 27 | GLM-4.7Z.AICarried from an upstream page | 84.3% | Carried from an upstream page | Not reported | Open |
| 28 | Ripple-65B PreviewRicursive IntelligenceReported by Ricursive Intelligence | 84.2% | Reported by Ricursive Intelligence | Not reported | Open |
| 29 | DeepSeek-R1DeepSeekReported by DeepSeek | 84% | Reported by DeepSeek | Not reported | Open |
| 30 | Nemotron 3 Super 100BNVIDIAReported by NVIDIA | 83.7% | Reported by NVIDIA | Not reported | Open |
| 31 | DeepSeek V3.1DeepSeekReported by DeepSeek | 83.7% | Reported by DeepSeek | Not reported | Open |
| 32 | K-EXAONE 2.0LG AI ResearchReported by LG AI Research | 83.5% | Reported by LG AI Research | Not reported | Open |
| 33 | Qwen3 235B 2507AlibabaReported by Alibaba | 83% | Reported by Alibaba | Not reported | Open |
| 34 | DeepSeek V4 FlashDeepSeekReported by DeepSeek | 83% | Reported by DeepSeek | Not reported | Open |
| 35 | DeepSeek V4 ProDeepSeekReported by DeepSeek | 82.9% | Reported by DeepSeek | Not reported | Open |
| 36 | Gemma 4 26B A4BGoogleReported by Google | 82.6% | Reported by Google | Not reported | Open |
| 37 | Claude Opus 4.6AnthropicAnother lab’s comparison table | 82% | Another lab’s comparison table | Not reported | Closed |
| 38 | Exaone 4.0 32BLG AI ResearchReported by LG AI Research | 81.8% | Reported by LG AI Research | Not reported | Open |
| 39 | Sarvam 105BSarvamReported by Sarvam | 81.7% | Reported by Sarvam | Not reported | Open |
| 40 | Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIAReported by NVIDIA | 81.6% | Reported by NVIDIA | Not reported | Open |
| 41 | Kimi K2Moonshot AIReported by Moonshot AI | 81.1% | Reported by Moonshot AI | Not reported | Closed |
| 42 | Sarvam 30BSarvamReported by Sarvam | 80% | Reported by Sarvam | Not reported | Open |
| 43 | Grok 3 [Beta]xAIReported by xAI | 79.9% | Reported by xAI | Not reported | Closed |
| 44 | LongCat-Flash-Lite-SparseMeituanReported by Meituan | 79.2% | Reported by Meituan | Not reported | Open |
| 45 | Claude Sonnet 4.6AnthropicAnother lab’s comparison table | 79.2% | Another lab’s comparison table | Not reported | Closed |
| 46 | Nemotron 3 Nano 30BNVIDIAReported by NVIDIA | 78.3% | Reported by NVIDIA | Not reported | Open |
| 47 | Granite 4.2 30BIBMReported by IBM | 77.6% | Reported by IBM | Not reported | Open |
| 48 | Nemotron 3 Nano Omni 30B A3BNVIDIAReported by NVIDIA | 77.3% | Reported by NVIDIA | Not reported | Open |
| 49 | Gemma 4 12BGoogleReported by Google | 77.2% | Reported by Google | Not reported | Open |
| 50 | Qwen2.5-MaxAlibabaReported by Alibaba | 76.1% | Reported by Alibaba | Not reported | Closed |
| 51 | Phi-4-reasoning-plusMicrosoftReported by Microsoft | 76% | Reported by Microsoft | Not reported | Open |
| 52 | Celeris-1CelerisReported by Celeris | 75.9% | Reported by Celeris | Not reported | Closed |
| 53 | DeepSeek V3DeepSeekCarried from an upstream page | 75.9% | Carried from an upstream page | Not reported | Open |
| 54 | Grok 2xAIReported by xAI | 75.5% | Reported by xAI | Not reported | Closed |
| 55 | Phi-4-reasoningMicrosoftReported by Microsoft | 74.3% | Reported by Microsoft | Not reported | Open |
| 56 | ZAYA1-8BZyphraReported by Zyphra | 74.2% | Reported by Zyphra | Not reported | Open |
| 57 | Granite 4.2 8BIBMReported by IBM | 74.0% | Reported by IBM | Not reported | Open |
| 58 | DeepSeek V4 Pro BaseDeepSeekReported by DeepSeek | 73.5% | Reported by DeepSeek | Not reported | Open |
| 59 | Grok 2 minixAIReported by xAI | 72% | Reported by xAI | Not reported | Closed |
| 60 | MiniCPM5-2BOpenBMBReported by OpenBMB | 70.8% | Reported by OpenBMB | Not reported | Open |
| 61 | Gemma 4 E4BGoogleCarried from an upstream page | 69.4% | Carried from an upstream page | Not reported | Open |
| 62 | Llama 3.3 70BMetaReported by Meta | 68.9% | Reported by Meta | Not reported | Open |
| 63 | Qwen2.5-VL-32BAlibabaReported by Alibaba | 68.8% | Reported by Alibaba | Not reported | Open |
| 64 | DeepSeek V4 Flash BaseDeepSeekReported by DeepSeek | 68.3% | Reported by DeepSeek | Not reported | Open |
| 65 | ZAYA1-74B-PreviewZyphraReported by Zyphra | 68.1% | Reported by Zyphra | Not reported | Open |
| 66 | Granite 4.2 3BIBMReported by IBM | 67.8% | Reported by IBM | Not reported | Open |
| 67 | Mistral Small 3.1MistralReported by Mistral | 66.8% | Reported by Mistral | Not reported | Open |
| 68 | Llama 3.1 70BMetaReported by Meta | 66.4% | Reported by Meta | Not reported | Open |
| 69 | Claude 3.5 HaikuAnthropicReported by Anthropic | 65% | Reported by Anthropic | Not reported | Closed |
| 70 | Gemma 4 E2BGoogleCarried from an upstream page | 60% | Carried from an upstream page | Not reported | Open |
| 71 | Soofi S 30B-A3BSoofi ProjectReported by Soofi Project | 51.4% | Reported by Soofi Project | Not reported | Open |
| 72 | MiniCPM5-1BOpenBMBReported by OpenBMB | 48.9% | Reported by OpenBMB | Not reported | Open |
| 73 | Llama 3.1 8BMetaReported by Meta | 48.3% | Reported by Meta | Not reported | Open |
| 74 | Phi-3.5-miniMicrosoftReported by Microsoft | 47.4% | Reported by Microsoft | Not reported | Open |
| 75 | Granite-4.0-1BIBMReported by IBM | 34.0% | Reported by IBM | Not reported | Open |
| 76 | Granite-4.0-H-1BIBMReported by IBM | 32.9% | Reported by IBM | Not reported | Open |
| 77 | LFM2.5-230MLiquidAIReported by LiquidAI | 20.3% | Reported by LiquidAI | Not reported | Open |
| 78 | LFM2.5-350MLiquidAIReported by LiquidAI | 20.0% | Reported by LiquidAI | Not reported | Open |
| 79 | LFM2.5-VL-450MLiquidAIReported by LiquidAI | 19.3% | Reported by LiquidAI | Not reported | Open |
| 80 | Granite-4.0-H-350MIBMReported by IBM | 14.4% | Reported by IBM | Not reported | Open |
| 81 | Granite-4.0-350MIBMReported by IBM | 12.1% | Reported by IBM | Not reported | Open |
- Lab-reported
- Benchmark-run
- Independent
MMLU-Pro: reported scores versus Kaggle’s run
The highest reported MMLU-Pro score in the table above is 89.6 (Qwen3.7 Max). The best row in Kaggle’s own run is 93.2 (Gemini 4 Argon). 4 models have both a reported score and a Kaggle run. Kaggle’s number is lower for 3 and higher for 1, and the median gap is −2.3 points. The widest gap is Qwen3 235B 2507: 83.0 reported, 66.1 on Kaggle.
Kaggle runs every model on the board with the benchmark authors’ code and one set of settings. A reported score uses the prompt, sampling, and reasoning settings its publisher chose, which are often not disclosed. How Kaggle Benchmarks reruns tests.
| Source | Rows | Top score | Top row | Median |
|---|---|---|---|---|
| Reported (table above) | 81 | 89.6 | Qwen3.7 Max | 81.1 |
| Kaggle’s run | 64 | 93.2 | Gemini 4 Argon | 83.0 |
Same model, both numbers
| Model | Reported | Kaggle’s run |
|---|---|---|
| Gemma 4 31BThe lab’s number is 1.5 points lower. | 85.2 | 86.7+1.5 |
| DeepSeek-R1-0528The lab’s number is 1.0 point higher. | 85.0 | 84.0−1.0 |
| Qwen3 235B 2507The lab’s number is 16.9 points higher. | 83.0 | 66.1−16.9 |
| Claude 3.5 HaikuThe lab’s number is 3.7 points higher. | 65.0 | 61.3−3.7 |
The small figure is Kaggle’s score minus the reported one, in points. A model is matched to its Kaggle row by Kaggle’s model version, and where Kaggle ran it at several settings the best row is shown. Reported is the score in the table above. Every Kaggle row is in the Kaggle table below. Source: kaggle.com/benchmarks/open-benchmarks/mmlu-pro.
AA MMLU-Pro
Stored separately from the core MMLU-Pro lane because Artificial Analysis controls the evaluation configuration.
15 of 15 rows
| Source rank | Source model or configuration | Score | Who ran it | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|---|
| 1 | Gemini 3 ProGoogle | 89.8% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 2 | Claude Opus 4.5 ThinkingAnthropic | 89.5% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 3 | Claude 4.1 Opus ThinkingAnthropic | 88.0% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 4 | GPT-5.2OpenAI | 87.4% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 5 | Claude 4 Opus ThinkingAnthropic | 87.3% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 6 | GPT-5 (high)OpenAI | 87.1% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 7 | GPT-5.1OpenAI | 87.0% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 8 | Grok 4xAI | 86.6% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 9 | Gemini 2.5 ProGoogle | 86.2% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 10 | GLM-4.7Z.AI | 85.6% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Open |
| 11 | Grok 4.1 Fast (Reasoning)xAI | 85.4% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 12 | o3OpenAI | 85.3% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 13 | DeepSeek V3.1 (Reasoning)DeepSeek | 85.1% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Open |
| 14 | Grok 4 Fast (Reasoning)xAI | 85.0% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Closed |
| 15 | DeepSeek-R1DeepSeek | 84.9% | Carried from an upstream page | Artificial AnalysisOriginal result | Not reported | Open |
MMLU-Pro (Arcee)
BenchLM stores this chart-specific MMLU-Pro row separately so it does not overwrite the standardized weighted MMLU-Pro benchmark values.
6 of 6 rows
| Source rank | Source model or configuration | Score | Who ran it | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6Anthropic | 89.1% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Closed |
| 2 | Kimi K2.5Moonshot AI | 87.1% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open |
| 3 | GLM-5Z.AI | 85.8% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open |
| 4 | Trinity-Large-ThinkingArcee AI | 83.4% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open |
| 5 | MiniMax M2.7MiniMax | 80.8% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open |
| 6 | Trinity-Large-PreviewArcee AI | 75.2% | Reported by Arcee AI | Arcee AIOriginal result | Not reported | Open |
MMLU-Pro (Vals)
BenchLM mirrors this Vals AI leaderboard for reference only. It is excluded from BenchLM overall and category rankings.
138 of 138 rows
| Source rank | Source model or configuration | Vals MMLU-Pro mirror score | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Fable 5.1Anthropic | 92.4% | Vals AIOriginal result | Not reported | Closed |
| 2 | Claude Opus 5Anthropic | 91.6% | Vals AIOriginal result | Not reported | Closed |
| 3 | Claude Fable 5Anthropic | 91.5% | Vals AIOriginal result | Not reported | Closed |
| 4 | Gemini 3.1 Pro PreviewGoogle | 91.0% | high reasoningOriginal result | Not reported | Unknown |
| 5 | Gemini 3.8 FlashGoogle | 90.2% | high reasoningOriginal result | Not reported | Closed |
| 6 | Gemini 3.7 FlashGoogle | 90.1% | high reasoningOriginal result | Not reported | Closed |
| 7 | Gemini 3 Pro PreviewGoogle | 90.1% | Vals AIOriginal result | Not reported | Unknown |
| 8 | Claude Opus 4.7Anthropic | 89.9% | Vals AIOriginal result | Not reported | Closed |
| 9 | Claude Opus 4.8Anthropic | 89.6% | Vals AIOriginal result | Not reported | Closed |
| 10 | Gemini 3.5 FlashGoogle | 89.5% | high reasoningOriginal result | Not reported | Closed |
| 11 | Grok 4.6xAI | 89.4% | high reasoningOriginal result | Not reported | Closed |
| 12 | Qwen3.7 MaxAlibaba | 89.3% | Vals AIOriginal result | Not reported | Closed |
| 13 | Gemini 3.6 FlashGoogle | 89.3% | high reasoningOriginal result | Not reported | Closed |
| 14 | Grok 4.5xAI | 89.2% | high reasoningOriginal result | Not reported | Closed |
| 15 | Claude Opus 4.6 (Adaptive)Anthropic | 89.1% | Vals AIOriginal result | Not reported | Closed |
| 16 | GPT-5.6 SolOpenAI | 89.1% | max reasoningOriginal result | Not reported | Closed |
| 17 | Muse Spark 1.1Meta | 88.7% | xhigh reasoningOriginal result | Not reported | Closed |
| 18 | Qwen3.8 MaxAlibaba | 88.6% | Vals AIOriginal result | Not reported | Open |
| 19 | Gemini 3 Flash PreviewGoogle | 88.6% | high reasoningOriginal result | Not reported | Unknown |
| 20 | Muse Spark 1.2Meta | 88.3% | xhigh reasoningOriginal result | Not reported | Closed |
| 21 | GPT-5.5OpenAI | 88.1% | xhigh reasoningOriginal result | Not reported | Closed |
| 22 | Kimi K3Moonshot AI | 88.0% | max reasoningOriginal result | Not reported | Open |
| 23 | Claude Opus 4.1 20250805 ThinkingAnthropic | 87.9% | Vals AIOriginal result | Not reported | Unknown |
| 24 | Qwen3.6 PlusAlibaba | 87.7% | Vals AIOriginal result | Not reported | Closed |
| 25 | Kimi K2.6Moonshot AI | 87.6% | Vals AIOriginal result | Not reported | Open |
| 26 | Claude Sonnet 5Anthropic | 87.5% | Vals AIOriginal result | Not reported | Closed |
| 27 | GPT-5.4OpenAI | 87.5% | xhigh reasoningOriginal result | Not reported | Closed |
| 28 | Claude Sonnet 4.5 ThinkingAnthropic | 87.4% | Vals AIOriginal result | Not reported | Closed |
| 29 | Claude Sonnet 4.6Anthropic | 87.3% | Vals AIOriginal result | Not reported | Closed |
| 30 | Muse SparkMeta | 87.3% | Vals AIOriginal result | Not reported | Closed |
| 31 | Claude Opus 4.5 ThinkingAnthropic | 87.3% | Vals AIOriginal result | Not reported | Closed |
| 32 | DeepSeek V4 Pro 0813DeepSeek | 87.2% | max reasoningOriginal result | Not reported | Open |
| 33 | Claude Opus 4.1Anthropic | 87.2% | Vals AIOriginal result | Not reported | Unknown |
| 34 | Qwen3.5 Plus ThinkingAlibaba | 87.2% | Vals AIOriginal result | Not reported | Unknown |
| 35 | MiniMax M2.1MiniMax | 87.0% | Vals AIOriginal result | Not reported | Unknown |
| 36 | DeepSeek V4 Pro 0813DeepSeek | 87.0% | max reasoningOriginal result | Not reported | Open |
| 37 | GLM-5.1Z.AI | 86.9% | Vals AIOriginal result | Not reported | Open |
| 38 | GLM-5.3Z.AI | 86.8% | max reasoningOriginal result | Not reported | Open |
| 39 | GLM-5.2Z.AI | 86.7% | Vals AIOriginal result | Not reported | Open |
| 40 | GPT-5.6 TerraOpenAI | 86.7% | xhigh reasoningOriginal result | Not reported | Closed |
| 41 | GPT-5OpenAI | 86.5% | high reasoningOriginal result | Not reported | Unknown |
| 42 | GPT-5.1OpenAI | 86.4% | high reasoningOriginal result | Not reported | Closed |
| 43 | InklingThinking Machines Lab | 86.3% | 0.99 reasoningOriginal result | Not reported | Open |
| 44 | Grok 4.20 0309 ReasoningxAI | 86.3% | Vals AIOriginal result | Not reported | Unknown |
| 45 | Gemini 3.1 Flash Lite PreviewGoogle | 86.2% | Vals AIOriginal result | Not reported | Unknown |
| 46 | GPT-5.2OpenAI | 86.2% | xhigh reasoningOriginal result | Not reported | Closed |
| 47 | DeepSeek V4 Flash 0731DeepSeek | 86.2% | high reasoningOriginal result | Not reported | Open |
| 48 | Claude Opus 4Anthropic | 86.2% | Vals AIOriginal result | Not reported | Unknown |
| 49 | GLM-5.3-FlashZ.AI | 86.1% | max reasoningOriginal result | Not reported | Open |
| 50 | GPT-5.6 LunaOpenAI | 86.0% | max reasoningOriginal result | Not reported | Closed |
| 51 | GLM 5 ThinkingZhipu AI | 86.0% | Vals AIOriginal result | Not reported | Unknown |
| 52 | Kimi K2.5 ThinkingMoonshot AI | 85.9% | Vals AIOriginal result | Not reported | Unknown |
| 53 | Grok 4.3xAI | 85.8% | high reasoningOriginal result | Not reported | Closed |
| 54 | Gemini 3.5 Flash-LiteGoogle | 85.8% | high reasoningOriginal result | Not reported | Closed |
| 55 | Nemotron 3 Ultra 550b A55bNvidia | 85.8% | Vals AIOriginal result | Not reported | Unknown |
| 56 | o3OpenAI | 85.6% | high reasoningOriginal result | Not reported | Closed |
| 57 | Claude Opus 4.5Anthropic | 85.6% | Vals AIOriginal result | Not reported | Closed |
| 58 | Inkling-SmallThinking Machines Lab | 85.6% | 0.99 reasoningOriginal result | Not reported | Open |
| 59 | Grok 4 0709xAI | 85.3% | Vals AIOriginal result | Not reported | Unknown |
| 60 | Qwen3 MaxAlibaba | 85.0% | Vals AIOriginal result | Not reported | Closed |
| 61 | DeepSeek V3p2 ThinkingFireworks AI | 84.9% | Vals AIOriginal result | Not reported | Unknown |
| 62 | MiMo-V2.5-ProXiaomi | 84.6% | Vals AIOriginal result | Not reported | Closed |
| 63 | GPT-5.4 miniOpenAI | 84.6% | xhigh reasoningOriginal result | Not reported | Closed |
| 64 | Qwen3 MaxAlibaba | 84.4% | Vals AIOriginal result | Not reported | Closed |
| 65 | Qwen3.8-27BAlibaba | 84.3% | xhigh reasoningOriginal result | Not reported | Open |
| 66 | MiniMax M3MiniMax | 84.2% | Vals AIOriginal result | Not reported | Open |
| 67 | Grok 4.1 Fast (Reasoning)xAI | 84.2% | Vals AIOriginal result | Not reported | Closed |
| 68 | Qwen3.5 FlashAlibaba | 84.1% | Vals AIOriginal result | Not reported | Closed |
| 69 | Gemini 2.5 Pro Exp 03 25Google | 84.1% | Vals AIOriginal result | Not reported | Unknown |
| 70 | Claude Sonnet 4 20250514 ThinkingAnthropic | 83.9% | Vals AIOriginal result | Not reported | Unknown |
| 71 | Gemini 2.5 Flash Preview 09 2025Google | 83.7% | Vals AIOriginal result | Not reported | Unknown |
| 72 | Gemini 2.5 Flash Preview 09 2025 ThinkingGoogle | 83.7% | Vals AIOriginal result | Not reported | Unknown |
| 73 | Qwen3 Max PreviewAlibaba | 83.5% | Vals AIOriginal result | Not reported | Unknown |
| 74 | o1OpenAI | 83.5% | high reasoningOriginal result | Not reported | Closed |
| 75 | DeepSeek-R1DeepSeek | 83.2% | Vals AIOriginal result | Not reported | Open |
| 76 | DeepSeek V3p2Fireworks AI | 83.1% | Vals AIOriginal result | Not reported | Unknown |
| 77 | MiMo-V2.5Xiaomi | 82.9% | Vals AIOriginal result | Not reported | Closed |
| 78 | GLM-4.7Z.AI | 82.7% | Vals AIOriginal result | Not reported | Open |
| 79 | Claude 3.7 Sonnet ThinkingAnthropic | 82.7% | Vals AIOriginal result | Not reported | Closed |
| 80 | GPT-5 miniOpenAI | 82.2% | high reasoningOriginal result | Not reported | Closed |
| 81 | GLM-4.6Z.AI | 82.2% | Vals AIOriginal result | Not reported | Open |
| 82 | Ling 3.0 Flash 2607Ant | 82.0% | Vals AIOriginal result | Not reported | Unknown |
| 83 | Grok 3 Mini Fast High ReasoningxAI | 81.4% | high reasoningOriginal result | Not reported | Unknown |
| 84 | Qwen3 235B-A22BAlibaba | 81.2% | Vals AIOriginal result | Not reported | Open |
| 85 | GLM-4.5Z.AI | 81.2% | Vals AIOriginal result | Not reported | Closed |
| 86 | Kimi K2 ThinkingMoonshot AI | 81.1% | Vals AIOriginal result | Not reported | Unknown |
| 87 | Claude 3.7 SonnetAnthropic | 80.7% | Vals AIOriginal result | Not reported | Closed |
| 88 | o4-miniOpenAI | 80.6% | high reasoningOriginal result | Not reported | Closed |
| 89 | GPT-4.1OpenAI | 80.5% | high reasoningOriginal result | Not reported | Closed |
| 90 | MiniMax M2.7MiniMax | 80.4% | Vals AIOriginal result | Not reported | Open |
| 91 | MiniMax M2.5MiniMax | 80.1% | Vals AIOriginal result | Not reported | Closed |
| 92 | Grok 3 Mini Fast Low ReasoningxAI | 80.0% | low reasoningOriginal result | Not reported | Unknown |
| 93 | Grok 3xAI | 79.9% | Vals AIOriginal result | Not reported | Unknown |
| 94 | Mistral Large 2512Mistral AI | 79.8% | Vals AIOriginal result | Not reported | Unknown |
| 95 | Grok 4 Fast (Reasoning)xAI | 79.7% | Vals AIOriginal result | Not reported | Closed |
| 96 | DeepSeek V3 0324DeepSeek | 79.5% | Vals AIOriginal result | Not reported | Open |
| 97 | Claude Sonnet 4Anthropic | 79.4% | Vals AIOriginal result | Not reported | Unknown |
| 98 | Llama4 Maverick Instruct BasicFireworks AI | 79.4% | Vals AIOriginal result | Not reported | Unknown |
| 99 | Moonshotai Kimi K2 InstructTogether AI | 79.4% | Vals AIOriginal result | Not reported | Unknown |
| 100 | GPT-OSS 120BOpenAI | 79.2% | Vals AIOriginal result | Not reported | Open |
| 101 | Gemini 2.5 Flash Lite Preview 09 2025 ThinkingGoogle | 79.1% | Vals AIOriginal result | Not reported | Unknown |
| 102 | Claude Haiku 4.5 ThinkingAnthropic | 78.7% | Vals AIOriginal result | Not reported | Closed |
| 103 | o3-miniOpenAI | 78.7% | high reasoningOriginal result | Not reported | Closed |
| 104 | Gemini 2.5 Flash Lite Preview 09 2025Google | 78.6% | Vals AIOriginal result | Not reported | Unknown |
| 105 | Claude 3.5 SonnetAnthropic | 78.4% | Vals AIOriginal result | Not reported | Closed |
| 106 | Gemini 2.0 Flash 001Google | 77.4% | Vals AIOriginal result | Not reported | Unknown |
| 107 | GPT-4.1 miniOpenAI | 77.2% | high reasoningOriginal result | Not reported | Closed |
| 108 | GPT-5.4 nanoOpenAI | 77.2% | high reasoningOriginal result | Not reported | Closed |
| 109 | GPT-5 nanoOpenAI | 76.1% | high reasoningOriginal result | Not reported | Closed |
| 110 | Grok 2 1212xAI | 75.5% | Vals AIOriginal result | Not reported | Unknown |
| 111 | Mistral Medium 3.5Mistral AI | 75.3% | high reasoningOriginal result | Not reported | Unknown |
| 112 | Gemini 1.5 Pro 002Google | 75.3% | Vals AIOriginal result | Not reported | Unknown |
| 113 | Mistral Medium 2505Mistral AI | 75.3% | Vals AIOriginal result | Not reported | Unknown |
| 114 | Grok 4.1 Fast Non ReasoningxAI | 75.2% | Vals AIOriginal result | Not reported | Unknown |
| 115 | GPT-4oOpenAI | 74.1% | high reasoningOriginal result | Not reported | Closed |
| 116 | DeepSeek V3DeepSeek | 73.8% | Vals AIOriginal result | Not reported | Open |
| 117 | GPT-4oOpenAI | 72.6% | high reasoningOriginal result | Not reported | Closed |
| 118 | GPT-OSS 20BOpenAI | 71.6% | Vals AIOriginal result | Not reported | Open |
| 119 | Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561Together AI | 70.8% | Vals AIOriginal result | Not reported | Unknown |
| 120 | Grok 4 Fast Non ReasoningxAI | 70.3% | Vals AIOriginal result | Not reported | Unknown |
| 121 | Meta Llama Llama 3.3 70B Instruct TurboTogether AI | 69.9% | Vals AIOriginal result | Not reported | Unknown |
| 122 | Mistral Large 2411Mistral AI | 69.7% | Vals AIOriginal result | Not reported | Unknown |
| 123 | Meta Llama Llama 4 Scout 17B 16E InstructTogether AI | 69.6% | Vals AIOriginal result | Not reported | Unknown |
| 124 | Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561 ThinkingTogether AI | 69.6% | Vals AIOriginal result | Not reported | Unknown |
| 125 | Command A 03 2025Cohere | 69.2% | Vals AIOriginal result | Not reported | Unknown |
| 126 | Laguna XS.2Poolside | 69.0% | Vals AIOriginal result | Not reported | Open |
| 127 | Laguna M.1Poolside | 68.8% | Vals AIOriginal result | Not reported | Closed |
| 128 | Magistral Medium 2509Mistral AI | 68.7% | Vals AIOriginal result | Not reported | Unknown |
| 129 | Mistral Small 2503Mistral AI | 66.0% | Vals AIOriginal result | Not reported | Unknown |
| 130 | Gemini 1.5 Flash 002Google | 65.6% | Vals AIOriginal result | Not reported | Unknown |
| 131 | Mistral Small 2402Mistral AI | 64.4% | Vals AIOriginal result | Not reported | Unknown |
| 132 | Claude 3.5 HaikuAnthropic | 64.1% | Vals AIOriginal result | Not reported | Closed |
| 133 | GPT-4.1 nanoOpenAI | 63.5% | high reasoningOriginal result | Not reported | Closed |
| 134 | GPT-4o miniOpenAI | 62.7% | high reasoningOriginal result | Not reported | Closed |
| 135 | Magistral Small 2509Mistral AI | 62.1% | Vals AIOriginal result | Not reported | Unknown |
| 136 | Jamba Large 1.6AI21 Labs | 49.8% | Vals AIOriginal result | Not reported | Unknown |
| 137 | Command R PlusCohere | 44.0% | Vals AIOriginal result | Not reported | Unknown |
| 138 | Jamba Mini 1.6AI21 Labs | 30.3% | Vals AIOriginal result | Not reported | Unknown |
MMLU-Pro (Kaggle)
Native proportions are displayed as percentages. Source profiles, task versions, evaluation dates, and symmetric intervals stay attached to each row. Missing results are not zero. How Kaggle Benchmarks reruns tests.
64 of 64 rows
| Source rank | Source model or configuration | Accuracy (%) | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | 93.2% | Task version 12; Evaluated 2026-10-09Original result | Not reported | Unknown |
| 2 | Claude Opus 5 | 91.7% | Task version 12; Evaluated 2026-07-27Original result | Not reported | Unknown |
| 3 | Gemini 3.1 Pro Preview | 91.3% | Task version 12; Evaluation date not reportedOriginal result | Not reported | Unknown |
| 4 | Gemini 3.8 Flash | 90.6% | Task version 12; Evaluated 2026-09-04Original result | Not reported | Unknown |
| 5 | Gemini 3.7 Flash | 90.3% | Task version 12; Evaluated 2026-08-28Original result | Not reported | Unknown |
| 6 | GPT-6 Astra | 89.8% | Task version 12; Evaluated 2026-10-21Original result | Not reported | Unknown |
| 7 | Gemini 3.6 Flash | 89.8% | Task version 12; Evaluated 2026-07-23Original result | Not reported | Unknown |
| 8 | Gemini 3.5 Flash | 89.6% | Task version 12; Evaluated 2026-06-01Original result | Not reported | Unknown |
| 9 | Grok 4.5 | 89.5% | Task version 12; Evaluated 2026-07-21Original result | Not reported | Unknown |
| 10 | GPT-6.1 Sol | 89.4% | Task version 12; Evaluated 2026-10-05Original result | Not reported | Unknown |
| 11 | Gemini 3 Flash Preview | 89.4% | Task version 12; Evaluation date not reportedOriginal result | Not reported | Unknown |
| 12 | Claude Opus 4.8 | 88.7% | Task version 12; Evaluated 2026-05-29Original result | Not reported | Unknown |
| 13 | GPT-6 Sol | 88.1% | Task version 12; Evaluated 2026-10-01Original result | Not reported | Unknown |
| 14 | Claude Opus 4.1 | 87.9% | Task version 12; Evaluated 2025-08-21Original result | Not reported | Unknown |
| 15 | GPT-5 | 87.1% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 16 | Claude Opus 4 | 86.9% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 17 | Gemma 4 31B | 86.7% | Task version 12; Evaluated 2026-06-02Original result | Not reported | Unknown |
| 18 | Gemini 2.5 Pro Preview | 86.7% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 19 | Grok 4 | 86.6% | Task version 12; Evaluated 2025-09-04Original result | Not reported | Unknown |
| 20 | o3 | 86.4% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 21 | GPT-5.6 Terra | 86.3% | Task version 12; Evaluated 2026-07-20Original result | Not reported | Unknown |
| 22 | Deepseek V4.1 Flash | 86.3% | Task version 12; Evaluated 2026-10-20Original result | Not reported | Unknown |
| 23 | Claude Haiku 5.5 | 85.8% | Task version 12; Evaluated 2026-10-09Original result | Not reported | Unknown |
| 24 | GPT-6 Luna | 85.4% | Task version 12; Evaluated 2026-10-05Original result | Not reported | Unknown |
| 25 | Gemini 3.1 Flash-Lite Preview | 84.5% | Task version 12; Evaluation date not reportedOriginal result | Not reported | Unknown |
| 26 | GPT-5.6 Luna | 84.4% | Task version 12; Evaluated 2026-07-20Original result | Not reported | Unknown |
| 27 | GPT-5.6 Sol | 84.4% | Task version 12; Evaluated 2026-07-21Original result | Not reported | Unknown |
| 28 | DeepSeek-R1 | 84.0% | Task version 12; Evaluated 2025-07-02Original result | Not reported | Unknown |
| 29 | o1 | 83.9% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 30 | Claude Sonnet 4 | 83.7% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 31 | Gemini 3.5 Flash-Lite | 83.2% | Task version 12; Evaluated 2026-07-23Original result | Not reported | Unknown |
| 32 | deepseek-v3.1 | 83.1% | Task version 12; Evaluated 2025-08-27Original result | Not reported | Unknown |
| 33 | Gemini 2.5 Flash Preview | 82.9% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 34 | Grok 3 Mini | 81.2% | Task version 12; Evaluated 2025-06-23Original result | Not reported | Unknown |
| 35 | GPT-4.1 | 81.0% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 36 | gpt-oss-120b | 78.5% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 37 | DeepSeek-V3 | 78.2% | Task version 12; Evaluated 2025-07-09Original result | Not reported | Unknown |
| 38 | o4 mini | 77.9% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 39 | o3 mini | 77.7% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 40 | Claude 3.5 Sonnet | 77.6% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 41 | Grok 3 | 77.2% | Task version 12; Evaluated 2025-06-23Original result | Not reported | Unknown |
| 42 | Gemini 2.0 Flash | 76.1% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 43 | Grok 2 | 74.1% | Task version 12; Evaluated 2025-06-23Original result | Not reported | Unknown |
| 44 | GPT-4o | 73.0% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 45 | o1 mini | 72.0% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 46 | Claude 3.7 Sonnet | 71.9% | Task version 12; Evaluated 2025-07-09Original result | Not reported | Unknown |
| 47 | Gemini 1.5 Pro | 70.1% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 48 | Mistral Large 2 | 70.0% | Task version 12; Evaluated 2025-07-07Original result | Not reported | Unknown |
| 49 | gpt-oss-20b | 67.1% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 50 | Gemma 3 27B | 66.2% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 51 | Qwen 3 235B A22B Instruct 2506 | 66.1% | Task version 12; Evaluated 2026-07-21Original result | Not reported | Unknown |
| 52 | Gemini 1.5 Flash | 64.7% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 53 | GPT-4o mini | 62.4% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 54 | Claude 3.5 Haiku | 61.3% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 55 | Qwen 3 235B A22B Thinking | 58.4% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 56 | Gemma 3 12B | 58.1% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 57 | Granite 4.0 Small | 57.4% | Task version 12; Evaluated 2025-09-30Original result | Not reported | Unknown |
| 58 | Mixtral 8x22B | 55.1% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 59 | Codestral | 51.6% | Task version 12; Evaluated 2025-07-07Original result | Not reported | Unknown |
| 60 | Gemini 1.5 Flash 8B | 48.2% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 61 | GPT-3.5 Turbo | 46.9% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 62 | Ministral 8B | 42.8% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 63 | Gemma 3 4B | 41.1% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 64 | Ministral 3B | 38.3% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
According to BenchLM.ai, Qwen3.7 Max leads the MMLU-Pro benchmark with a score of 89.6%, followed by Claude Opus 4.5 (89.5%) and Qwen3.7 Plus (88.5%). The top models are clustered within 1.1 points, suggesting this benchmark is nearing saturation for frontier models.
81 models have been evaluated on MMLU-Pro. The benchmark falls in the Knowledge category. BenchAlign v5.8 gives MMLU-Pro 6% of the Knowledge reference weight, so it moves the Knowledge leaderboard and the overall ranking. Reference weights are relative weights in the calibrated model, not fixed shares of a score.
About MMLU-Pro
Year
2024
Tasks
Multiple subjects
Format
10-way multiple choice
Difficulty
Professional level
MMLU-Pro increases the number of choices from 4 to 10 and integrates more reasoning-focused problems, reducing the chance of correct guessing and better evaluating true understanding. It serves as a more robust discriminator of model capabilities.
Freshness and provenance
Version
MMLU-Pro
Refresh cadence
Static
Staleness state
Refreshing
Question availability
Public benchmark set
Refreshing
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MMLU-Pro measure?
An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.
Which model scores highest on MMLU-Pro?
Qwen3.7 Max by Alibaba currently leads with a score of 89.6% on MMLU-Pro.
How many models are evaluated on MMLU-Pro?
81 AI models have published results on MMLU-Pro in the BenchLM catalog.
Compare top models on MMLU-Pro
Last updated: November 12, 2026 · BenchLM version MMLU-Pro
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.