MMLU is saturated: frontier models score 97-99% and the top 5 models are separated by just 2 points. MMLU-Pro fixes this with 10-choice questions and harder reasoning problems, creating a meaningful 88-93 spread that actually differentiates today's best models.
MMLU (Massive Multitask Language Understanding) has been the go-to knowledge benchmark since 2020. It tests models across 57 academic subjects with multiple-choice questions ranging from elementary to professional difficulty. But with frontier models now scoring 97-99%, it's lost its ability to separate the best from the rest.
MMLU-Pro was designed to fix this.
How MMLU works, and what MMLU-Pro changes
MMLU presents 4-choice multiple-choice questions across subjects like history, biology, computer science, law, and mathematics. A model reads a question and picks A, B, C, or D.
With 4 choices, random guessing gives you 25%. Early models struggled to beat 40-50%. Today's frontier models score 97-99%, meaning the benchmark is effectively saturated.
See current scores: MMLU leaderboard
MMLU-Pro makes three key improvements:
- 10 answer choices instead of 4 — Random guessing drops from 25% to 10%, reducing the role of luck
- More reasoning-focused questions — Harder questions that require multi-step thinking, not just recall
- Better discrimination — Top model scores range from ~88-93 instead of 97-99, creating meaningful separation
This makes MMLU-Pro a much better benchmark for comparing frontier models. A 5-point gap on MMLU-Pro is more informative than a 1-point gap on MMLU.
See current scores: MMLU-Pro leaderboard
Current rankings comparison
| Model | MMLU | MMLU-Pro |
|---|---|---|
| GPT-5.4 | 99 | 93 |
| Claude Opus 4.6 | 99 | 82 |
| GPT-5.3 Codex | 99 | 90 |
| GPT-5.2 | 99 | 88 |
| Gemini 3.1 Pro | 97 | 92 |
On MMLU, the top models are within 2 points. On MMLU-Pro, the spread widens to 11 points. That's the difference between "all models are basically the same" and "there are real performance differences here."
Which should you look at?
If you're evaluating models in 2026:
- Use MMLU-Pro for comparing frontier models. It's harder and better at showing real differences.
- Use MMLU as a baseline: a model scoring below 90 on MMLU may not be competitive for knowledge-intensive tasks.
- Combine with HLE: Humanity's Last Exam is even harder and shows the largest spread among frontier models (10-46% scores).
For a complete view, check our knowledge benchmark rankings or compare specific models on their benchmark detail pages.
→ See all models ranked on the full leaderboard
Reader questions
Frequently asked questions
01What is MMLU?
MMLU (Massive Multitask Language Understanding) is a benchmark that tests AI models across 57 academic subjects with 4-choice multiple-choice questions. Introduced in 2020, it covers everything from history and biology to law and mathematics. Frontier models now score 97-99%, meaning it is effectively saturated and no longer useful for comparing the best models.
02What is MMLU-Pro and how does it differ from MMLU?
MMLU-Pro extends MMLU with 10-choice questions instead of 4, harder reasoning-focused problems, and a wider score spread (88-93 for top models vs 97-99 for MMLU). The extra answer choices reduce the role of lucky guessing and force models to reason rather than recall. MMLU-Pro is the better benchmark for comparing frontier models in 2026.
03Is MMLU still a useful benchmark in 2026?
MMLU has limited value for comparing frontier models because the top 5 models all score 97-99%, leaving almost no signal. It remains useful as a baseline — a model scoring below 90 on MMLU is unlikely to be competitive for knowledge-intensive tasks — but MMLU-Pro and HLE are better discriminators at the frontier.
04Which model scores highest on MMLU-Pro?
As of July 2026, GPT-5.4 leads MMLU-Pro with a score of 93, followed by Gemini 3.1 Pro at 92 and GPT-5.3 Codex at 90. See the full rankings on the BenchLM.ai MMLU-Pro leaderboard.
05What benchmark should I use instead of MMLU?
For comparing frontier models, use MMLU-Pro for broad knowledge depth, SuperGPQA for breadth across 285 disciplines, or HLE (Humanity's Last Exam) for the hardest questions where top models score only 10-46%. Any of these provides more signal than MMLU for today's frontier models.
Share or save