Benchmark profile

Vals-hosted MMLU-Pro mirror (Vals MMLU-Pro mirror)

Vals AI hosted MMLU-Pro view with subject-level task splits.

Data verified July 19, 2026

How BenchLM shows Vals MMLU-Pro mirror

BenchLM mirrors the public Vals AI Vals MMLU-Pro mirror leaderboard captured from https://www.vals.ai/benchmarks/mmlu_pro and updated by Vals on July 19, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.

Vals MMLU-Pro mirror is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.

122 Vals rows15 task viewspublic datasetTasks: Overall, Biology, Business, Chemistry, Computer ScienceDisplay only

Vals MMLU-Pro mirror on Vals AI Vals methodology Vals home

Vals MMLU-Pro mirror score on Vals MMLU-Pro mirror — July 19, 2026

BenchLM mirrors the published vals mmlu-pro mirror score view for Vals MMLU-Pro mirror. Claude Fable 5 leads the public snapshot at 91.50% , followed by Gemini 3.1 Pro Preview (90.99%) and Gemini 3 Pro Preview (90.10%). BenchLM does not use these results to rank models overall.

Claude Fable 5

Anthropic

anthropic/claude-fable-5

91.50%

Overall —

Gemini 3.1 Pro Preview

Google

google/gemini-3.1-pro-preview

90.99%

Overall —

Gemini 3 Pro Preview

Google

google/gemini-3-pro-preview

90.10%

Overall —

122 modelsExternal benchmark mirrorsCurrentDisplay onlyUpdated July 19, 2026

Vals MMLU-Pro mirror score table (122 models)

Score

Claude Fable 5Anthropic

91.50%

Gemini 3.1 Pro PreviewGoogle

90.99%

Gemini 3 Pro PreviewGoogle

90.10%

Claude Opus 4.7Anthropic

89.87%

Claude Opus 4.8Anthropic

89.58%

Gemini 3.5 FlashGoogle

89.52%

Qwen3.7 MaxAlibaba

89.31%

Grok 4.5SpaceXAI

89.22%

Claude Opus 4.6 ThinkingAnthropic

89.11%

GPT-5.6 SolOpenAI

89.10%

Muse Spark 1 1Meta

88.73%

Gemini 3 Flash PreviewGoogle

88.59%

GPT-5.5OpenAI

88.14%

Kimi K3Moonshot AI

87.97%

Claude Opus 4.1 20250805 ThinkingAnthropic

87.92%

Qwen3.6 PlusAlibaba

87.67%

Kimi K2.6Moonshot AI

87.57%

Claude Sonnet 5Anthropic

87.55%

GPT-5.4OpenAI

87.48%

Claude Sonnet 4.5 20250929 ThinkingAnthropic

87.36%

Claude Sonnet 4.6Anthropic

87.34%

Muse SparkMeta

87.32%

Claude Opus 4.5 20251101 ThinkingAnthropic

87.26%

DeepSeek V4 ProDeepSeek

87.25%

Claude Opus 4.1Anthropic

87.21%

Qwen3.5 Plus ThinkingAlibaba

87.18%

MiniMax M2.1MiniMax

87.05%

GLM 5.1Zhipu AI

86.90%

GLM 5.2Zhipu AI

86.71%

GPT-5.6 TerraOpenAI

86.66%

GPT-5OpenAI

86.54%

GPT-5.1OpenAI

86.38%

InklingThinkingmachines

86.30%

Grok 4.20 0309 ReasoningSpaceXAI

86.25%

Gemini 3.1 Flash Lite PreviewGoogle

86.24%

GPT-5.2OpenAI

86.23%

Claude Opus 4Anthropic

86.17%

GPT-5.6 LunaOpenAI

86.04%

GLM 5 ThinkingZhipu AI

86.03%

Kimi K2.5 ThinkingMoonshot AI

85.91%

Grok 4.3SpaceXAI

85.84%

Nemotron 3 Ultra 550b A55bNvidia

85.76%

O3OpenAI

85.59%

Claude Opus 4.5Anthropic

85.59%

Grok 4 0709SpaceXAI

85.30%

Qwen3 MaxAlibaba

84.98%

DeepSeek V3p2 ThinkingFireworks AI

84.92%

Mimo V2.5 ProXiaomi

84.59%

GPT-5.4 MiniOpenAI

84.55%

Qwen3 MaxAlibaba

84.36%

MiniMax M3MiniMax

84.22%

Grok 4.1 Fast ReasoningSpaceXAI

84.18%

Qwen3.5 FlashAlibaba

84.06%

Gemini 2.5 Pro Exp 03 25Google

84.06%

Claude Sonnet 4 20250514 ThinkingAnthropic

83.86%

Gemini 2.5 Flash Preview 09 2025Google

83.69%

Gemini 2.5 Flash Preview 09 2025 ThinkingGoogle

83.66%

Qwen3 Max PreviewAlibaba

83.54%

O1OpenAI

83.49%

DeepSeek R1Fireworks AI

83.18%

DeepSeek V3p2Fireworks AI

83.06%

Mimo V2.5Xiaomi

82.93%

GLM 4.7Zhipu AI

82.74%

Claude 3.7 Sonnet 20250219 ThinkingAnthropic

82.73%

GPT-5 MiniOpenAI

82.23%

GLM 4.6Zhipu AI

82.20%

Grok 3 Mini Fast High ReasoningSpaceXAI

81.37%

Qwen3 235b A22bFireworks AI

81.25%

GLM 4.5Zhipu AI

81.22%

Kimi K2 ThinkingMoonshot AI

81.07%

Claude 3.7 SonnetAnthropic

80.66%

O4 MiniOpenAI

80.56%

GPT-4.1OpenAI

80.50%

MiniMax M2.7MiniMax

80.43%

MiniMax M2.5MiniMax

80.09%

Grok 3 Mini Fast Low ReasoningSpaceXAI

80.01%

Grok 3SpaceXAI

79.95%

Mistral Large 2512Mistral AI

79.82%

Grok 4 Fast ReasoningSpaceXAI

79.70%

DeepSeek V3 0324Fireworks AI

79.47%

Claude Sonnet 4Anthropic

79.43%

Llama4 Maverick Instruct BasicFireworks AI

79.42%

Moonshotai Kimi K2 InstructTogether AI

79.39%

GPT Oss 120bFireworks AI

79.17%

Gemini 2.5 Flash Lite Preview 09 2025 ThinkingGoogle

79.12%

Claude Haiku 4.5 20251001 ThinkingAnthropic

78.72%

O3 MiniOpenAI

78.69%

Gemini 2.5 Flash Lite Preview 09 2025Google

78.64%

Claude 3.5 SonnetAnthropic

78.40%

Gemini 2.0 Flash 001Google

77.38%

GPT-4.1 MiniOpenAI

77.22%

GPT-5.4 NanoOpenAI

77.17%

GPT-5 NanoOpenAI

76.07%

Grok 2 1212SpaceXAI

75.47%

Mistral Medium 3.5Mistral AI

75.33%

Gemini 1.5 Pro 002Google

75.29%

Mistral Medium 2505Mistral AI

75.29%

Grok 4.1 Fast Non ReasoningSpaceXAI

75.21%

GPT-4oOpenAI

74.13%

100

DeepSeek V3Fireworks AI

73.82%

101

GPT-4oOpenAI

72.56%

102

GPT Oss 20bFireworks AI

71.64%

103

Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561Together AI

70.78%

104

Grok 4 Fast Non ReasoningSpaceXAI

70.34%

105

Meta Llama Llama 3.3 70B Instruct TurboTogether AI

69.86%

106

Mistral Large 2411Mistral AI

69.71%

107

Meta Llama Llama 4 Scout 17B 16E InstructTogether AI

69.63%

108

Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561 ThinkingTogether AI

69.58%

109

Laguna Xs.2Poolside

69.41%

110

Command A 03 2025Cohere

69.17%

111

Magistral Medium 2509Mistral AI

68.66%

112

Mistral Small 2503Mistral AI

66.02%

113

Gemini 1.5 Flash 002Google

65.61%

114

Mistral Small 2402Mistral AI

64.44%

115

Claude 3.5 HaikuAnthropic

64.12%

116

GPT-4.1 NanoOpenAI

63.48%

117

Laguna M.1Poolside

63.48%

118

GPT-4o MiniOpenAI

62.73%

119

Magistral Small 2509Mistral AI

62.13%

120

Jamba Large 1.6AI21 Labs

49.78%

121

Command R PlusCohere

44.00%

122

Jamba Mini 1.6AI21 Labs

30.28%

The published Vals MMLU-Pro mirror snapshot places Claude Fable 5 first at 91.50%. The third row is 1.40 points behind. The broader top-10 range is 2.40 points, so many of the published results sit in a relatively narrow band.

122 models have been evaluated on Vals MMLU-Pro mirror. The benchmark falls in the External benchmark mirrors category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Vals MMLU-Pro mirror is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Vals MMLU-Pro mirror

Year

2026

Tasks

MMLU-Pro subject splits

Format

Accuracy score

Difficulty

Professional academic reasoning

BenchLM keeps this Vals-hosted MMLU-Pro table separate from canonical MMLU-Pro source records.

Vals MMLU-Pro Public benchmark source

BenchLM freshness & provenance

Version

Vals MMLU-Pro mirror 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

What does Vals MMLU-Pro mirror measure?

Vals AI hosted MMLU-Pro view with subject-level task splits.

Which model leads the published Vals MMLU-Pro mirror snapshot?

Claude Fable 5 currently leads the published Vals MMLU-Pro mirror snapshot with 91.50% vals mmlu-pro mirror score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on Vals MMLU-Pro mirror?

122 AI models are included in BenchLM's mirrored Vals MMLU-Pro mirror snapshot, based on the public leaderboard captured on July 19, 2026.

Last updated: July 19, 2026 · mirrored from the public benchmark leaderboard

Choose a model with this week’s evidence

Join 2,000+ readers for ranking moves, pricing changes, and the claims that still need proof.

One email each week. Unsubscribe anytime.