Skip to main content
BenchLM
Advertise

Benchmark profile

KMMLU-Hard

A filtered hard subset of KMMLU containing ~5,000 questions that most models get wrong.

Data verified

How BenchLM shows KMMLU-Hard right now

BenchLM is tracking KMMLU-Hard in the local dataset, but exact-source verification records for these rows are still being attached. To avoid a blank benchmark page, BenchLM shows the current tracked rows below as a display-only reference table.

These tracked rows are useful for inspection and spot-checking, but until exact-source attachments are completed they should not be treated as fully verified public benchmark rows.

11 tracked modelsLocal tracked rowsAwaiting exact-source attachmentsDisplay only
Evaluating LLMs on Hard Korean Queries

Tracked score on KMMLU-Hard — July 23, 2026

BenchLM mirrors the published tracked score view for KMMLU-Hard. GPT-5.4 leads the public snapshot at 72.8% , followed by GPT-5 mini (60.6%) and GPT-5 nano (51.7%). BenchLM does not use these results to rank models overall.

1

GPT-5.4

OpenAI

gpt-5-4

72.8%
Overall —
2

GPT-5 mini

OpenAI

gpt-5-mini

60.6%
Overall —
3

GPT-5 nano

OpenAI

gpt-5-nano

51.7%
Overall —
11 modelsKorean BenchmarksKorean-language benchmarkCurrentDisplay onlyUpdated July 23, 2026

Tracked score table (11 models)

Score
1
GPT-5.4OpenAI
72.8%
2
GPT-5 miniOpenAI
60.6%
3
GPT-5 nanoOpenAI
51.7%
4
GPT-5.2OpenAI
51.1%
5
GPT-5.1OpenAI
43.9%
6
GPT-4.1OpenAI
42.8%
7
GPT-4oOpenAI
39.6%
8
GPT-4.1 miniOpenAI
35.6%
9
GPT-4 TurboOpenAI
30.6%
10
GPT-4o miniOpenAI
24.6%
11
GPT-4.1 nanoOpenAI
24.3%

The published KMMLU-Hard snapshot places GPT-5.4 first at 72.8%. The third row is 21.0 points behind. The broader top-10 range is 48.2 points, so the table still separates the published systems.

11 models have been evaluated on KMMLU-Hard. The benchmark falls in the Korean Benchmarks category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. KMMLU-Hard is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About KMMLU-Hard

Year

2025

Tasks

~5,000 questions

Format

Multiple choice questions

Difficulty

Advanced Korean reasoning

Provides strong signals for advanced frontier models attempting reasoning in Korean.

Evaluating LLMs on Hard Korean QueriesPublic benchmark source

BenchLM freshness & provenance

Version

KMMLU-Hard 2025

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

What does KMMLU-Hard measure?

A filtered hard subset of KMMLU containing ~5,000 questions that most models get wrong.

Which model leads the published KMMLU-Hard snapshot?

GPT-5.4 currently leads the published KMMLU-Hard snapshot with 72.8% tracked score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on KMMLU-Hard?

11 AI models are included in BenchLM's mirrored KMMLU-Hard snapshot, based on the public leaderboard captured on July 23, 2026.

Last updated: July 23, 2026 · mirrored from the public benchmark leaderboard

Choose a model with this week’s evidence

Join 2,000+ readers for ranking moves, pricing changes, and the claims that still need proof.

One email each week. Unsubscribe anytime.