Skip to main content
BenchLM
Advertise

Benchmark profile

Korean Massive Multitask Language Understanding (KMMLU)

Evaluates Korean expert-level knowledge across 45 subjects. 20% of questions require Korean cultural context.

Data verified

How BenchLM shows KMMLU right now

BenchLM is tracking KMMLU in the local dataset, but exact-source verification records for these rows are still being attached. To avoid a blank benchmark page, BenchLM shows the current tracked rows below as a display-only reference table.

These tracked rows are useful for inspection and spot-checking, but until exact-source attachments are completed they should not be treated as fully verified public benchmark rows.

17 tracked modelsLocal tracked rowsAwaiting exact-source attachmentsDisplay only
KMMLU: Measuring Massive Multitask Language Understanding in Korean

Tracked score on KMMLU — July 23, 2026

BenchLM mirrors the published tracked score view for KMMLU. Claude Sonnet 4.6 leads the public snapshot at 85% , followed by GPT-5.4 (83.7%) and Solar Pro 2 (80.1%). BenchLM does not use these results to rank models overall.

1

Claude Sonnet 4.6

Anthropic

claude-sonnet-4-6

85%
Overall —
2

GPT-5.4

OpenAI

gpt-5-4

83.7%
Overall —
3

Solar Pro 2

Upstage

solar-pro-2

80.1%
Overall —
17 modelsKorean BenchmarksKorean-language benchmarkRefreshingDisplay onlyUpdated July 23, 2026

Tracked score table (17 models)

Score
1
Claude Sonnet 4.6Anthropic
85%
2
GPT-5.4OpenAI
83.7%
3
Solar Pro 2Upstage
80.1%
4
o1OpenAI
79.5%
5
HyperClova X Think 32BNaver Cloud
78.4%
6
A.X seriesSK Telecom
78%
7
GPT-5 miniOpenAI
76.5%
8
Exaone 4.0 32BLG AI Research
75.2%
9
GPT-5.2OpenAI
71.5%
10
GPT-5 nanoOpenAI
69.3%
11
GPT-5.1OpenAI
65.9%
12
GPT-4.1OpenAI
65.5%
13
GPT-4oOpenAI
64.3%
14
GPT-4.1 miniOpenAI
59.3%
15
GPT-4 TurboOpenAI
58.8%
16
GPT-4o miniOpenAI
52.6%
17
GPT-4.1 nanoOpenAI
48.6%

The published KMMLU snapshot places Claude Sonnet 4.6 first at 85%. The third row is 4.9 points behind. The broader top-10 range is 15.7 points, so the table still separates the published systems.

17 models have been evaluated on KMMLU. The benchmark falls in the Korean Benchmarks category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. KMMLU is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About KMMLU

Year

2024

Tasks

35,030 questions

Format

Multiple choice questions

Difficulty

Elementary to professional level in Korean

Tests human-level understanding and reasoning in the Korean language across diverse subjects.

KMMLU: Measuring Massive Multitask Language Understanding in KoreanPublic benchmark source

BenchLM freshness & provenance

Version

KMMLU 2024

Refresh cadence

Annual

Staleness state

Refreshing

Question availability

Public benchmark set

RefreshingDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

What does KMMLU measure?

Evaluates Korean expert-level knowledge across 45 subjects. 20% of questions require Korean cultural context.

Which model leads the published KMMLU snapshot?

Claude Sonnet 4.6 currently leads the published KMMLU snapshot with 85% tracked score. BenchLM shows this benchmark for display only and does not use it in overall rankings.

How many models are evaluated on KMMLU?

17 AI models are included in BenchLM's mirrored KMMLU snapshot, based on the public leaderboard captured on July 23, 2026.

Last updated: July 23, 2026 · mirrored from the public benchmark leaderboard

Choose a model with this week’s evidence

Join 2,000+ readers for ranking moves, pricing changes, and the claims that still need proof.

One email each week. Unsubscribe anytime.