●●●●High67.22provisional score
Benchmark Confidence & Contamination Flags
Not all benchmark scores are equally trustworthy. BenchLM now separates verified ranking from provisionalranking while still tracking the provenance of every stored score. The confidence indicator (1-4 dots) shows how much sourced benchmark coverage supports each model's score.
●●●●High
7+ categories, 20+ non-generated benchmarks
●●●○Good
5+ categories, 12+ non-generated benchmarks
●●○○Moderate
3+ categories, 8+ non-generated benchmarks
●○○○Low / Estimated
Limited sourced data, score is estimated
Confidence Distribution (Ranked Models)
8
High (4%)
20
Good (10%)
22
Moderate (11%)
150
Low / Estimated (75%)
How BenchLM Scores Work
Verified, provisional, and generated
Each benchmark value is tagged as manual (a hand-entered public row) or generated (inferred from related models). Generated rows are excluded from all public ranking logic. Manual rows are now split again into sourced rows for the verified leaderboard and source-unverified rows that can still appear in provisional mode.
Ranking Eligibility
A model must have at least 8 qualifying benchmarks across 2+ categories to rank in a lane. The provisional leaderboard uses rankable non-generated rows; the verified leaderboard uses sourced rows only. Models below the threshold are shown as tracked but unranked.
Category Eligibility
For category leaderboards, a model needs qualifying scores on at least half of the weighted benchmarks in that category. BenchLM computes this separately for provisional and verified ranking so sparse exact-source coverage cannot silently borrow strength from provisional rows.
Display-Only Benchmarks
Some benchmarks (MMLU, BBH, HumanEval, older AIME/HMMT variants) are shown for context but don't affect scoring. These are either saturated (top models all score 97%+) or have been superseded by harder versions.
200 models match these filters.
Model / evidenceConfidence / score
●●●●High64.22provisional score
●●●●High59.66provisional score
●●●●High65.2provisional score
●●●●High66.06provisional score
●●●●High57.01provisional score
●●●●High72.84provisional score
●●●●High64.75provisional score
●●●○Good51.47provisional score
●●●○Good53.82provisional score
●●●○Good68.59provisional score
●●●○Good74.24provisional score
●●●○Good73.51provisional score
●●●○Good56.79provisional score
●●●○Good78.34provisional score
●●●○Good71.04provisional score
●●●○Good81.96provisional score
●●●○Good72.57provisional score
●●●○Good67.17provisional score
●●●○Good66.27provisional score
●●●○Good55.3provisional score
●●●○Good65.07provisional score
●●●○Good67.54provisional score
●●●○Good58.43provisional score
●●●○Good60.56provisional score
●●●○Good60.7provisional score
●●●○Good56.97provisional score
●●●○Good56.77provisional score
●●○○Moderate55.47provisional score
●●○○Moderate53.95provisional score
●●○○Moderate60.66provisional score
●●○○Moderate58.88provisional score
●●○○Moderate77.44provisional score
●●○○Moderate67.74provisional score
●●○○Moderate69.75provisional score
●●○○Moderate54.68provisional score
●●○○Moderate83.93provisional score
●●○○Moderate65.32provisional score
●●○○Moderate63.96provisional score
●●○○Moderate44.24provisional score
●●○○Moderate83.68provisional score
●●○○Moderate67.73provisional score
●●○○Moderate66.79provisional score
●●○○Moderate60.89provisional score
●●○○Moderate47.29provisional score
●●○○Moderate59.72provisional score
●●○○Moderate61.16provisional score
●●○○Moderate53.61provisional score
●●○○Moderate59.35provisional score
●●○○Moderate58.62provisional score
●○○○Low / Estimated~80.96provisional score
●○○○Low / Estimated~64.11provisional score
●○○○Low / Estimated~50.87provisional score
●○○○Low / Estimated~66.69provisional score
●○○○Low / Estimated~70.19provisional score
●○○○Low / Estimated~63.69provisional score
●○○○Low / Estimated~76.72provisional score
●○○○Low / Estimated~71.94provisional score
●○○○Low / Estimated~55.4provisional score
●○○○Low / Estimated~65.1provisional score
●○○○Low / Estimated~41.42provisional score
●○○○Low / Estimated~61.08provisional score
●○○○Low / Estimated~51.11provisional score
●○○○Low / Estimated~60.49provisional score
●○○○Low / Estimated~47.41provisional score
●○○○Low / Estimated~44.19provisional score
●○○○Low / Estimated~57.25provisional score
●○○○Low / Estimated~48.1provisional score
●○○○Low / Estimated~56.58provisional score
●○○○Low / Estimated~56.02provisional score
●○○○Low / Estimated~47.2provisional score
●○○○Low / Estimated~57.96provisional score
●○○○Low / Estimated~42.06provisional score
●○○○Low / Estimated~55.94provisional score
●○○○Low / Estimated~52.32provisional score
●○○○Low / Estimated~53.65provisional score
●○○○Low / Estimated~60.42provisional score
●○○○Low / Estimated~55.12provisional score
●○○○Low / Estimated~43.87provisional score
●○○○Low / Estimated~47.51provisional score
●○○○Low / Estimated~40.44provisional score
●○○○Low / Estimated~45.88provisional score
●○○○Low / Estimated~42.79provisional score
●○○○Low / Estimated~50.83provisional score
●○○○Low / Estimated~49.98provisional score
●○○○Low / Estimated~27.19provisional score
●○○○Low / Estimated~47.89provisional score
●○○○Low / Estimated~58.01provisional score
●○○○Low / Estimated~48.09provisional score
●○○○Low / Estimated~47.65provisional score
●○○○Low / Estimated~40.43provisional score
●○○○Low / Estimated~47.74provisional score
●○○○Low / Estimated~58.61provisional score
●○○○Low / Estimated~59.1provisional score
●○○○Low / Estimated~55provisional score
●○○○Low / Estimated~52.72provisional score
●○○○Low / Estimated~54.06provisional score
●○○○Low / Estimated~61.31provisional score
●○○○Low / Estimated~44.97provisional score
●○○○Low / Estimated~41.49provisional score
●○○○Low / Estimated~39.87provisional score
●○○○Low / Estimated~23.49provisional score
●○○○Low / Estimated~64.18provisional score
●○○○Low / Estimated~59.77provisional score
●○○○Low / Estimated~66.89provisional score
●○○○Low / Estimated~67.78provisional score
●○○○Low / Estimated~63.5provisional score
●○○○Low / Estimated~60.51provisional score
●○○○Low / Estimated~63.15provisional score
●○○○Low / Estimated~58.15provisional score
●○○○Low / Estimated~56.59provisional score
●○○○Low / Estimated~59.52provisional score
●○○○Low / Estimated~50.08provisional score
●○○○Low / Estimated~54.48provisional score
●○○○Low / Estimated~42.74provisional score
●○○○Low / Estimated~50.08provisional score
●○○○Low / Estimated~43.93provisional score
●○○○Low / Estimated~48.16provisional score
●○○○Low / Estimated~57.44provisional score
●○○○Low / Estimated~59.97provisional score
●○○○Low / Estimated~59.5provisional score
●○○○Low / Estimated~59provisional score
●○○○Low / Estimated~67.01provisional score
●○○○Low / Estimated~58.9provisional score
●○○○Low / Estimated~51.25provisional score
●○○○Low / Estimated~53.43provisional score
●○○○Low / Estimated~53.64provisional score
●○○○Low / Estimated~50.4provisional score
●○○○Low / Estimated~57.56provisional score
●○○○Low / Estimated~51.67provisional score
●○○○Low / Estimated~56.91provisional score
●○○○Low / Estimated~55.1provisional score
●○○○Low / Estimated~49.11provisional score
●○○○Low / Estimated~47.7provisional score
●○○○Low / Estimated~51.25provisional score
●○○○Low / Estimated~41.57provisional score
●○○○Low / Estimated~46.23provisional score
●○○○Low / Estimated~54.53provisional score
●○○○Low / Estimated~51.28provisional score
●○○○Low / Estimated~55.15provisional score
●○○○Low / Estimated~41.13provisional score
●○○○Low / Estimated~52.94provisional score
●○○○Low / Estimated~37.87provisional score
●○○○Low / Estimated~41.77provisional score
●○○○Low / Estimated~52.15provisional score
●○○○Low / Estimated~51.71provisional score
●○○○Low / Estimated~50.97provisional score
●○○○Low / Estimated~50.82provisional score
●○○○Low / Estimated~34.7provisional score
●○○○Low / Estimated~50.28provisional score
●○○○Low / Estimated~50.23provisional score
●○○○Low / Estimated~35.71provisional score
●○○○Low / Estimated~22.69provisional score
●○○○Low / Estimated~49.89provisional score
●○○○Low / Estimated~49.89provisional score
●○○○Low / Estimated~49.84provisional score
●○○○Low / Estimated~49.34provisional score
●○○○Low / Estimated~48.33provisional score
●○○○Low / Estimated~34.47provisional score
●○○○Low / Estimated~48.66provisional score
●○○○Low / Estimated~48.54provisional score
●○○○Low / Estimated~38.64provisional score
●○○○Low / Estimated~27.44provisional score
●○○○Low / Estimated~45.13provisional score
●○○○Low / Estimated~45.08provisional score
●○○○Low / Estimated~45.08provisional score
●○○○Low / Estimated~44.99provisional score
●○○○Low / Estimated~44.79provisional score
●○○○Low / Estimated~44.6provisional score
●○○○Low / Estimated~44.4provisional score
●○○○Low / Estimated~21.78provisional score
●○○○Low / Estimated~21.39provisional score
●○○○Low / Estimated~18.87provisional score
●○○○Low / Estimated~36.55provisional score
●○○○Low / Estimated~40.38provisional score
●○○○Low / Estimated~20.32provisional score
●○○○Low / Estimated~20.96provisional score
●○○○Low / Estimated~39.9provisional score
●○○○Low / Estimated~39.85provisional score
●○○○Low / Estimated~39.8provisional score
●○○○Low / Estimated~16.23provisional score
●○○○Low / Estimated~39.52provisional score
●○○○Low / Estimated~25.12provisional score
●○○○Low / Estimated~18.27provisional score
●○○○Low / Estimated~39.13provisional score
●○○○Low / Estimated~15.52provisional score
●○○○Low / Estimated~55.64provisional score
●○○○Low / Estimated~41.82provisional score
●○○○Low / Estimated~43.2provisional score
●○○○Low / Estimated~42.97provisional score
●○○○Low / Estimated~40.7provisional score
●○○○Low / Estimated~43.2provisional score
●○○○Low / Estimated~38.32provisional score
●○○○Low / Estimated~38.32provisional score
●○○○Low / Estimated~46.36provisional score
●○○○Low / Estimated~42.58provisional score
●○○○Low / Estimated~45.94provisional score
●○○○Low / Estimated~39.06provisional score
●○○○Low / Estimated~48.45provisional score
●○○○Low / Estimated~41.19provisional score
Sourced = exact-source benchmark coverage. Rankable = non-generated benchmark coverage used by the provisional leaderboard. Generated = inferred from related models and excluded from ranking. Coverage = sourced share of the visible benchmark footprint.
Frequently Asked Questions
What is benchmark confidence on BenchLM?
Score confidence (1-4 dots) indicates how much sourced benchmark data supports a model's score. A 4-dot score has at least 20 sourced rows across seven categories. A 1-dot score has limited coverage. Some benchmark-sparse models can appear provisionally when several independent public evaluation families agree.
What does "estimated" mean on BenchLM scores?
Scores marked with "Est." or "~" have limited source-backed benchmark coverage or qualify through corroborating external evaluation families. Unresolved manual values and generated rows cannot score. The verified leaderboard requires sourced benchmark support; external consensus alone never upgrades an estimated row to verified status.
How does BenchLM detect contamination risk?
BenchLM tracks two key signals: (1) benchmark provenance — whether each score is a hand-entered public row ("manual") or was generated/inferred from related data, and (2) benchmark freshness — older benchmarks that haven't been updated are more likely to have been contaminated through training data inclusion. Models with mostly generated data or stale benchmarks receive lower confidence ratings. Exact-source verification is tracked separately from this manual-vs-generated split.
What is benchmark provenance?
Provenance records where each benchmark value came from and whether the exact model, benchmark variant, harness, and score can be supported. Source-backed rows can score. Unresolved manual values and generated estimates remain in the audit trail but do not affect either public ranking view.
Which LLM benchmarks are most reliable?
Fresh, held-out benchmarks like SWE-Rebench (rolling window), Terminal-Bench 2.0, and HLE are the hardest to game. Older, saturated benchmarks like MMLU (where top models all score 97-99%) provide little signal. BenchLM weights newer, harder benchmarks more heavily and flags saturated ones as display-only.
Choose a model with this week’s evidence
Join 2,000+ readers for ranking moves, pricing changes, and the claims that still need proof.
One email each week. Unsubscribe anytime.