LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (LiveCodeBench)
We show this table for reference; we do not rank on it.
Solar Pro 4 has the highest reported LiveCodeBench score at 87.8%, ahead of DeepSeek V3.1 (Reasoning) (74.8%) and Grok 3 Mini (74.8%) among 13 models. We compile the table from provider self-reports and secondary reports and keep it for display only; it does not affect overall rankings.
A continuously updated coding benchmark built from newly collected LeetCode, AtCoder, and Codeforces problems. Fresh problem windows reduce one contamination path, but results still need a release and setup check.
Benchmark score on LiveCodeBench — November 12, 2026
We compile the LiveCodeBench rows from provider self-reports and secondary reports. Solar Pro 4 leads the table at 87.8%, followed by DeepSeek V3.1 (Reasoning) (74.8%) and Grok 3 Mini (74.8%). We do not use these results to rank models overall.
Solar Pro 4
Upstage
Reported by UpstageDeepSeek V3.1 (Reasoning)
DeepSeek
Reported by DeepSeekGrok 3 Mini
xAI
Reported by xAI13 modelsCodingCurrentDisplay onlyUpdated November 12, 2026
API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.
Benchmark score table (13 models)
None of the 13 results on this page was run by the benchmark or an independent evaluator: 12 reported by the labs that built the models and 1 carried from an upstream page, where who ran it is not recorded.
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|
| 1 | Solar Pro 4UpstageReported by Upstage | 87.8% | Reported by Upstage | Not reported | Closed | $0.3 / $1.2 | 512K | Proprietary |
| 2 | DeepSeek V3.1 (Reasoning)DeepSeekReported by DeepSeek | 74.8% | Reported by DeepSeek | Not reported | Open | Not recorded | 128K | Not recorded |
| 3 | Grok 3 MinixAIReported by xAI | 74.8% | Reported by xAI | Not reported | Closed | $0.3 / $0.5 | 128K | Proprietary |
| 4 | DeepSeek-R1-0528DeepSeekReported by DeepSeek | 73.3% | Reported by DeepSeek | Not reported | Open | Not recorded | 128K | Not recorded |
| 5 | Mercury 2InceptionReported by Inception | 67.3% | Reported by Inception | Not reported | Closed | $0.25 / $0.75 | 128K | Proprietary |
| 6 | Nemotron Ultra 253BNVIDIAReported by NVIDIA | 66.3% | Reported by NVIDIA | Not reported | Open | Not recorded | 32K | Not recorded |
| 7 | MiniMax M1 80kMiniMaxReported by MiniMax | 65% | Reported by MiniMax | Not reported | Closed | Not recorded | 80K | Proprietary |
| 8 | Grok 3 [Beta]xAIReported by xAI | 57% | Reported by xAI | Not reported | Closed | Not recorded | 128K | Proprietary |
| 9 | DeepSeek V3.1DeepSeekReported by DeepSeek | 56.4% | Reported by DeepSeek | Not reported | Open | Not recorded | 128K | Not recorded |
| 10 | Phi-4-reasoningMicrosoftReported by Microsoft | 53.8% | Reported by Microsoft | Not reported | Open | Not recorded | 32K | Not recorded |
| 11 | Phi-4-reasoning-plusMicrosoftReported by Microsoft | 53.1% | Reported by Microsoft | Not reported | Open | Not recorded | 32K | Not recorded |
| 12 | Qwen2.5-MaxAlibabaReported by Alibaba | 38.7% | Reported by Alibaba | Not reported | Closed | Not recorded | Unknown | Proprietary |
| 13 | DeepSeek V3DeepSeekCarried from an upstream page | 37.6% | Carried from an upstream page | Not reported | Open | $0.27 / $1.1 | 128K | MIT |
- Lab-reported
- Benchmark-run
- Independent
How to read this leaderboard
Prior editorial review by Glevd · 2026-07-15
Variant check: Catalog input/output prices and context limits describe current model access. They do not record the benchmark run’s token use, retries, or actual cost. A model’s weight-access label is separate from its license terms.
Use LiveCodeBench as a competitive-programming signal after matching the release, date window, task scenario, pass@k metric, sampling count, temperature, and execution policy. Explicit v5, v6, and Pass@1-COT rows now have separate lanes; remaining generic rows can still use different or incompletely labeled windows.
Operator receipt: 13 sourced rows are currently displayable on this page; the leading published row is Solar Pro 4 at 87.8%.
Honest limit: LiveCodeBench does not test work inside an existing repository. Release and date-window choice, the pruned versus original task set, generation settings, timeouts, and known autograding issues can affect a score. This legacy lane is a sourced result ledger, not one controlled BenchLM rerun.
Among the reported LiveCodeBench rows, Solar Pro 4 is first at 87.8%. The third row is 13.0 points behind. The broader top-10 range is 34.0 points, so the table still separates the published systems.
13 models have been evaluated on LiveCodeBench. The benchmark falls in the Coding category. LiveCodeBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About LiveCodeBench
Year
2024
Tasks
Continuously updated contest problems
Format
Competitive-programming evaluation
Difficulty
Competitive programming level
The official suite evaluates code generation, code execution, test-output prediction, and self-repair on contest problems added over time. Its releases and selectable date windows let evaluators separate older and newer problem sets instead of treating the benchmark as one fixed test.
Freshness and provenance
Version
Rolling 2026 set
Refresh cadence
Rolling
Staleness state
Current
Question availability
Delayed public release
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does LiveCodeBench measure?
LiveCodeBench evaluates coding on newly collected contest problems from LeetCode, AtCoder, and Codeforces. Its suite covers code generation, execution, test-output prediction, and self-repair, although provider tables commonly report code-generation pass rates. It tests competitive-programming correctness, not work inside an existing software repository.
Are LiveCodeBench scores directly comparable?
Only when the release, date window, scenario, pass@k metric, sampling count, temperature, and execution policy match. The official benchmark can change as new problems arrive, while providers may report different slices. BenchLM preserves sourced published rows, so unmatched settings support a directional read, not a precise ranking.
Can LiveCodeBench pick the best coding agent?
No. A high score supports competitive-programming performance under the reported setup. It does not establish codebase navigation, issue interpretation, tool use, patch review, or regression safety. Pair LiveCodeBench with a repository benchmark such as SWE-bench Pro and a trial drawn from your own languages, libraries, and runtime constraints.
Compare top models on LiveCodeBench
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.