SWE-bench Pro
Claude Fable 5.2 leads the SWE-bench Pro leaderboard on BenchLM's November 2026 update with 93%, ahead of Claude Opus 5.5 (89.9%) and Claude Sonnet 5.5 (81.3%), across 87 models.
A long-horizon repository benchmark built to test realistic software engineering work. Its scores need a task-quality and setup check before they support a coding-agent decision.
Top models on SWE-bench Pro — November 12, 2026
As of November 12, 2026, Claude Fable 5.2 leads the SWE-bench Pro leaderboard with 93% , followed by Claude Opus 5.5 (89.9%) and Claude Sonnet 5.5 (81.3%).
Claude Fable 5.2
Anthropic
Reported by AnthropicClaude Opus 5.5
Anthropic
Reported by AnthropicClaude Sonnet 5.5
Anthropic
Reported by Anthropic87 modelsCoding26% of Coding reference weightCurrentUpdated November 12, 2026
API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.
Leaderboard (87 models)
None of the 87 results on this page was run by the benchmark or an independent evaluator: 83 reported by the labs that built the models and 4 reported by a third party.
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2AnthropicReported by Anthropic | 93% | Reported by Anthropic | Not reported | Closed | $8 / $40 | 1M | Proprietary |
| 2 | Claude Opus 5.5AnthropicReported by Anthropic | 89.9% | Reported by Anthropic | Not reported | Closed | $4 / $20 | 1M | Proprietary |
| 3 | Claude Sonnet 5.5AnthropicReported by Anthropic | 81.3% | Reported by Anthropic | Not reported | Closed | $2 / $10 | 1M | Proprietary |
| 4 | Claude Fable 5.1AnthropicReported by Anthropic | 81.2% | Reported by Anthropic | Not reported | Closed | $10 / $50 | 1M | Proprietary |
| 5 | Claude Mythos 5AnthropicReported by Anthropic | 80.3% | Reported by Anthropic | Not reported | Closed | $10 / $50 | 1M+ | Proprietary |
| 6 | Claude Fable 5AnthropicReported by Anthropic | 80% | Reported by Anthropic | Not reported | Closed | $10 / $50 | 1M+ | Proprietary |
| 7 | Claude Opus 5AnthropicReported by Anthropic | 79.2% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 8 | Sakana Fugu-UltraSakana AIReported by Sakana AI | 73.7% | Reported by Sakana AI | Not reported | Closed | Not recorded | 1M | Proprietary |
| 9 | Claude Opus 4.8AnthropicReported by Anthropic | 69.2% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 10 | Ling 3.1 FlashInclusionAIReported by InclusionAI | 68.7% | Reported by InclusionAI | Not reported | Open | Not recorded | 262K | MIT |
| 11 | Qwen3.8 MaxAlibabaReported by Alibaba | 67.7% | Reported by Alibaba | Not reported | Open | Not recorded | 1M | Qwen3.8-Max License |
| 12 | Beam 1.0 501B-A23BReflection AIReported by Reflection AI | 66.1% | Reported by Reflection AI | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 13 | Beam 1.0 501B-A23B FP8Reflection AIReported by Reflection AI | 65.9% | Reported by Reflection AI | Not reported | Open | Not recorded | 1M | Not recorded |
| 14 | Hy4 previewTencentReported by Tencent | 65.7% | Reported by Tencent | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 15 | Beam 1.0 501B-A23B NVFP4Reflection AIReported by Reflection AI | 65.6% | Reported by Reflection AI | Not reported | Open | Not recorded | 1M | Not recorded |
| 16 | Ornith-1.5-397BOrnith AIReported by Ornith AI | 65.1% | Reported by Ornith AI | Not reported | Open | Not recorded | 262K | MIT |
| 17 | Grok 4.5xAIReported by xAI | 64.7% | Reported by xAI | Not reported | Closed | $2 / $6 | 500K | Proprietary |
| 18 | GPT-5.6 SolOpenAIReported by OpenAI | 64.6% | Reported by OpenAI | Not reported | Closed | $4 / $20 | 1.05M | Proprietary |
| 19 | Claude Opus 4.7 (Adaptive)AnthropicReported by Anthropic | 64.3% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 20 | GPT-5.6 TerraOpenAIReported by OpenAI | 63.4% | Reported by OpenAI | Not reported | Closed | $2 / $12 | 1.05M | Proprietary |
| 21 | Qwen3.8-Omni-FlashAlibabaReported by Alibaba | 63.3% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 22 | Claude Sonnet 5AnthropicReported by Anthropic | 63.2% | Reported by Anthropic | Not reported | Closed | $2 / $10 | 1M | Proprietary |
| 23 | GPT-5.6 LunaOpenAIReported by OpenAI | 62.7% | Reported by OpenAI | Not reported | Closed | $0.2 / $1.2 | 1.05M | Proprietary |
| 24 | Qwen3.8-Flash-NextAlibabaReported by Alibaba | 62.5% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Qwen Community 1.0 |
| 25 | Ornith-1.0-397BDeepReinforce AIReported by DeepReinforce AI | 62.2% | Reported by DeepReinforce AI | Not reported | Open | Not recorded | 256K | MIT |
| 26 | GLM-5.2Z.AIReported by Z.AI | 62.1% | Reported by Z.AI | Not reported | Open | $1.4 / $4.4 | 1M | MIT |
| 27 | Qwen3.8-27BAlibabaReported by Alibaba | 61.7% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 28 | Muse Spark 1.1MetaReported by Meta | 61.5% | Reported by Meta | Not reported | Closed | Not recorded | 1M | Proprietary |
| 29 | Ripple-65B PreviewRicursive IntelligenceReported by Ricursive Intelligence | 61.4% | Reported by Ricursive Intelligence | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 30 | Sage-A1.5InternScienceReported by InternScience | 61.2% | Reported by InternScience | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 31 | dots3-note PreviewDots StudioReported by Dots Studio | 61% | Reported by Dots Studio | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 32 | Qwen3.7 MaxAlibabaReported by Alibaba | 60.6% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 33 | Atria Dawn PreviewShanghai Artificial Intelligence LaboratoryReported by the lab | 59.6% | Reported by the lab | Not reported | Open | Not recorded | 256K | MIT |
| 34 | Ornith-1.5-35B-A3BOrnith AIReported by Ornith AI | 59.6% | Reported by Ornith AI | Not reported | Open | Not recorded | 262K | Not recorded |
| 35 | Laguna S 2.1PoolsideReported by Poolside | 59.4% | Reported by Poolside | Not reported | Open | $0.1 / $0.2 | 1M | OpenMDW-1.1 |
| 36 | MiniMax M3MiniMaxReported by MiniMax | 59% | Reported by MiniMax | Not reported | Open | $0.3 / $1.2 | 1M | MiniMax Community License |
| 37 | Sakana FuguSakana AIReported by Sakana AI | 59% | Reported by Sakana AI | Not reported | Closed | Not recorded | 1M | Proprietary |
| 38 | GPT-5.5OpenAIReported by OpenAI | 58.6% | Reported by OpenAI | Not reported | Closed | $5 / $30 | 1M | Proprietary |
| 39 | Kimi K2.6Moonshot AIReported by Moonshot AI | 58.6% | Reported by Moonshot AI | Not reported | Open | $0.95 / $4 | 256K | Modified MIT |
| 40 | GLM-5.1Z.AIReported by Z.AI | 58.4% | Reported by Z.AI | Not reported | Open | $1.4 / $4.4 | 203K | MIT |
| 41 | GPT-5.4OpenAIReported by OpenAI | 57.7% | Reported by OpenAI | Not reported | Closed | $2.5 / $15 | 1.05M | Proprietary |
| 42 | Qwen3.7 PlusAlibabaReported by Alibaba | 57.6% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 43 | Qwen 3.6 Max (preview)AlibabaReported by Alibaba | 57.3% | Reported by Alibaba | Not reported | Closed | Not recorded | 256K | Proprietary |
| 44 | MiMo-V2.5-ProXiaomiReported by Xiaomi | 57.2% | Reported by Xiaomi | Not reported | Closed | Not recorded | 1M | Proprietary |
| 45 | Claude Opus 4.5AnthropicAnother lab’s comparison table | 57.1% | Another lab’s comparison table | Not reported | Closed | $5 / $25 | 200K | Proprietary |
| 46 | GPT-5.3 CodexOpenAIReported by OpenAI | 56.8% | Reported by OpenAI | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 47 | Qwen3.6 PlusAlibabaReported by Alibaba | 56.6% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 48 | Ling 3.0 FlashInclusionAIReported by InclusionAI | 56.6% | Reported by InclusionAI | Not reported | Open | Not recorded | 262K | MIT |
| 49 | Step 3.7 FlashStepFunReported by StepFun | 56.3% | Reported by StepFun | Not reported | Open | $0.2 / $1.15 | 256K | Apache-2.0 |
| 50 | MiniMax M2.7MiniMaxReported by MiniMax | 56.2% | Reported by MiniMax | Not reported | Open | $0.3 / $1.2 | 200K | MiniMax License |
| 51 | MiMo-V2.5XiaomiReported by Xiaomi | 56.1% | Reported by Xiaomi | Not reported | Closed | Not recorded | 1M | Proprietary |
| 52 | Inkling-SmallThinking Machines LabReported by Thinking Machines Lab | 55.9% | Reported by Thinking Machines Lab | Not reported | Open | $0.58 / $1.44 | 1M | Not recorded |
| 53 | GPT-5.2OpenAIReported by OpenAI | 55.6% | Reported by OpenAI | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 54 | DeepSeek V4 Pro 0813DeepSeekReported by DeepSeek | 55.4% | Reported by DeepSeek | Not reported | Open | $1.32 / $3.96 | 1M | MIT |
| 55 | Gemini 3.5 FlashGoogleReported by Google | 55.1% | Reported by Google | Not reported | Closed | $1.5 / $9 | 1M | Proprietary |
| 56 | GLM-5Z.AIAnother lab’s comparison table | 55.1% | Another lab’s comparison table | Not reported | Open | $1 / $3.2 | 200K | MIT |
| 57 | DeepSeek V4 Pro (High)DeepSeekReported by DeepSeek | 54.4% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 58 | InklingThinking Machines LabReported by Thinking Machines Lab | 54.3% | Reported by Thinking Machines Lab | Not reported | Open | $1.87 / $4.68 | 1M | Apache-2.0 |
| 59 | Spark-X2.6-4BSparkLLMReported by SparkLLM | 54.3% | Reported by SparkLLM | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 60 | Gemini 3.5 Flash-LiteGoogleReported by Google | 54.2% | Reported by Google | Not reported | Closed | $0.3 / $2.5 | 1M | Proprietary |
| 61 | Qwen3.6-27BAlibabaReported by Alibaba | 53.5% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 62 | Claude Opus 4.6AnthropicReported by Meta AI (comparison table) | 53.4% | Reported by Meta AI (comparison table) | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 63 | MAI-Thinking-1MicrosoftReported by Microsoft | 52.8% | Reported by Microsoft | Not reported | Closed | Not recorded | 256K | Proprietary |
| 64 | DeepSeek V4 Flash 0731DeepSeekReported by DeepSeek | 52.6% | Reported by DeepSeek | Not reported | Open | $0.14 / $0.28 | 1M | MIT |
| 65 | Muse SparkMetaReported by Meta | 52.4% | Reported by Meta | Not reported | Closed | Not recorded | 262K | Proprietary |
| 66 | DeepSeek V4 Flash (High)DeepSeekReported by DeepSeek | 52.3% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 67 | DeepSeek V4 ProDeepSeekReported by DeepSeek | 52.1% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 68 | Grok 4.20xAIReported by Meta AI (comparison table) | 51.8% | Reported by Meta AI (comparison table) | Not reported | Closed | $1.25 / $2.5 | 2M | Proprietary |
| 69 | Nemotron 3.5 Super VL 120B A12BNVIDIAReported by NVIDIA | 51.8% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 70 | Nemotron 3.5 Super VL 120B A12B NVFP4NVIDIAReported by NVIDIA | 51.3% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | Not recorded |
| 71 | Muse Glimmer 30BMetaReported by Meta | 51.2% | Reported by Meta | Not reported | Open | Not recorded | 131K | Apache-2.0 |
| 72 | Qwen3.5 397BAlibabaReported by Alibaba | 50.9% | Reported by Alibaba | Not reported | Open | $0.6 / $3.6 | 128K | Apache-2.0 |
| 73 | Kimi K2.5Moonshot AIReported by Moonshot AI | 50.7% | Reported by Moonshot AI | Not reported | Open | $0.6 / $3 | 256K | Modified MIT |
| 74 | Ornith-1.0-35BDeepReinforce AIReported by DeepReinforce AI | 50.4% | Reported by DeepReinforce AI | Not reported | Open | Not recorded | 256K | Not recorded |
| 75 | Qwen3.6-35B-A3BAlibabaReported by Alibaba | 49.5% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 76 | Laguna M.1PoolsideReported by Poolside | 49.2% | Reported by Poolside | Not reported | Closed | Not recorded | 256K | Proprietary |
| 77 | DeepSeek V4 FlashDeepSeekReported by DeepSeek | 49.1% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 78 | Laguna XS 2.1PoolsideReported by Poolside | 47.6% | Reported by Poolside | Not reported | Open | $0.1 / $0.2 | 262K | Not recorded |
| 79 | Ornith-1.5-9BOrnith AIReported by Ornith AI | 47.5% | Reported by Ornith AI | Not reported | Open | Not recorded | 262K | Not recorded |
| 80 | Laguna XS.2PoolsideReported by Poolside | 46.3% | Reported by Poolside | Not reported | Open | Not recorded | 256K | Not recorded |
| 81 | Ornith-1.0-9BDeepReinforce AIReported by DeepReinforce AI | 42.9% | Reported by DeepReinforce AI | Not reported | Open | Not recorded | 256K | Not recorded |
| 82 | LongCat-Flash-Lite-SparseMeituanReported by Meituan | 40.6% | Reported by Meituan | Not reported | Open | Not recorded | 1M | MIT |
| 83 | Granite 4.2 30BIBMReported by IBM | 33.3% | Reported by IBM | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 84 | LLaDA2.2-flashInclusionAIReported by InclusionAI | 30.1% | Reported by InclusionAI | Not reported | Open | Not recorded | 128K | Not recorded |
| 85 | Mellum2.1-12B-A2.5B-ThinkingJetBrainsReported by JetBrains | 28% | Reported by JetBrains | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 86 | Granite 4.2 8BIBMReported by IBM | 19.1% | Reported by IBM | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 87 | MiniCPM5-2BOpenBMBReported by OpenBMB | 14.4% | Reported by OpenBMB | Not reported | Open | Not recorded | 131K | Apache-2.0 |
- Lab-reported
- Benchmark-run
- Independent
SWE-Bench Pro (Public Dataset)
BenchLM mirrors this Scale Labs leaderboard for display-only reference. It is excluded from overall and category rankings.
| Source rank | Source model or configuration | Resolve rate (%) | Source setup | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|
| 1 | Muse Spark 1.1*Meta | 61.5% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 2 | gpt-5.4 (xHigh)*OpenAI | 59.1% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 3 | Muse Spark*Meta | 55% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 4 | claude-opus-4-6 (thinking)*Anthropic | 51.9% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 5 | gemini-3.1-pro (thinking)*Google | 46.1% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 6 | Claude Opus 4.5Anthropic | 45.9% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Closed | $5 / $25 | 200K | Proprietary |
| 7 | claude-4-5-SonnetAnthropic | 43.6% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 8 | gemini-3-pro-previewGoogle | 43.3% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 9 | Claude 4 SonnetAnthropic | 42.7% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 10 | gpt-5-2025-08-07 (High)OpenAI | 41.8% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 11 | GPT-5.2-CodexOpenAI | 41.0% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 12 | claude-4-5-haikuAnthropic | 39.5% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 13 | Qwen3-Coder 480B-A35BAlibaba | 38.7% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Open | Not recorded | 256K | Not recorded |
| 14 | minimax-2.1MiniMax | 36.8% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 15 | Gemini 3 FlashGoogle | 34.6% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Closed | $0.5 / $3 | 1M | Proprietary |
| 16 | GPT-5.2OpenAI | 29.9% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 17 | kimi-k2-instructMoonshot AI | 27.7% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 18 | Qwen3 235B-A22BAlibaba | 21.4% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 19 | GPT-OSS 120BOpenAI | 16.2% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 20 | deepseek-v3p2Deepseek | 15.6% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 21 | gemma-3-27b-itGoogle | 11.4% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 22 | llama3-1-405b-instructMeta | 11.2% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 23 | GLM-4.6Z.AI | 9.7% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Open | Not recorded | 200K | Not recorded |
| 24 | llama4-maverick-17b-instructMeta | 5.2% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 25 | codestral-2405Mistral | 1.5% | SWE-Bench Pro (Public Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
SWE-Bench Pro (Private Dataset)
BenchLM mirrors this Scale Labs leaderboard for display-only reference. It is excluded from overall and category rankings.
| Source rank | Source model or configuration | Resolve rate (%) | Source setup | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|
| 1 | Muse Spark 1.1*Meta | 51.5% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 2 | claude-opus-4-6 (thinking)*Anthropic | 47.1% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 3 | Muse Spark*Meta | 44.7% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 4 | gpt-5.4(xHigh)*OpenAI | 43.4% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 5 | gemini-3.1-pro (thinking)*Google | 32.2% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 6 | GPT-5.2-CodexOpenAI | 27.7% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 7 | GPT-5.2OpenAI | 23.8% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 8 | Opus 4.5Anthropic | 23.4% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 9 | Gemini 3 ProGoogle | 17.9% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Closed | $2 / $12 | 2M | Proprietary |
| 10 | Claude Opus 4.1Anthropic | 17.8% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 11 | OpenAI GPT-5OpenAI | 14.9% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 12 | Gemini 2.5 Pro PreviewGoogle | 10.1% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 13 | Claude Sonnet 4Anthropic | 9.1% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
| 14 | OpenAI GPT-4oOpenAI | 3.6% | SWE-Bench Pro (Private Dataset)Original result | Not reported | Unknown | Not recorded | Not recorded | Not recorded |
How to read this leaderboard
Prior editorial review by Glevd · 2026-07-15
Variant check: Catalog input/output prices and context limits describe current model access. They do not record the benchmark run’s token use, retries, or actual cost. A model’s weight-access label is separate from its license terms.
Use SWE-bench Pro as evidence for long-horizon repository work only after matching the split, scaffold, tool budget, retry policy, and run count. The table preserves exact published rows from different providers, so a small score gap is directional unless the setups match.
Operator receipt: 87 sourced rows are currently displayable on this page; the leading published row is Claude Fable 5.2 at 93%.
Honest limit: OpenAI's July 2026 audit estimated that about 30% of the 731-task public split is broken and retracted its earlier recommendation to adopt the benchmark. The current rows remain useful published receipts, but this page should not decide a coding-agent purchase by itself.
According to BenchLM.ai, Claude Fable 5.2 leads the SWE-bench Pro benchmark with a score of 93%, followed by Claude Opus 5.5 (89.9%) and Claude Sonnet 5.5 (81.3%). There is significant spread across the leaderboard, making this benchmark effective at differentiating model capabilities.
87 models have been evaluated on SWE-bench Pro. The benchmark falls in the Coding category. BenchAlign v5.8 gives SWE-bench Pro 26% of the Coding reference weight, so it moves the Coding leaderboard and the overall ranking. Reference weights are relative weights in the calibrated model, not fixed shares of a score.
About SWE-bench Pro
Year
2025
Tasks
1,865 repository problems
Format
Repository task completion
Difficulty
Long-horizon professional engineering
The authors assembled 1,865 problems from 41 repositories across public, held-out, and commercial splits. Agents receive a repository and issue, then produce a patch that must pass the evaluation tests without breaking existing behavior.
Teams weighing long-horizon repository work usually read this table next to DeepSWE scores, a second long-horizon benchmark built on original tasks.
Freshness and provenance
Version
SWE-bench Pro 2025
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SWE-bench Pro measure?
SWE-bench Pro gives an agent a repository and issue description, then checks whether its patch passes new tests without breaking existing behavior. The full benchmark contains 1,865 problems from 41 repositories across public, held-out, and commercial splits, with work that can span multiple files and long execution horizons.
Are SWE-bench Pro scores directly comparable?
Only when the split and evaluation setup match. Public, held-out, and commercial tasks are different pools, while scaffold, tool budget, retry policy, and token budget can change pass rates. This page preserves exact published rows, but it does not pretend every provider ran the same harness.
Should SWE-bench Pro decide which coding agent to use?
No. The benchmark covers realistic repository work, but OpenAI's July 2026 audit estimated that about 30% of the public tasks are broken and retracted its earlier adoption recommendation. Use SWE-bench Pro alongside LiveCodeBench, other repository evaluations, and a workload-specific trial instead of treating one score as a procurement decision.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.