Four-Factor Integrity Breakdown & Primary-Source Evidence
Access Model: Public Plaintext
"We're releasing SWE-Bench Pro V2, a refreshed public split with a modified benchmark and a locked evaluation protocol, co-developed with Reflection."
"The same data is on HuggingFace as ScaleAI/SWE-bench_Pro (config default = V2, hard = HARD-51, v1 = the original 731 tasks)."
Originality
Weight 25%
4/10
Reuse Profile: Medium Reuse (25-75%)
"Instructions: 529 problem statements rewritten so that every graded assertion traces back to a sentence in the text (only problem statement, requirements and interfaces changed); each correction was implemented blind by a second engineer and graded by the task's own hidden tests."
Freshness
Weight 20%
7/10
Initial Release: 2026-09Latest Refresh: 2026-09Rolling Updates: No
"V2 (2026-09-22) is the default config: 642 tasks (go 256, python 237, js 145, ts 4)."
Integrity: Independent Offline (Leak-Verified)Sandbox: offline isolated
"Every agent diff is re-graded on a pristine image, and we publish both grades."
Frontier Model Standings on SWE-bench Pro V2
Top 12 active models tracked
Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:
[](https://benchaudit.org/benchmarks/swe-bench-pro-v2/)