Domains: Coding
Skills: CodingAgenticTool UseReasoningLong Context

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
0/10
Access Model: Public Plaintext
"A human-validated subset of 500 SWE-bench instances for reliable evaluation of coding agents and language models."

Originality

Weight 25%
4/10
Reuse Profile: Medium Reuse (25-75%)
"To this end, we introduce SWE-bench, an evaluation framework consisting of $2,294$ software engineering problems drawn from real GitHub issues and corresponding pull requests across $12$ popular Python repositories."
Primary source (Paper) ↗ Verified 2026-11-17

Freshness

Weight 20%
0/10
Initial Release: 2024-08Latest Refresh: 2024-08Rolling Updates: No
"**[Aug. 13, 2024]**: Introducing *SWE-bench Verified*! Part 2 of our collaboration with [OpenAI Preparedness](https://openai.com/preparedness/). A subset of 500 problems that real software engineers have confirmed are solvable."
Primary source (GitHub) ↗ Verified 2026-11-17

Grading

Weight 15%
3.3/10
Integrity: Self-Reported / Live NetworkSandbox: offline isolated
"Check out the main SWE-bench repository docs for instructions on how to generate and evaluate predictions on SWE-bench [Lite, Verified, Multimodal]."

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit SWE-bench Verified Tier D 1.5/10
[![BenchAudit Tier D](https://benchaudit.org/badges/swe-bench-verified.svg)](https://benchaudit.org/benchmarks/swe-bench-verified/)