Four-Factor Integrity Breakdown & Primary-Source Evidence
Access Model: Public Plaintext
"The reference patch in `solution/` is never used at grading time; it exists so reviewers can spot-check correctness offline."
"The benchmark includes 113 tasks across TypeScript, Go, Python, JavaScript, and Rust, with isolated environments and program-based verifiers."
Originality
Weight 25%
7.2/10
Reuse Profile: Low Reuse (<25%)
"Every DeepSWE task is original: the reference solution is written from scratch rather than copied or adapted from an existing pull request, commit, or public patch."
Freshness
Weight 20%
7/10
Initial Release: 2026-05Latest Refresh: 2026-06Rolling Updates: No
"DeepSWE v1.1 keeps the same long-horizon engineering tasks as v1, but updates how agents are executed and scored by grading their committed code in a clean, isolated environment, making results easier to reproduce, audit, and analyze."
Grading
Weight 15%
7.3/10
Integrity: Independent StandardizedSandbox: live internet
"The agent commits its proposed changes, and we extract the git patch to evaluate in an isolated container, separate from where the agent worked."
Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:
[](https://benchaudit.org/benchmarks/deepswe/)