Domains: Coding
Skills: CodingAgenticTool UseReasoning

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
0/10
Access Model: Public Plaintext
"The reference patch in `solution/` is never used at grading time; it exists so reviewers can spot-check correctness offline."
Primary source (GitHub) ↗ Verified 2026-11-17
"The benchmark includes 113 tasks across TypeScript, Go, Python, JavaScript, and Rust, with isolated environments and program-based verifiers."

Originality

Weight 25%
7.2/10
Reuse Profile: Low Reuse (<25%)
"Every DeepSWE task is original: the reference solution is written from scratch rather than copied or adapted from an existing pull request, commit, or public patch."

Freshness

Weight 20%
7/10
Initial Release: 2026-05Latest Refresh: 2026-06Rolling Updates: No
"DeepSWE v1.1 keeps the same long-horizon engineering tasks as v1, but updates how agents are executed and scored by grading their committed code in a clean, isolated environment, making results easier to reproduce, audit, and analyze."

Grading

Weight 15%
7.3/10
Integrity: Independent StandardizedSandbox: live internet
"The agent commits its proposed changes, and we extract the git patch to evaluate in an isolated container, separate from where the agent worked."

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit DeepSWE v1.1 Tier C 4.3/10
[![BenchAudit Tier C](https://benchaudit.org/badges/deepswe.svg)](https://benchaudit.org/benchmarks/deepswe/)