# SWE-bench Pro V2 Contamination Audit

> SWE-bench Pro V2 scores 3.9/10 (Tier C, High Risk) on BenchAudit as of November 2026, reflecting publicly downloadable plaintext test files and isolated offline third-party grading.

- **Canonical URL:** https://benchaudit.org/benchmarks/swe-bench-pro-v2/
- **Benchmark ID:** `swe-bench-pro-v2`
- **Hub:** Scale AI / Reflection
- **Category:** Coding
- **Domains:** Coding
- **Skills:** Coding, Agentic, Tool Use, Reasoning, Long Context
- **BenchAudit Score:** 3.9/10 (Tier C - High Risk)
- **Last Audited:** 2026-11-20 (Ref 2026-11)

## Factor Scores

| Factor | Score (/10) | Weight |
| --- | ---: | ---: |
| Privacy | 0 | 40% |
| Originality | 4 | 25% |
| Freshness | 7 | 20% |
| Grading | 10 | 15% |

## Primary-Source Evidence

### Privacy
> "We're releasing SWE-Bench Pro V2, a refreshed public split with a modified benchmark and a locked evaluation protocol, co-developed with Reflection."
- Source: https://labs.scale.com/leaderboard/swe_bench_pro_public_v2

### Originality
> "Instructions: 529 problem statements rewritten so that every graded assertion traces back to a sentence in the text (only problem statement, requirements and interfaces changed); each correction was implemented blind by a second engineer and graded by the task's own hidden tests."
- Source: https://raw.githubusercontent.com/scaleapi/SWE-bench_Pro-os/main/v2/README.md

### Freshness
> "V2 (2026-09-22) is the default config: 642 tasks (go 256, python 237, js 145, ts 4)."
- Source: https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro

### Grading
> "Every agent diff is re-graded on a pristine image, and we publish both grades."
- Source: https://labs.scale.com/leaderboard/swe_bench_pro_public_v2

## Top Frontier Models on SWE-bench Pro V2

| Rank | Model | Creator | Percentile |
| ---: | --- | --- | ---: |
| #1 | [Gemini 4 Argon](https://benchaudit.org/models/gemini-4-argon__high/) | Google | 92.5p |
| #2 | [Claude Fable 5.1](https://benchaudit.org/models/claude-fable-5-1__max/) | Anthropic | 90.9p |
| #2 | [Inkling 2 Small](https://benchaudit.org/models/inkling-2-small/) | Thinking Machines | 82.9p |
| #3 | [Muse Wonder](https://benchaudit.org/models/muse-wonder__max/) | Meta | 73.3p |
| #4 | [GPT-6 Astra](https://benchaudit.org/models/gpt-6-astra__max/) | OpenAI | 72.7p |
| #4 | [MiniMax-M3.1-Flash](https://benchaudit.org/models/minimax-m3-1-flash__max/) | MiniMax | 63.6p |
| #5 | [GPT-6.1 Sol](https://benchaudit.org/models/gpt-6-1-sol__max/) | OpenAI | 54.0p |
| #6 | [Claude Sonnet 5.5](https://benchaudit.org/models/claude-sonnet-5-5__max/) | Anthropic | 44.4p |
| #7 | [Muse Spark Open](https://benchaudit.org/models/muse-spark-open__xhigh/) | Meta | 34.8p |
| #8 | [Claude Haiku 5.5](https://benchaudit.org/models/claude-haiku-5-5__max/) | Anthropic | 25.1p |
