Domains: Coding
Skills: CodingAgenticTool UseReasoningLong Context

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
0/10
Access Model: Public Plaintext
"Codebase QnA consists of 124 tasks that targets the upstream capability of Software Engineering - deep code comprehension that precedes any code change."
"Artificial Analysis's SWE-Atlas-QnA-AA dataset provided wording for several updated Codebase QnA rubric criteria, and Inductive ML's reward-hacking feedback further hardened the task environments."

Originality

Weight 25%
7.2/10
Reuse Profile: Low Reuse (<25%)
"Professional software engineers and technical experts with significant coding and agentic experience write problem statements that require multi-step reasoning across the codebase."

Freshness

Weight 20%
7/10
Initial Release: 2026-05Latest Refresh: 2026-11Rolling Updates: No
"Update November 2026. SWE Atlas v1.1 is live. It combines the three category leaderboards into a single, easy-to-run index covering the same 284 tasks as v1.0."
Primary source (GitHub) ↗ Verified 2026-11-20

Grading

Weight 15%
10/10
Integrity: Independent Offline (Leak-Verified)Sandbox: offline isolated
"Agents are instructed to avoid modifying source code files and to clean up any temporary scripts created. A programmatic check automatically fails any task that contains code changes."

Frontier Model Standings on SWE Atlas - Codebase QnA

Top 14 active models tracked
Rank Model Official Score Percentile
#1Claude Fable 5.2 max Anthropic83.9%100.0p
#2Claude Fable 5.1 max Anthropic79.0%100.0p
#3Claude Opus 5.5 max Anthropic77.4%97.5p
#4Claude Sonnet 5.5 max Anthropic74.2%88.1p
#4Inkling 2 Small default Thinking Machines74.2%88.1p
#6GPT-6.1 Sol max OpenAI71.0%78.7p
#7GLM-5.3 max Z AI70.2%69.4p
#7GPT-6 Astra max OpenAI70.2%69.4p
#9GPT-6.2 Astra max OpenAI67.7%60.0p
#11Claude Haiku 5.5 max Anthropic62.1%47.5p
#12GPT-6.2 Sol max OpenAI61.3%41.2p
#13GPT-6.1 Terra max OpenAI54.0%35.0p
#14GPT-6 Luna max OpenAI50.8%28.6p
#15Nemotron 3.5 Super VL high NVIDIA26.6%22.4p

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit SWE Atlas - Codebase QnA Tier C 4.7/10
[![BenchAudit Tier C](https://benchaudit.org/badges/swe-atlas-qna.svg)](https://benchaudit.org/benchmarks/swe-atlas-qna/)