Domains: ScienceCoding
Skills: ReasoningCodingTool UseAgentic

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
5.5/10
Access Model: Public Prompts / Private Answers
"To ensure data integrity and prevent leakage, our grading server only accepts complete batches containing responses to all 70 problems. Any submission with missing or unmatched problem IDs will be automatically rejected."
Primary source (GitHub) ↗ Verified 2026-11-17
"This dataset is intended only as a test set to evaluate the genuine reasoning ability of LLMs on research-level physics problems."

Originality

Weight 25%
10/10
Reuse Profile: All Original (0% reuse)
"CritPt consists of 71 composite research challenges designed to simulate full-scale research projects at the entry level, which are also decomposed to 190 simpler checkpoint tasks for more fine-grained insights. All problems are newly created by 50+ active physics researchers based on their own research."
Primary source (Paper) ↗ Verified 2026-11-17

Freshness

Weight 20%
9/10
Initial Release: 2025-09Latest Refresh: 2026-10Rolling Updates: No
"Version 5.0 October 2026 to November 2026 Added HLE-Diamond (10%) to the General category Added Terminal-Bench-Science (15%) to the Science category Added FrontierCode-AA 2.0 (15%) to the Coding category Reduced the GDP.pdf weight from 10% to 5% Replaced CritPt with CritPt v2 (Hard Subset), keeping its 10% weight"

Grading

Weight 15%
10/10
Integrity: Independent Offline (Leak-Verified)Sandbox: offline isolated
"CritPt v2 is the revised version of CritPt, a benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 70 composite research challenges. Models solve problems with a code-execution tool in an offline Linux sandbox."

Frontier Model Standings on CritPt v2

Top 15 active models tracked
Rank Model Official Score Percentile
#1Claude Fable 5.2 max Anthropic35.4%91.1p
#2Muse Wonder Full max Meta32.9%88.7p
#3Claude Opus 5.5 max Anthropic31.7%84.0p
#3GPT-6 Astra max OpenAI31.7%84.0p
#3GPT-6.1 Sol max OpenAI31.7%84.0p
#6Claude Sonnet 5.5 max Anthropic31.4%79.3p
#7GPT-6 Sol max OpenAI30.9%76.9p
#8Claude Fable 5.1 max Anthropic29.7%74.6p
#9Muse Wonder max Meta28.3%72.2p
#10GPT-6.1 Terra max OpenAI28.0%69.8p
#11Gemini 4 Argon high Google27.1%67.5p
#12MiMo-V2.6-Pro default Xiaomi26.6%65.1p
#13Ling 3.1 Flash 1019 default InclusionAI26.0%62.7p
#14Muse Spark 1.3 max Meta24.9%60.3p
#15Kimi K3.1 Pro max Kimi24.6%57.9p

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit CritPt v2 Tier A 8/10
[![BenchAudit Tier A](https://benchaudit.org/badges/critpt-v2.svg)](https://benchaudit.org/benchmarks/critpt-v2/)