Domains: Math
Skills: AgenticReasoningCodingTool Use

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
0/10
Access Model: Public Plaintext
"This repository contains all 341 audited AutoformBench tasks."
Primary source (GitHub) ↗ Verified 2026-11-22
"This release contains no textbook excerpts, textbook PDFs, informal source files, model traces, accepted submissions, gold proofs, or internal audit records."

Originality

Weight 25%
7.2/10
Reuse Profile: Low Reuse (<25%)
"The statements were written and audited by PhD-level experts and chosen to go beyond what Mathlib already covers. Solving a task usually means building theory, not writing one proof: accepted solutions in the original study add a median of about 300 lines of Lean, and the hardest run to thousands."

Freshness

Weight 20%
7/10
Initial Release: 2026-09Latest Refresh: 2026-11Rolling Updates: No
"The board uses 56 tasks: every task in the benchmark’s 100-task validation set that none of the ten frontier models in the original study solved, except three that duplicate another task. Our audit corrected five of them: three statements were false as written, and two had packaging errors in the starting files."

Grading

Weight 15%
10/10
Integrity: Independent Offline (Leak-Verified)Sandbox: offline isolated
"A submission is accepted only if it passes clean-room grading: the graded theorems are rebuilt from a trusted copy of the task, their statements must match the originals exactly, the benchmark’s definitions must be unchanged"

Frontier Model Standings on Autoform-Hard

Top 8 active models tracked
Rank Model Official Score Percentile
#1GPT-6.2 Astra max OpenAI87.5%100.0p
#2Claude Fable 5.2 max Anthropic85.7%94.9p
#2GPT-6.2 Sol max OpenAI85.7%94.9p
#4Gemini 4 Carbon high Google71.4%74.5p
#5Gemini 4 Hydrogen xhigh Google64.3%61.0p
#6Gemini 4 Argon high Google48.2%47.4p
#7Beam 1.0 default Reflection0.0%27.0p
#7MiniMax-M3.1-Flash max MiniMax0.0%27.0p

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit Autoform-Hard Tier C 4.7/10
[![BenchAudit Tier C](https://benchaudit.org/badges/autoform-hard.svg)](https://benchaudit.org/benchmarks/autoform-hard/)