Four-Factor Integrity Breakdown & Primary-Source Evidence
Access Model: Public Plaintext
"This repository contains all 341 audited AutoformBench tasks."
"This release contains no textbook excerpts, textbook PDFs, informal source files, model traces, accepted submissions, gold proofs, or internal audit records."
Originality
Weight 25%
7.2/10
Reuse Profile: Low Reuse (<25%)
"The statements were written and audited by PhD-level experts and chosen to go beyond what Mathlib already covers. Solving a task usually means building theory, not writing one proof: accepted solutions in the original study add a median of about 300 lines of Lean, and the hardest run to thousands."
Freshness
Weight 20%
7/10
Initial Release: 2026-09Latest Refresh: 2026-11Rolling Updates: No
"The board uses 56 tasks: every task in the benchmark’s 100-task validation set that none of the ten frontier models in the original study solved, except three that duplicate another task. Our audit corrected five of them: three statements were false as written, and two had packaging errors in the starting files."
Integrity: Independent Offline (Leak-Verified)Sandbox: offline isolated
"A submission is accepted only if it passes clean-room grading: the graded theorems are rebuilt from a trusted copy of the task, their statements must match the originals exactly, the benchmark’s definitions must be unchanged"
Frontier Model Standings on Autoform-Hard
Top 8 active models tracked
Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:
[](https://benchaudit.org/benchmarks/autoform-hard/)