# Autoform-Hard Contamination Audit

> Autoform-Hard scores 4.7/10 (Tier C, High Risk) on BenchAudit as of November 2026, reflecting publicly downloadable plaintext test files and isolated offline third-party grading.

- **Canonical URL:** https://benchaudit.org/benchmarks/autoform-hard/
- **Benchmark ID:** `autoform-hard`
- **Hub:** Vals AI
- **Category:** Science
- **Domains:** Math
- **Skills:** Agentic, Reasoning, Coding, Tool Use
- **BenchAudit Score:** 4.7/10 (Tier C - High Risk)
- **Last Audited:** 2026-11-22 (Ref 2026-11)

## Factor Scores

| Factor | Score (/10) | Weight |
| --- | ---: | ---: |
| Privacy | 0 | 40% |
| Originality | 7.2 | 25% |
| Freshness | 7 | 20% |
| Grading | 10 | 15% |

## Primary-Source Evidence

### Privacy
> "This repository contains all 341 audited AutoformBench tasks."
- Source: https://github.com/facebookresearch/autoformbench

### Originality
> "The statements were written and audited by PhD-level experts and chosen to go beyond what Mathlib already covers. Solving a task usually means building theory, not writing one proof: accepted solutions in the original study add a median of about 300 lines of Lean, and the hardest run to thousands."
- Source: https://www.vals.ai/benchmarks/autoform_hard

### Freshness
> "The board uses 56 tasks: every task in the benchmark’s 100-task validation set that none of the ten frontier models in the original study solved, except three that duplicate another task. Our audit corrected five of them: three statements were false as written, and two had packaging errors in the starting files."
- Source: https://www.vals.ai/benchmarks/autoform_hard

### Grading
> "A submission is accepted only if it passes clean-room grading: the graded theorems are rebuilt from a trusted copy of the task, their statements must match the originals exactly, the benchmark’s definitions must be unchanged"
- Source: https://www.vals.ai/benchmarks/autoform_hard

## Top Frontier Models on Autoform-Hard

| Rank | Model | Creator | Percentile |
| ---: | --- | --- | ---: |
| #1 | [GPT-6.2 Astra](https://benchaudit.org/models/gpt-6-2-astra__max/) | OpenAI | 100.0p |
| #2 | [Claude Fable 5.2](https://benchaudit.org/models/claude-fable-5-2__max/) | Anthropic | 94.9p |
| #2 | [GPT-6.2 Sol](https://benchaudit.org/models/gpt-6-2-sol__max/) | OpenAI | 94.9p |
| #4 | [Gemini 4 Carbon](https://benchaudit.org/models/gemini-4-carbon__high/) | Google | 74.5p |
| #5 | [Gemini 4 Hydrogen](https://benchaudit.org/models/gemini-4-hydrogen__xhigh/) | Google | 61.0p |
| #6 | [Gemini 4 Argon](https://benchaudit.org/models/gemini-4-argon__high/) | Google | 47.4p |
| #7 | [Beam 1.0](https://benchaudit.org/models/reflection-beam/) | Reflection | 27.0p |
| #7 | [MiniMax-M3.1-Flash](https://benchaudit.org/models/minimax-m3-1-flash__max/) | MiniMax | 27.0p |
