Category
Integrity Tier
Domains
Skills (multi-select)

Showing 0 benchmarks

# Benchmark & Hub Category & Domains Evaluates whether test prompts and answers are kept strictly private, or exposed via public downloads, GitHub repos, or auto-approved click-through gates. Evaluates how much of the benchmark was newly authored from scratch versus reused from older public exams, competition archives, or web corpora. Evaluates how recently tasks were refreshed relative to exposure risk, including rolling post-cutoff updates and leaked-task pruning. Evaluates whether models run in an isolated offline sandbox scored by an independent evaluator, free from live-internet or container/git leaks. Tier Details

Insights

Cross-study findings connecting benchmark integrity scores with live official model leaderboards, and more

Syncing standings…

1. Fortified Leaders vs. Exposed Over-Performers

Compares current-generation models across actively updated Tier S held-out evaluations and active Tier C & D benchmarks. Frozen legacy leaderboards missing recent frontier models are excluded to prevent cohort bias.

Min Evals

2. Temporal Half-Life: Age vs. Integrity Score

Each shape is one audited benchmark: left to right shows how many months old it is since its last major update, and bottom to top shows its integrity score (0–100). Circles are private held-out tests that stay trustworthy with age; diamonds are public or easy-to-access datasets that tend to leak and lose value over time. Hover or click any shape to inspect it.

Private / Hybrid Held-Out Public / Auto-Gated / Prompts S A B C D

3. Domain & Category Contamination Landscape

Distribution of Tiers S–D and average scores across fields. Click any bar to filter the table above.

4. Access Barrier Effectiveness

Groups benchmarks by how strictly their questions and answers are protected—from locked private servers at the top to open public downloads at the bottom. Each bar shows that group's average integrity score (out of 100), its lowest-to-highest range, and clickable benchmark examples.