Domains: ScienceHumanitiesMathMedicalBusiness
Skills: MultimodalReasoningKnowledge Work

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
0/10
Access Model: Public Plaintext
"An enhanced MMMU benchmark that eliminates shortcuts and guessing strategies to more rigorously test multimodal models across 30 academic disciplines."
"MMMU-Pro introduces a vision-only input setting and increases the number of candidate options from 4 to 10, making it significantly harder for models to rely on guessing or exploiting shortcuts."

Originality

Weight 25%
4/10
Reuse Profile: Medium Reuse (25-75%)
"MMMU-Pro rigorously assesses multimodal models' true understanding and reasoning capabilities through a three-step process based on MMMU: (1) filtering out questions answerable by text-only models, (2) augmenting candidate options, and (3) introducing a vision-only input setting where questions are embedded within images."
Primary source (Paper) ↗ Verified 2026-11-17

Freshness

Weight 20%
0/10
Initial Release: 2024-09Latest Refresh: 2024-09Rolling Updates: No
"arXiv:2409.02813 (cs) [Submitted on 4 Sep 2024"
Primary source (Paper) ↗ Verified 2026-11-17

Grading

Weight 15%
7.3/10
Integrity: Independent StandardizedSandbox: not applicable single turn
"Multiple Choice (10 options) Regex extraction, pass@1"

Frontier Model Standings on MMMU-Pro

Top 15 active models tracked
Rank Model Official Score Percentile
#1GPT-6.2 Astra max OpenAI89.5%91.4p
#2Claude Opus 5.5 max Anthropic87.7%87.1p
#3GPT-6 Astra max OpenAI86.9%82.8p
#4GPT-6.1 Sol max OpenAI86.0%78.5p
#5Gemini 3.8 Flash high Google85.6%74.2p
#6Gemini 3.7 Flash high Google85.5%69.9p
#7GPT-6.1 Terra max OpenAI84.7%65.5p
#8Kimi K3.1 Pro max Kimi83.9%61.3p
#9GPT-6 Sol max OpenAI83.0%57.0p
#10Qwen3.8 Max (0902) xhigh Alibaba82.8%52.6p
#11Inkling 2 Small default Thinking Machines81.6%48.4p
#12Muse Spark Open xhigh Meta80.9%44.0p
#13DeepSeek V4.1 Flash 1017 max DeepSeek80.6%39.8p
#14GPT-6 Luna max OpenAI79.7%35.5p
#15Apodex 1.1 default Apodex79.2%31.1p

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit MMMU-Pro Tier D 2.1/10
[![BenchAudit Tier D](https://benchaudit.org/badges/mmmu-pro.svg)](https://benchaudit.org/benchmarks/mmmu-pro/)