Four-Factor Integrity Breakdown & Primary-Source Evidence
Privacy
Weight 40%
5.5/10
Access Model: Public Prompts / Private Answers
"Each level is divided into a fully public dev set for validation, and a test set with private answers and metadata."
"Using GAIA's methodology, we devise 466 questions and their answer. We release our questions while retaining answers to 300 of them to power a leader-board"
Originality
Weight 25%
10/10
Reuse Profile: All Original (0% reuse)
"We introduce GAIA, a benchmark for General AI Assistants that, if solved, would represent a milestone in AI research. GAIA proposes real-world questions that require a set of fundamental abilities such as reasoning, multi-modality handling, web browsing, and generally tool-use proficiency."
Freshness
Weight 20%
2/10
Initial Release: 2023-11Latest Refresh: 2023-11Rolling Updates: No
"arXiv:2311.12983 (cs) [Submitted on 21 Nov 2023]"
Grading
Weight 15%
3.3/10
Integrity: Self-Reported / Live NetworkSandbox: live internet
"GAIA is a benchmark which aims at evaluating next-generation LLMs (LLMs with augmented capabilities due to added tooling, efficient prompting, access to search, etc)."
Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:
[](https://benchaudit.org/benchmarks/gaia/)