Four-Factor Integrity Breakdown & Primary-Source Evidence
Access Model: Auto-Gated / Encrypted
"Visit https://huggingface.co/datasets/xlangai/osworld_v2_tasks while logged in to Hugging Face and accept the gated access request, your request will automatically be approved."
"The official OSWorld 2.1 Python task classes are distributed through the gated Hugging Face dataset xlangai/osworld_v2_tasks, not directly in this public GitHub checkout."
Originality
Weight 25%
10/10
Reuse Profile: All Original (0% reuse)
"The bulk of OSWorld 2.0 tasks were produced by a small group of internally trained annotators who worked end-to-end on the tasks they proposed, covering task design, input artifact construction, environment setup, and evaluation function implementation."
Freshness
Weight 20%
7/10
Initial Release: 2026-06Latest Refresh: 2026-09Rolling Updates: No
"2026-09-16: We released osworld-v2.1, a new bug-fix release with updated agents, evaluation workflows, and pinned task, asset, website, and provider image references."
Grading
Weight 15%
3.3/10
Integrity: Self-Reported / Live NetworkSandbox: offline isolated
"Offline set: 82 tasks runnable without internet"
Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:
[](https://benchaudit.org/benchmarks/osworld-2-1-offline/)