Domains: BusinessCodingFinanceScience
Skills: AgenticTool UseMultimodalCodingKnowledge WorkReasoning

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
2/10
Access Model: Auto-Gated / Encrypted
"Visit https://huggingface.co/datasets/xlangai/osworld_v2_tasks while logged in to Hugging Face and accept the gated access request, your request will automatically be approved."
Primary source (GitHub) ↗ Verified 2026-11-19
"The official OSWorld 2.1 Python task classes are distributed through the gated Hugging Face dataset xlangai/osworld_v2_tasks, not directly in this public GitHub checkout."

Originality

Weight 25%
10/10
Reuse Profile: All Original (0% reuse)
"The bulk of OSWorld 2.0 tasks were produced by a small group of internally trained annotators who worked end-to-end on the tasks they proposed, covering task design, input artifact construction, environment setup, and evaluation function implementation."
Primary source (Paper) ↗ Verified 2026-11-19

Freshness

Weight 20%
7/10
Initial Release: 2026-06Latest Refresh: 2026-09Rolling Updates: No
"2026-09-16: We released osworld-v2.1, a new bug-fix release with updated agents, evaluation workflows, and pinned task, asset, website, and provider image references."
Primary source (GitHub) ↗ Verified 2026-11-19

Grading

Weight 15%
3.3/10
Integrity: Self-Reported / Live NetworkSandbox: offline isolated
"Offline set: 82 tasks runnable without internet"

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit OSWorld-2.1 (Offline subset) Tier B 5.2/10
[![BenchAudit Tier B](https://benchaudit.org/badges/osworld-2-1-offline.svg)](https://benchaudit.org/benchmarks/osworld-2-1-offline/)