Domains: Business
Skills: Knowledge WorkAgenticTool UseReasoningInstruction Following

Four-Factor Integrity Breakdown & Primary-Source Evidence

Privacy

Weight 40%
10/10
Access Model: Strictly Private
"We evaluate a private 657-task held-out split from AutomationBench dataset version 1.0.6"

Originality

Weight 25%
10/10
Reuse Profile: All Original (0% reuse)
"It tests whether models can complete realistic SaaS workflows that span multiple simulated business apps, using REST APIs as the tool interface."

Freshness

Weight 20%
10/10
Initial Release: 2026-04Latest Refresh: 2026-08Rolling Updates: No
"Version 4.3 September 2026 Replaced 饾湉鲁-Banking with AutomationBench-AA (5%) in the Agents category"

Grading

Weight 15%
10/10
Integrity: Independent Offline (Leak-Verified)Sandbox: offline isolated
"Objectives and guardrails are graded using programmatic checks on the final environment state. AutomationBench-AA does not use a separate LLM judge for grading"

Frontier Model Standings on AutomationBench-AA

Top 15 active models tracked
Rank Model Official Score Percentile
#1Gemini 4 Neon xhigh Google79.3%100.0p
#2Gemini 4 Carbon high Google77.8%98.0p
#3Gemini 4 Argon high Google77.5%95.9p
#4GPT-6.2 Astra max OpenAI77.2%93.9p
#5Claude Fable 5.2 max Anthropic76.4%91.8p
#6Gemini 4 Hydrogen xhigh Google76.1%89.8p
#7Kimi K3.1 Pro max Kimi73.3%87.8p
#8Qwen4 Max xhigh Alibaba72.6%85.7p
#9MiniMax-M3.1-Flash max MiniMax72.5%83.7p
#10Claude Sonnet 5.5 max Anthropic71.8%81.6p
#11Muse Wonder Full max Meta71.3%79.6p
#12GPT-6.2 Sol max OpenAI71.3%77.6p
#13Claude Opus 5.5 max Anthropic69.5%75.5p
#14DeepSeek V4.1 Flash max DeepSeek68.9%73.5p
#15GPT-6 Astra max OpenAI68.5%71.4p

Embeddable Audit Badge

View Markdown (.md)

Benchmark maintainers can embed this live SVG badge in GitHub READMEs or documentation:

BenchAudit AutomationBench-AA Tier S 10/10
[![BenchAudit Tier S](https://benchaudit.org/badges/automationbench-aa.svg)](https://benchaudit.org/benchmarks/automationbench-aa/)