Model & system cards
8 scores · 8 models · first recorded Nov 12, 2026
higher is better · success_rate · scale 0–100 · record ClawBench
Current standings — best score per model × subset
| Model | Value (%) | Subset / variant | Test conditions | Reported by | Reported | Document |
|---|---|---|---|---|---|---|
| Claude Sonnet 4.6 | 33.3% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; high reasoning effort; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| Qwen3.5-397B-A17B | 26.1% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; greedy decoding at provider default; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| GLM-5 | 24.2% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; greedy decoding at provider default; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| Gemini 3 Flash | 19% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; greedy decoding at provider default; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| Claude Haiku 4.5 | 18.3% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; greedy decoding at provider default; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| Gemini 3.1 Pro | 9.80% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; greedy decoding at provider default; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| GPT-5.4 | 6.50% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; high reasoning effort; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |
| Gemini 3.1 Flash-Lite | 3.30% | clawbench_paper_153_tasks | OpenClaw harness; 153 live-web tasks; trajectory Agent-as-Judge (Claude Sonnet 4.6) against human references; paper Table 3 overall success rate; greedy decoding at provider default; provider-default top_p (1.0); 8192 output tokens per turn; 1800s wall-clock limit; 5min idle timeout | TIGER-AI-Lab | Jul 20, 2026 | clawbench_v1_paper_results |