37 observations · 37 standings · first ingest Oct 2, 2026
Source record llm-stats-toolathlon · higher is better · declared max 1
| Subject | Value | Instrument | Protocol | Measured by | Reported | Document |
|---|---|---|---|---|---|---|
| Muse Spark 1.1 | 0.76 | – | – | Meta | Jul 9, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V4-Pro-0813 | 0.74 | – | – | DeepSeek | Aug 13, 2026 |
| //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Kimi K3 | 0.73 | – | – | Moonshot AI | Jul 16, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GLM-5.3 | 0.73 | – | – | Z.ai | Aug 14, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Qwen3.8 Max | 0.72 | – | – | Qwen | Aug 2, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V4-Flash-0731 | 0.70 | – | – | DeepSeek | Jul 31, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Claude Opus 4.8 | 0.60 | – | – | Anthropic | May 28, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.6 Sol | 0.58 | – | – | OpenAI | Jul 9, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Gemini 3.5 Flash | 0.56 | – | – | May 19, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.5 | 0.56 | – | – | OpenAI | Apr 23, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.4 | 0.55 | – | – | OpenAI | Mar 5, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Inkling-Small | 0.54 | – | – | Thinking Machines Lab | Jul 30, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Claude Sonnet 5 | 0.54 | – | – | Anthropic | Jun 30, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.6 Luna | 0.53 | – | – | OpenAI | Jul 9, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.6 Terra | 0.53 | – | – | OpenAI | Jul 9, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V4-Pro-Max | 0.52 | – | – | DeepSeek | Apr 23, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Seed 2.1 Pro | 0.51 | – | – | ByteDance | Jun 24, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Kimi K2.6 | 0.50 | – | – | Moonshot AI | Apr 20, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Laguna S 2.1 | 0.50 | – | – | Poolside | Jul 21, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Gemini 3 Flash | 0.49 | – | – | Dec 17, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Seed 2.1 Turbo | 0.49 | – | – | ByteDance | Jun 24, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Hy3 | 0.48 | – | – | Tencent | Jul 6, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GLM-5.2 | 0.48 | – | – | Z.ai | Jun 16, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V4-Flash-Max | 0.48 | – | – | DeepSeek | Apr 23, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| MiniMax M2.7 | 0.46 | – | – | MiniMax | Mar 18, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.2 | 0.46 | – | – | OpenAI | Dec 11, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V4-Flash-0423 | 0.43 | – | – | DeepSeek | Apr 23, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| MiniMax M2.1 | 0.43 | – | – | MiniMax | Dec 23, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.4 mini | 0.43 | – | – | OpenAI | Mar 17, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GLM-5.1 | 0.41 | – | – | Z.ai | Apr 7, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Qwen3.6 Plus | 0.40 | – | – | Qwen | Apr 2, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Qwen3.5-397B-A17B | 0.38 | – | – | Qwen | Feb 16, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| GPT-5.4 nano | 0.35 | – | – | OpenAI | Mar 17, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V3.2 | 0.35 | – | – | DeepSeek | Dec 1, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V3.2 (Thinking) | 0.35 | – | – | DeepSeek | Dec 1, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| DeepSeek-V3.2-Speciale | 0.35 | – | – | DeepSeek | Dec 1, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Qwen3.6-35B-A3B | 0.27 | – | – | Qwen | Apr 16, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500 |
| Subject | Value | Reported | Kind | Ingested |
|---|---|---|---|---|
| GLM-5.3 | 0.73 | Aug 14, 2026 | unspecified | Oct 2, 2026 |
| DeepSeek-V4-Pro-0813 | 0.74 | Aug 13, 2026 | unspecified | Oct 2, 2026 |
| Qwen3.8 Max | 0.72 | Aug 2, 2026 | unspecified | Oct 2, 2026 |
| DeepSeek-V4-Flash-0731 | 0.70 | Jul 31, 2026 | unspecified | Oct 2, 2026 |
| Inkling-Small | 0.54 | Jul 30, 2026 | unspecified | Oct 2, 2026 |
| Laguna S 2.1 | 0.50 | Jul 21, 2026 | unspecified | Oct 2, 2026 |
| Kimi K3 | 0.73 | Jul 16, 2026 | unspecified | Oct 2, 2026 |
| Muse Spark 1.1 | 0.76 | Jul 9, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.6 Sol | 0.58 | Jul 9, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.6 Luna | 0.53 | Jul 9, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.6 Terra | 0.53 | Jul 9, 2026 | unspecified | Oct 2, 2026 |
| Hy3 | 0.48 | Jul 6, 2026 | unspecified | Oct 2, 2026 |
| Claude Sonnet 5 | 0.54 | Jun 30, 2026 | unspecified | Oct 2, 2026 |
| Seed 2.1 Pro | 0.51 | Jun 24, 2026 | unspecified | Oct 2, 2026 |
| Seed 2.1 Turbo | 0.49 | Jun 24, 2026 | unspecified | Oct 2, 2026 |
| GLM-5.2 | 0.48 | Jun 16, 2026 | unspecified | Oct 2, 2026 |
| Claude Opus 4.8 | 0.60 | May 28, 2026 | unspecified | Oct 2, 2026 |
| Gemini 3.5 Flash | 0.56 | May 19, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.5 | 0.56 | Apr 23, 2026 | unspecified | Oct 2, 2026 |
| DeepSeek-V4-Pro-Max | 0.52 | Apr 23, 2026 | unspecified | Oct 2, 2026 |
| DeepSeek-V4-Flash-Max | 0.48 | Apr 23, 2026 | unspecified | Oct 2, 2026 |
| DeepSeek-V4-Flash-0423 | 0.43 | Apr 23, 2026 | unspecified | Oct 2, 2026 |
| Kimi K2.6 | 0.50 | Apr 20, 2026 | unspecified | Oct 2, 2026 |
| Qwen3.6-35B-A3B | 0.27 | Apr 16, 2026 | unspecified | Oct 2, 2026 |
| GLM-5.1 | 0.41 | Apr 7, 2026 | unspecified | Oct 2, 2026 |
| Qwen3.6 Plus | 0.40 | Apr 2, 2026 | unspecified | Oct 2, 2026 |
| MiniMax M2.7 | 0.46 | Mar 18, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.4 mini | 0.43 | Mar 17, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.4 nano | 0.35 | Mar 17, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.4 | 0.55 | Mar 5, 2026 | unspecified | Oct 2, 2026 |
| Qwen3.5-397B-A17B | 0.38 | Feb 16, 2026 | unspecified | Oct 2, 2026 |
| MiniMax M2.1 | 0.43 | Dec 23, 2025 | unspecified | Oct 2, 2026 |
| Gemini 3 Flash | 0.49 | Dec 17, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.2 | 0.46 | Dec 11, 2025 | unspecified | Oct 2, 2026 |
| DeepSeek-V3.2 | 0.35 | Dec 1, 2025 | unspecified | Oct 2, 2026 |
| DeepSeek-V3.2 (Thinking) | 0.35 | Dec 1, 2025 | unspecified | Oct 2, 2026 |
| DeepSeek-V3.2-Speciale | 0.35 | Dec 1, 2025 | unspecified | Oct 2, 2026 |
10 observations · 10 standings · first ingest Oct 2, 2026
Source record toolathlon_verified · higher is better · metric pass@1
| Subject | Value | Instrument | Protocol | Measured by | Reported | Document |
|---|---|---|---|---|---|---|
| GLM-5.3-Flash | 78.4% | toolathlon_verified | official evaluation service, pass@1 averaged over 3 independent runs | Z.ai | Aug 27, 2026(publication date) | zai_glm_5_3_flash_model_card |
| Kimi K3 | 76.5% | tool_decathlon | reasoning=max | Moonshot AI | Jun 13, 2026(publication date) | moonshot_kimi_k3_model_card |
| Hy4 preview | 74.1% | toolathlon_verified | as printed in Table 3 (Large Models) | Tencent | Aug 28, 2026(publication date) | tencent_hy4_preview |
| DeepSeek-V4-Pro-0813 | 74.1% | tool_decathlon | max reasoning effort, temp 1.0, top_p 0.95 | DeepSeek | Aug 13, 2026(publication date) | deepseek_v4_pro_0813_model_card |
| DeepSeek-V4-Pro | 51.8% | tool_decathlon | think max, pass@1 | DeepSeek | Apr 22, 2026(publication date) | deepseek_v4_model_card |
| GLM-5.2 | 48.2% | tool_decathlon | official evaluation service, max_token 128K | Z.ai | Jun 16, 2026(publication date) | zai_glm_5_2_model_card |
| DeepSeek-V4-Flash | 47.8% | tool_decathlon | think max, pass@1 | DeepSeek | Jun 25, 2026(publication date) | deepseek_v4_technical_report |
| GLM-5.1 | 40.7% | tool_decathlon | as reported | Z.ai | Apr 3, 2026(publication date) | zai_glm_5_1_model_card |
| Qwen3.5-397B-A17B | 38.3% | tool_decathlon | thinking (temp 0.6, top_p 0.95, top_k 20) | Qwen | Feb 16, 2026(publication date) | qwen3_5_model_card |
| GLM-5 | 38% | tool_decathlon | as reported | Z.ai | Feb 11, 2026(publication date) | zai_glm_5_model_card |
| Subject | Value | Reported | Kind | Ingested |
|---|---|---|---|---|
| Hy4 preview | 74.1% | Aug 28, 2026 | self-reported | Oct 2, 2026 |
| GLM-5.3-Flash | 78.4% | Aug 27, 2026 | self-reported | Oct 2, 2026 |
| DeepSeek-V4-Pro-0813 | 74.1% | Aug 13, 2026 | self-reported | Oct 2, 2026 |
| DeepSeek-V4-Flash | 47.8% | Jun 25, 2026 | self-reported |
documents that define or report scores for this benchmark
| other | – | Toolathlon– |
| other | DeepSeek | DeepSeek-V4-ProApr 22, 2026 |
| other | DeepSeek | DeepSeek-V4-Pro-0813Aug 13, 2026 |
| tech report | DeepSeek | DeepSeek-V4Jun 25, 2026 |
| other | Moonshot AI | Kimi K3Jun 13, 2026 |
| other | Qwen | Qwen3.5-397B-A17BFeb 16, 2026 |
| tech report | Tencent | Hy4 previewAug 28, 2026 |
| other | Z.ai | GLM-5.1Apr 3, 2026 |
| other | Z.ai | GLM-5.2Jun 16, 2026 |
| other | Z.ai | GLM-5.3-FlashAug 27, 2026 |
| other | Z.ai | GLM-5Feb 11, 2026 |
Observations and source documents are located via the tracking corpus. Every row cites its primary source document; observations from different sources stay in separate partitions and are never merged into a cross-source ranking.
| Oct 2, 2026 |
| GLM-5.2 | 48.2% | Jun 16, 2026 | self-reported | Oct 2, 2026 |
| Kimi K3 | 76.5% | Jun 13, 2026 | self-reported | Oct 2, 2026 |
| DeepSeek-V4-Pro | 51.8% | Apr 22, 2026 | self-reported | Oct 2, 2026 |
| GLM-5.1 | 40.7% | Apr 3, 2026 | self-reported | Oct 2, 2026 |
| Qwen3.5-397B-A17B | 38.3% | Feb 16, 2026 | self-reported | Oct 2, 2026 |
| GLM-5 | 38% | Feb 11, 2026 | self-reported | Oct 2, 2026 |