84 observations · 37 standings · first ingest Oct 2, 2026
Source record llm-stats-tau2-airline · higher is better · declared max 1
| Subject | Value | Instrument | Protocol | Measured by | Reported | Document |
|---|---|---|---|---|---|---|
| LongCat-Flash-Thinking-2601 | 0.99 | – | – | Meituan | Jan 14, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Claude Opus 4.6 | 0.99 | – | – | Anthropic | Feb 5, 2026 |
| //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.4 | 0.99 | – | – | OpenAI | Mar 5, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.2 | 0.99 | – | – | OpenAI | Dec 11, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Claude Opus 4.5 | 0.98 | – | – | Anthropic | Nov 24, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.5 | 0.98 | – | – | OpenAI | Apr 23, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Claude Sonnet 4.6 | 0.98 | – | – | Anthropic | Feb 17, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| MiMo-V2-Pro | 0.97 | – | – | Xiaomi | Mar 18, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5 | 0.97 | – | – | OpenAI | Aug 7, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.1 | 0.96 | – | – | OpenAI | Nov 13, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.1 Thinking | 0.96 | – | – | OpenAI | Nov 12, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.1 Instant | 0.96 | – | – | OpenAI | Nov 12, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.4 mini | 0.93 | – | – | OpenAI | Mar 17, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Nova 2 Pro | 0.93 | – | – | Amazon | Dec 2, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-5.4 nano | 0.93 | – | – | OpenAI | Mar 17, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Muse Spark | 0.92 | – | – | Meta | Apr 8, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| MiniMax M2.1 | 0.87 | – | – | MiniMax | Dec 23, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| MiniMax M2 | 0.87 | – | – | MiniMax | Oct 27, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Command A+ | 0.85 | – | – | Cohere | May 20, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Claude Haiku 4.5 | 0.83 | – | – | Anthropic | Oct 15, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| LongCat-Flash-Thinking | 0.83 | – | – | Meituan | Sep 22, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| o3 | 0.80 | – | – | OpenAI | Apr 16, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Nova 2 Omni | 0.80 | – | – | Amazon | Dec 2, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Nova 2 Lite | 0.77 | – | – | Amazon | Dec 2, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| LongCat-Flash-Chat | 0.74 | – | – | Meituan | Aug 29, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| LongCat-Flash-Lite | 0.73 | – | – | Meituan | Feb 5, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Qwen3-235B-A22B-Thinking-2507 | 0.72 | – | – | Qwen | Jul 25, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| MAI-Code-1-Flash | 0.72 | – | – | Microsoft | Jun 2, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| Qwen3-235B-A22B-Instruct-2507 | 0.71 | – | – | Qwen | Jul 22, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Kimi K2-Instruct-0905 | 0.71 | – | – | Moonshot AI | Sep 5, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Kimi K2 Instruct | 0.71 | – | – | Moonshot AI | Jul 11, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Qwen3-Next-80B-A3B-Thinking | 0.68 | – | – | Qwen | Sep 10, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Nemotron 3 Super (120B A12B) | 0.64 | – | – | NVIDIA | Mar 11, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500 |
| GPT-4o | 0.63 | – | – | OpenAI | Aug 6, 2024(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Qwen3-Next-80B-A3B-Instruct | 0.57 | – | – | Qwen | Sep 10, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Nemotron 3 Nano (30B A3B) | 0.57 | – | – | NVIDIA | Dec 15, 2025(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-retail?top_n=500 |
| Mercury 2 | 0.53 | – | – | Inception | Feb 24, 2026(model_announcement) | //api.zeroeval.com/leaderboard/benchmarks/tau2-airline?top_n=500 |
| Subject | Value | Reported | Kind | Ingested |
|---|---|---|---|---|
| MAI-Code-1-Flash | 0.72 | Jun 2, 2026 | unspecified | Oct 2, 2026 |
| Command A+ | 0.85 | May 20, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.5 | 0.98 | Apr 23, 2026 | unspecified | Oct 2, 2026 |
| Muse Spark | 0.92 | Apr 8, 2026 | unspecified | Oct 2, 2026 |
| MiMo-V2-Pro | 0.97 | Mar 18, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.4 mini | 0.93 | Mar 17, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.4 nano | 0.93 | Mar 17, 2026 | unspecified | Oct 2, 2026 |
| Nemotron 3 Super (120B A12B) | 0.56 | Mar 11, 2026 | unspecified | Oct 2, 2026 |
| Nemotron 3 Super (120B A12B) | 0.63 | Mar 11, 2026 | unspecified | Oct 2, 2026 |
| Nemotron 3 Super (120B A12B) | 0.64 | Mar 11, 2026 | unspecified | Oct 2, 2026 |
| GPT-5.4 | 0.99 | Mar 5, 2026 | unspecified | Oct 2, 2026 |
| Mercury 2 | 0.53 | Feb 24, 2026 | unspecified | Oct 2, 2026 |
| Claude Sonnet 4.6 | 0.92 | Feb 17, 2026 | unspecified | Oct 2, 2026 |
| Claude Sonnet 4.6 | 0.98 | Feb 17, 2026 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Lite | 0.58 | Feb 5, 2026 | unspecified | Oct 2, 2026 |
| Claude Opus 4.6 | 0.92 | Feb 5, 2026 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Lite | 0.73 | Feb 5, 2026 | unspecified | Oct 2, 2026 |
| Claude Opus 4.6 | 0.99 | Feb 5, 2026 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Lite | 0.73 | Feb 5, 2026 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Thinking-2601 | 0.77 | Jan 14, 2026 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Thinking-2601 | 0.89 | Jan 14, 2026 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Thinking-2601 | 0.99 | Jan 14, 2026 | unspecified | Oct 2, 2026 |
| MiniMax M2.1 | 0.87 | Dec 23, 2025 | unspecified | Oct 2, 2026 |
| Nemotron 3 Nano (30B A3B) | 0.48 | Dec 15, 2025 | unspecified | Oct 2, 2026 |
| Nemotron 3 Nano (30B A3B) | 0.57 | Dec 15, 2025 | unspecified | Oct 2, 2026 |
| Nemotron 3 Nano (30B A3B) | 0.42 | Dec 15, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.2 | 0.82 | Dec 11, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.2 | 0.99 | Dec 11, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Omni | 0.69 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Pro | 0.65 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Lite | 0.65 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Omni | 0.78 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Pro | 0.78 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Lite | 0.77 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Pro | 0.93 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Omni | 0.80 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Nova 2 Lite | 0.76 | Dec 2, 2025 | unspecified | Oct 2, 2026 |
| Claude Opus 4.5 | 0.89 | Nov 24, 2025 | unspecified | Oct 2, 2026 |
| Claude Opus 4.5 | 0.98 | Nov 24, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 | 0.67 | Nov 13, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 | 0.78 | Nov 13, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 | 0.96 | Nov 13, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 Thinking | 0.67 | Nov 12, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 Instant | 0.67 | Nov 12, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 Instant | 0.78 | Nov 12, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 Thinking | 0.78 | Nov 12, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 Thinking | 0.96 | Nov 12, 2025 | unspecified | Oct 2, 2026 |
| GPT-5.1 Instant | 0.96 | Nov 12, 2025 | unspecified | Oct 2, 2026 |
| MiniMax M2 | 0.87 | Oct 27, 2025 | unspecified | Oct 2, 2026 |
| Claude Haiku 4.5 | 0.64 | Oct 15, 2025 | unspecified | Oct 2, 2026 |
| Claude Haiku 4.5 | 0.83 | Oct 15, 2025 | unspecified | Oct 2, 2026 |
| Claude Haiku 4.5 | 0.83 | Oct 15, 2025 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Thinking | 0.68 | Sep 22, 2025 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Thinking | 0.71 | Sep 22, 2025 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Thinking | 0.83 | Sep 22, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-Next-80B-A3B-Thinking | 0.60 | Sep 10, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-Next-80B-A3B-Instruct | 0.46 | Sep 10, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-Next-80B-A3B-Thinking | 0.68 | Sep 10, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-Next-80B-A3B-Instruct | 0.57 | Sep 10, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-Next-80B-A3B-Thinking | 0.44 | Sep 10, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-Next-80B-A3B-Instruct | 0.13 | Sep 10, 2025 | unspecified | Oct 2, 2026 |
| Kimi K2-Instruct-0905 | 0.56 | Sep 5, 2025 | unspecified | Oct 2, 2026 |
| Kimi K2-Instruct-0905 | 0.71 | Sep 5, 2025 | unspecified | Oct 2, 2026 |
| Kimi K2-Instruct-0905 | 0.66 | Sep 5, 2025 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Chat | 0.58 | Aug 29, 2025 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Chat | 0.71 | Aug 29, 2025 | unspecified | Oct 2, 2026 |
| LongCat-Flash-Chat | 0.74 | Aug 29, 2025 | unspecified | Oct 2, 2026 |
| GPT-5 | 0.63 | Aug 7, 2025 | unspecified | Oct 2, 2026 |
| GPT-5 | 0.81 | Aug 7, 2025 | unspecified | Oct 2, 2026 |
| GPT-5 | 0.97 | Aug 7, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-235B-A22B-Thinking-2507 | 0.58 | Jul 25, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-235B-A22B-Thinking-2507 | 0.72 | Jul 25, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-235B-A22B-Thinking-2507 | 0.46 | Jul 25, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-235B-A22B-Instruct-2507 | 0.44 | Jul 22, 2025 | unspecified | Oct 2, 2026 |
| Qwen3-235B-A22B-Instruct-2507 | 0.71 | Jul 22, 2025 | unspecified | Oct 2, 2026 |
| Kimi K2 Instruct | 0.56 | Jul 11, 2025 | unspecified | Oct 2, 2026 |
| Kimi K2 Instruct | 0.71 | Jul 11, 2025 | unspecified | Oct 2, 2026 |
| Kimi K2 Instruct | 0.66 | Jul 11, 2025 | unspecified | Oct 2, 2026 |
| o3 | 0.65 | Apr 16, 2025 | unspecified | Oct 2, 2026 |
| o3 | 0.80 | Apr 16, 2025 | unspecified | Oct 2, 2026 |
| o3 | 0.58 | Apr 16, 2025 | unspecified | Oct 2, 2026 |
| GPT-4o | 0.46 | Aug 6, 2024 | unspecified | Oct 2, 2026 |
| GPT-4o | 0.63 | Aug 6, 2024 | unspecified | Oct 2, 2026 |
| GPT-4o | 0.23 | Aug 6, 2024 | unspecified | Oct 2, 2026 |
5 observations · 4 standings · first ingest Oct 2, 2026
Source record tau2_bench · higher is better · metric accuracy
| Subject | Value | Instrument | Protocol | Measured by | Reported | Document |
|---|---|---|---|---|---|---|
| Gemini 3.1 Pro | 99.3% | tau2_bench | Thinking (High), Telecom | Feb 19, 2026(publication date) | google_gemini_3_1_pro_model_card | |
| GLM-5 | 89.7% | tau2_bench | Retail and Telecom prompt adjustment; Airline domain fixes per Claude Opus 4.5 system card | Z.ai | Feb 11, 2026(publication date) | zai_glm_5_model_card |
| Qwen3.5-397B-A17B | 86.7% | tau2_bench | thinking (temp 0.6, top_p 0.95, top_k 20) | Qwen | Feb 16, 2026(publication date) | qwen3_5_model_card |
| Gemma 4 (31B) | 76.9% | tau2_bench | average over 3 | Mar 11, 2026(publication date) | google_gemma_4_model_card |
| Subject | Value | Reported | Kind | Ingested |
|---|---|---|---|---|
| Gemma 4 (31B) | 76.9% | Mar 11, 2026 | self-reported | Oct 2, 2026 |
| Gemini 3.1 Pro | 90.8% | Feb 19, 2026 | self-reported | Oct 2, 2026 |
| Gemini 3.1 Pro | 99.3% | Feb 19, 2026 | self-reported | Oct 2, 2026 |
| Qwen3.5-397B-A17B | 86.7% | Feb 16, 2026 | self-reported | Oct 2, 2026 |
documents that define or report scores for this benchmark
| other | – | Tau2 Airline– |
| other | – | Tau2 Retail– |
| other | – | Tau2 Telecom– |
| other | Gemini 3.1 ProFeb 19, 2026 | |
| other | Gemma 4 (31B)Mar 11, 2026 | |
| other | Qwen | Qwen3.5-397B-A17BFeb 16, 2026 |
| other | Z.ai | GLM-5.1Apr 3, 2026 |
| other | Z.ai | GLM-5Feb 11, 2026 |
| tech report | Z.ai | GLM-5Feb 11, 2026 |
Observations and source documents are located via the tracking corpus. Every row cites its primary source document; observations from different sources stay in separate partitions and are never merged into a cross-source ranking.
| GLM-5 | 89.7% | Feb 11, 2026 | self-reported | Oct 2, 2026 |