๐ WritingBench: A Comprehensive Benchmark for Generative Writing
Evaluating LLMs' writing capabilities across 1,000 real-world queries, spanning 6 primary domains and 100 fine-grained subdomains.
๐ Last Updated: 2025-09-05 | ๐ Scored by: Claude-Sonnet-4
10 | claude-3-7-sonnet-20250219-thinking | Mistral AI | 81.34 |
10 | claude-3-7-sonnet-20250219-thinking | 81.34 | 82.35 | 81.57 | 79.10 | 82.17 | 83.24 | 80.28 |
1 | qwen3-235b-a22b-thinking-2507 | 81.34 | 82.35 | 81.57 | 79.10 | 82.17 | 83.24 | 80.28 |
2 | gpt-5-2025-08-07 | 81.21 | 81.50 | 80.81 | 79.87 | 82.77 | 82.85 | 79.97 |
3 | kimi-k2-instruct | 79.04 | 79.81 | 78.35 | 78.20 | 80.64 | 78.65 | 78.52 |
4 | doubao-seed-1-6-thinking-250715 | 78.92 | 79.32 | 78.16 | 77.87 | 79.80 | 80.68 | 78.47 |
5 | qwen3-235b-a22b-instruct-2507 | 78.66 | 78.53 | 78.76 | 77.84 | 79.87 | 79.21 | 77.73 |
6 | doubao-seed-1-6-thinking-250615 | 77.91 | 78.96 | 77.90 | 76.91 | 76.91 | 80.54 | 77.28 |
7 | o3-2025-04-16 | 77.68 | 77.99 | 76.99 | 76.25 | 78.54 | 78.16 | 79.00 |
8 | deepseek-r1-0528 | 77.49 | 77.23 | 76.20 | 76.09 | 79.88 | 79.23 | 77.22 |
9 | gemini-2.5-pro | 77.35 | 76.42 | 75.85 | 77.45 | 79.96 | 79.69 | 75.12 |
10 | doubao-seed-1-6-250615 | 77.03 | 78.17 | 76.74 | 72.30 | 79.22 | 79.64 | 78.08 |
11 | claude-opus-4-1-20250805 | 76.10 | 76.20 | 74.95 | 74.14 | 78.74 | 77.03 | 76.33 |
12 | claude-opus-4-20250514-thinking | 74.75 | 74.54 | 73.79 | 73.24 | 77.20 | 75.53 | 74.77 |
13 | claude-opus-4-20250514 | 74.22 | 72.86 | 72.84 | 73.31 | 76.86 | 75.59 | 74.69 |
14 | claude-sonnet-4-20250514 | 74.13 | 74.68 | 73.50 | 71.96 | 75.80 | 74.81 | 74.83 |
15 | claude-sonnet-4-20250514-thinking | 73.92 | 74.30 | 73.90 | 70.93 | 76.22 | 75.89 | 73.12 |
16 | qwen3-235b-a22b-thinking | 73.12 | 72.65 | 71.90 | 72.75 | 74.05 | 75.35 | 73.05 |
17 | claude-3-7-sonnet-20250219-thinking | 72.36 | 71.02 | 71.54 | 71.93 | 73.53 | 73.48 | 73.50 |
18 | claude-3-7-sonnet-20250219 | 71.70 | 71.39 | 71.16 | 70.42 | 72.03 | 73.53 | 72.97 |
19 | qwq-32b | 71.22 | 70.92 | 69.65 | 71.09 | 70.21 | 74.36 | 73.08 |
20 | chatgpt-4o-latest | 70.97 | 69.16 | 70.09 | 70.56 | 72.38 | 71.66 | 72.78 |
21 | deepseek-r1 | 70.54 | 69.45 | 67.97 | 70.56 | 71.38 | 73.05 | 72.81 |
22 | o4-mini-2025-04-16 | 70.52 | 72.97 | 69.15 | 69.70 | 68.28 | 73.48 | 71.48 |
23 | qwen3-235b-a22b-nothinking | 70.47 | 69.07 | 69.90 | 69.95 | 71.81 | 72.25 | 70.56 |
24 | grok-3 | 69.80 | 68.54 | 68.83 | 70.99 | 70.39 | 72.43 | 68.08 |
25 | gpt-4.1-2025-04-14 | 69.00 | 68.23 | 67.39 | 69.08 | 69.89 | 70.18 | 70.23 |
26 | deepseek-v3-250324 | 67.74 | 67.13 | 65.08 | 67.24 | 68.94 | 69.64 | 70.31 |
27 | o3-mini | 64.88 | 65.01 | 64.63 | 64.28 | 63.69 | 69.44 | 63.84 |
28 | qwen-max | 64.68 | 63.93 | 62.31 | 63.35 | 66.31 | 67.53 | 66.80 |
29 | gpt-4.5-preview | 64.31 | 63.75 | 61.49 | 64.79 | 64.14 | 67.15 | 66.69 |
30 | gemini-2.0-flash | 64.14 | 63.70 | 62.36 | 63.65 | 64.49 | 67.19 | 65.25 |
31 | o1-preview-2024-09-12 | 61.46 | 61.17 | 58.89 | 60.58 | 62.37 | 64.18 | 63.78 |
32 | deepseek-v3 | 56.32 | 56.50 | 53.67 | 55.50 | 55.57 | 59.66 | 59.88 |
33 | gemini-1.5-pro | 55.94 | 55.27 | 51.23 | 54.77 | 58.54 | 59.39 | 59.72 |
34 | qwen-2.5-72b-instruct | 54.19 | 55.47 | 52.09 | 54.26 | 51.66 | 59.24 | 55.06 |
35 | claude-3-5-sonnet-20241022 | 53.25 | 52.00 | 50.82 | 51.06 | 53.52 | 55.41 | 60.05 |
36 | gpt-4o-mini | 52.36 | 52.91 | 49.61 | 50.35 | 52.44 | 55.44 | 56.55 |
37 | mistral-large-instruct-2411 | 50.86 | 51.62 | 49.51 | 50.46 | 48.48 | 54.18 | 53.25 |
38 | llama-4-maverick-17b-128e | 48.68 | 47.49 | 46.62 | 48.12 | 47.89 | 51.12 | 53.52 |
39 | llama-4-scout-17b-16e-instruct | 47.84 | 47.84 | 47.02 | 47.53 | 45.72 | 50.20 | 50.69 |
40 | claude-3-5-haiku-20241022 | 47.57 | 47.66 | 45.93 | 45.68 | 46.07 | 48.54 | 54.38 |
41 | qwen-2.5-7b-instruct | 45.33 | 47.47 | 45.10 | 45.45 | 39.84 | 50.16 | 46.41 |
42 | llama-3.3-70b-instruct | 43.97 | 43.41 | 43.27 | 42.22 | 42.23 | 46.97 | 48.47 |
43 | qwen3-0.6b-thinking | 42.00 | 46.54 | 42.90 | 42.36 | 35.91 | 44.45 | 40.66 |
44 | qwen3-0.6b-nothinking | 38.53 | 42.49 | 39.45 | 38.94 | 32.54 | 39.84 | 38.64 |
45 | llama-3.1-8b-instruct | 36.52 | 37.66 | 36.68 | 33.84 | 33.40 | 39.24 | 41.08 |
46 | llama-3.2-3b-instruct | 34.52 | 36.14 | 35.08 | 31.78 | 31.01 | 37.80 | 37.97 |
47 | llama-3.2-1b-instruct | 26.17 | 29.84 | 27.00 | 23.36 | 22.68 | 27.57 | 28.23 |
10 | claude-3-7-sonnet-20250219-thinking | 81.34 | 86.73 | 86.17 | 81.82 | 85.25 | 82.00 | 80.73 | 78.00 | 83.50 | 83.67 | 84.40 | 82.50 | 82.20 | 82.80 | 84.17 | 76.89 | 84.00 | 72.25 | 82.40 | 81.75 | 81.50 | 81.40 | 84.40 | 81.09 | 84.83 | 83.67 | 83.64 | 82.50 | 81.82 | 79.33 | 83.00 | 84.15 | 79.29 | 84.57 | 77.75 | 81.71 | 77.50 | 78.73 | 87.40 | 82.46 | 81.60 | 76.44 | 76.44 | 73.78 | 85.82 | 84.50 | 80.33 | 82.29 | 82.86 | 75.56 | 77.87 | 82.25 | 72.15 | 77.33 | 73.14 | 75.67 | 76.33 | 74.18 | 84.80 | 84.00 | 80.55 | 74.00 | 85.50 | 86.50 | 82.33 | 87.50 | 82.18 | 76.22 | 85.09 | 85.47 | 83.83 | 85.27 | 86.67 | 83.67 | 77.45 | 83.60 | 75.33 | 68.67 | 85.71 | 86.25 | 86.92 | 79.33 | 80.22 | 87.14 | 83.14 | 84.57 | 78.00 | 81.75 | 79.50 | 80.67 | 81.14 | 80.62 | 78.92 | 85.09 | 78.33 | 85.08 | 80.00 | 82.73 | 79.20 | 77.75 | 74.94 |
1 | qwen3-235b-a22b-thinking-2507 | 81.34 | 86.73 | 86.17 | 81.82 | 85.25 | 82.00 | 80.73 | 78.00 | 83.50 | 83.67 | 84.40 | 82.50 | 82.20 | 82.80 | 84.17 | 76.89 | 84.00 | 72.25 | 82.40 | 81.75 | 81.50 | 81.40 | 84.40 | 81.09 | 84.83 | 83.67 | 83.64 | 82.50 | 81.82 | 79.33 | 83.00 | 84.15 | 79.29 | 84.57 | 77.75 | 81.71 | 77.50 | 78.73 | 87.40 | 82.46 | 81.60 | 76.44 | 76.44 | 73.78 | 85.82 | 84.50 | 80.33 | 82.29 | 82.86 | 75.56 | 77.87 | 82.25 | 72.15 | 77.33 | 73.14 | 75.67 | 76.33 | 74.18 | 84.80 | 84.00 | 80.55 | 74.00 | 85.50 | 86.50 | 82.33 | 87.50 | 82.18 | 76.22 | 85.09 | 85.47 | 83.83 | 85.27 | 86.67 | 83.67 | 77.45 | 83.60 | 75.33 | 68.67 | 85.71 | 86.25 | 86.92 | 79.33 | 80.22 | 87.14 | 83.14 | 84.57 | 78.00 | 81.75 | 79.50 | 80.67 | 81.14 | 80.62 | 78.92 | 85.09 | 78.33 | 85.08 | 80.00 | 82.73 | 79.20 | 77.75 | 74.94 |
2 | gpt-5-2025-08-07 | 81.21 | 84.73 | 88.50 | 80.55 | 84.25 | 80.67 | 79.09 | 77.67 | 81.00 | 78.50 | 84.40 | 83.50 | 81.00 | 79.60 | 82.67 | 82.00 | 80.50 | 75.75 | 86.00 | 76.75 | 78.00 | 83.60 | 87.20 | 80.18 | 82.00 | 83.50 | 85.64 | 83.50 | 86.36 | 81.83 | 72.67 | 80.77 | 79.53 | 84.71 | 80.00 | 66.57 | 75.30 | 78.18 | 83.80 | 80.46 | 83.87 | 80.67 | 87.11 | 80.00 | 86.91 | 84.00 | 79.83 | 78.86 | 78.00 | 76.44 | 74.00 | 69.75 | 81.38 | 78.33 | 80.00 | 75.00 | 79.33 | 74.73 | 86.80 | 80.33 | 87.45 | 89.43 | 83.50 | 87.00 | 85.67 | 78.00 | 82.00 | 76.89 | 76.00 | 83.47 | 82.33 | 83.82 | 82.89 | 85.17 | 83.09 | 81.80 | 81.67 | 66.67 | 88.29 | 88.50 | 86.00 | 85.11 | 83.33 | 85.71 | 80.57 | 81.86 | 81.25 | 81.25 | 72.50 | 74.33 | 85.86 | 78.77 | 80.00 | 84.55 | 74.00 | 85.54 | 76.57 | 80.73 | 80.40 | 81.75 | 70.59 |
3 | kimi-k2-instruct | 79.04 | 87.09 | 87.00 | 78.18 | 81.00 | 80.67 | 80.00 | 74.17 | 80.67 | 79.67 | 83.40 | 83.75 | 80.80 | 75.40 | 79.67 | 73.33 | 80.25 | 69.75 | 79.20 | 80.75 | 76.00 | 79.60 | 82.00 | 77.82 | 83.00 | 80.50 | 81.82 | 80.25 | 79.09 | 76.83 | 79.00 | 79.85 | 79.65 | 80.57 | 75.00 | 77.14 | 72.00 | 69.64 | 82.40 | 80.15 | 84.80 | 77.33 | 82.00 | 75.78 | 81.64 | 83.50 | 82.83 | 76.57 | 80.57 | 78.22 | 72.00 | 81.00 | 71.54 | 76.00 | 79.14 | 77.00 | 60.00 | 74.55 | 86.00 | 81.00 | 84.36 | 86.57 | 79.50 | 85.25 | 85.00 | 85.50 | 82.18 | 74.22 | 79.64 | 78.67 | 78.00 | 80.91 | 82.22 | 77.83 | 80.18 | 78.40 | 78.17 | 71.33 | 84.29 | 82.25 | 81.23 | 79.11 | 81.56 | 80.14 | 77.43 | 79.00 | 76.25 | 71.75 | 72.00 | 75.00 | 83.86 | 75.54 | 79.23 | 82.00 | 76.00 | 83.38 | 77.43 | 78.73 | 78.80 | 79.25 | 70.47 |
4 | doubao-seed-1-6-thinking-250715 | 78.92 | 84.36 | 87.50 | 78.91 | 83.00 | 70.67 | 77.27 | 75.00 | 81.50 | 80.83 | 80.40 | 80.00 | 76.40 | 81.00 | 76.83 | 75.33 | 77.75 | 73.25 | 82.80 | 82.50 | 76.00 | 79.80 | 83.00 | 76.73 | 81.67 | 79.17 | 79.45 | 78.75 | 79.82 | 79.00 | 79.33 | 79.85 | 78.00 | 80.71 | 73.25 | 69.71 | 72.00 | 74.55 | 83.60 | 80.46 | 82.80 | 80.44 | 81.33 | 78.22 | 83.64 | 80.83 | 79.83 | 81.14 | 78.29 | 80.22 | 75.07 | 65.50 | 69.23 | 76.50 | 77.71 | 75.33 | 64.67 | 75.45 | 86.00 | 84.33 | 82.36 | 82.86 | 85.50 | 81.75 | 82.00 | 85.00 | 77.27 | 80.67 | 76.91 | 80.00 | 80.33 | 79.09 | 83.11 | 79.50 | 76.36 | 78.80 | 70.83 | 75.33 | 85.71 | 85.50 | 83.08 | 84.00 | 80.44 | 82.86 | 75.14 | 81.14 | 78.50 | 78.00 | 74.00 | 74.33 | 84.29 | 75.38 | 76.77 | 83.09 | 77.00 | 84.77 | 74.00 | 81.09 | 76.93 | 77.25 | 72.12 |
5 | qwen3-235b-a22b-instruct-2507 | 78.66 | 83.27 | 82.83 | 78.91 | 82.50 | 77.33 | 74.91 | 71.17 | 78.83 | 78.17 | 82.40 | 79.75 | 75.60 | 79.00 | 80.00 | 74.00 | 79.50 | 76.50 | 80.80 | 78.25 | 74.00 | 81.40 | 78.00 | 78.73 | 80.00 | 81.33 | 80.73 | 81.25 | 79.64 | 77.67 | 81.67 | 80.15 | 79.29 | 77.57 | 78.25 | 78.00 | 75.30 | 75.27 | 84.60 | 78.62 | 78.67 | 80.89 | 78.00 | 78.44 | 83.27 | 81.83 | 77.83 | 80.86 | 80.86 | 80.67 | 74.80 | 77.00 | 78.77 | 75.50 | 75.14 | 70.00 | 66.00 | 69.27 | 84.40 | 75.00 | 82.18 | 85.71 | 83.50 | 86.25 | 82.83 | 84.50 | 75.64 | 71.56 | 82.00 | 81.33 | 76.67 | 80.73 | 80.00 | 82.50 | 74.55 | 82.80 | 76.17 | 68.67 | 83.14 | 82.50 | 82.15 | 81.78 | 81.78 | 78.29 | 72.57 | 80.57 | 78.25 | 75.50 | 73.25 | 77.00 | 83.86 | 77.38 | 76.77 | 82.18 | 75.00 | 83.54 | 72.00 | 79.64 | 79.20 | 72.00 | 69.88 |
6 | doubao-seed-1-6-thinking-250615 | 77.91 | 80.91 | 85.17 | 77.45 | 83.75 | 72.00 | 77.45 | 72.33 | 81.67 | 79.67 | 82.00 | 77.75 | 77.20 | 80.00 | 77.50 | 77.33 | 79.25 | 75.75 | 82.40 | 79.25 | 80.50 | 80.00 | 81.80 | 74.73 | 79.33 | 78.17 | 81.27 | 80.25 | 80.73 | 75.67 | 76.67 | 79.23 | 77.29 | 80.71 | 73.75 | 72.00 | 73.70 | 75.27 | 81.00 | 79.08 | 80.67 | 80.22 | 82.22 | 77.78 | 82.91 | 81.67 | 79.00 | 78.00 | 79.43 | 78.89 | 75.87 | 63.25 | 69.23 | 73.50 | 75.14 | 71.33 | 62.67 | 76.36 | 82.80 | 87.00 | 78.55 | 82.86 | 81.00 | 76.00 | 79.17 | 83.50 | 72.91 | 80.00 | 74.55 | 78.00 | 79.33 | 72.18 | 80.22 | 77.83 | 70.55 | 72.20 | 72.33 | 66.00 | 83.43 | 86.00 | 81.85 | 84.89 | 80.44 | 82.43 | 74.29 | 80.86 | 77.50 | 79.25 | 75.50 | 75.33 | 84.57 | 75.69 | 76.00 | 77.82 | 76.00 | 83.69 | 73.43 | 78.00 | 77.33 | 74.75 | 70.94 |
7 | o3-2025-04-16 | 77.68 | 86.18 | 85.17 | 76.36 | 83.00 | 79.33 | 78.91 | 76.17 | 81.17 | 81.83 | 68.20 | 79.75 | 78.60 | 69.20 | 80.17 | 74.44 | 75.00 | 68.00 | 79.20 | 81.50 | 77.00 | 76.60 | 77.60 | 78.18 | 80.50 | 78.33 | 79.09 | 82.75 | 77.82 | 76.83 | 78.33 | 78.31 | 76.35 | 76.43 | 68.00 | 74.86 | 72.30 | 74.91 | 82.80 | 79.85 | 75.87 | 76.00 | 80.89 | 75.56 | 80.18 | 83.17 | 79.67 | 78.00 | 76.57 | 78.00 | 74.40 | 79.50 | 72.62 | 77.17 | 72.57 | 72.33 | 59.00 | 62.91 | 80.40 | 82.67 | 86.36 | 83.71 | 79.50 | 82.75 | 81.67 | 86.00 | 78.73 | 77.78 | 79.27 | 78.67 | 76.50 | 76.36 | 77.33 | 78.00 | 78.18 | 64.40 | 77.17 | 65.33 | 83.14 | 82.25 | 80.15 | 80.44 | 80.44 | 79.00 | 74.57 | 79.71 | 77.75 | 76.00 | 72.50 | 65.33 | 85.14 | 79.23 | 78.62 | 81.27 | 78.00 | 82.00 | 79.43 | 78.73 | 80.13 | 79.00 | 69.65 |
8 | deepseek-r1-0528 | 77.49 | 83.64 | 82.33 | 77.82 | 79.50 | 80.00 | 73.82 | 75.00 | 78.67 | 76.00 | 75.60 | 77.75 | 77.20 | 74.60 | 79.00 | 76.44 | 72.75 | 72.00 | 81.20 | 78.00 | 76.50 | 80.20 | 81.40 | 76.55 | 79.50 | 78.00 | 77.45 | 80.75 | 76.36 | 75.33 | 74.67 | 75.69 | 75.88 | 73.29 | 74.25 | 76.86 | 73.70 | 65.64 | 84.80 | 79.38 | 78.80 | 79.78 | 81.78 | 75.11 | 80.36 | 79.33 | 80.00 | 73.14 | 74.86 | 73.33 | 72.80 | 73.00 | 72.77 | 73.00 | 78.57 | 71.00 | 65.33 | 67.64 | 85.60 | 78.00 | 82.36 | 85.71 | 83.50 | 84.25 | 82.17 | 83.00 | 77.64 | 76.22 | 81.45 | 79.07 | 81.17 | 82.73 | 75.11 | 79.50 | 78.55 | 77.40 | 75.83 | 68.67 | 85.43 | 82.75 | 81.23 | 81.11 | 80.67 | 81.29 | 74.86 | 79.43 | 76.75 | 71.50 | 75.25 | 76.67 | 80.00 | 76.62 | 77.38 | 83.45 | 76.00 | 81.69 | 72.00 | 80.36 | 76.27 | 74.25 | 70.59 |
9 | gemini-2.5-pro | 77.35 | 82.00 | 83.33 | 76.00 | 82.00 | 77.33 | 69.45 | 66.83 | 79.17 | 78.67 | 79.40 | 78.25 | 74.80 | 79.00 | 74.50 | 73.11 | 76.50 | 69.25 | 83.20 | 74.50 | 76.00 | 75.60 | 79.60 | 74.18 | 76.17 | 79.67 | 76.55 | 77.75 | 77.27 | 76.33 | 71.67 | 76.15 | 74.82 | 73.86 | 74.50 | 74.57 | 74.20 | 74.18 | 83.40 | 78.00 | 78.80 | 78.44 | 79.33 | 77.56 | 80.91 | 79.67 | 78.50 | 78.00 | 78.29 | 74.00 | 72.80 | 75.25 | 76.31 | 76.33 | 76.00 | 75.67 | 77.33 | 74.55 | 83.20 | 75.33 | 85.09 | 84.86 | 79.00 | 87.00 | 82.33 | 81.00 | 82.36 | 70.89 | 81.09 | 74.67 | 80.83 | 81.82 | 82.44 | 77.50 | 81.64 | 81.20 | 74.00 | 71.33 | 83.43 | 84.00 | 83.38 | 79.56 | 79.11 | 83.86 | 71.71 | 81.57 | 76.25 | 76.00 | 72.50 | 77.00 | 76.86 | 74.62 | 75.38 | 79.27 | 71.67 | 80.77 | 69.71 | 75.45 | 75.20 | 74.00 | 70.59 |
10 | doubao-seed-1-6-250615 | 77.03 | 82.18 | 83.67 | 78.73 | 79.75 | 78.00 | 74.18 | 75.00 | 81.67 | 78.83 | 80.60 | 78.50 | 75.40 | 78.80 | 77.50 | 75.56 | 78.00 | 69.50 | 81.60 | 75.75 | 75.50 | 79.00 | 75.80 | 75.45 | 77.67 | 81.33 | 79.64 | 79.50 | 76.91 | 75.00 | 77.67 | 78.46 | 76.94 | 75.86 | 74.00 | 74.57 | 74.20 | 72.91 | 84.60 | 76.62 | 56.27 | 81.33 | 80.00 | 69.11 | 80.91 | 79.83 | 78.17 | 68.86 | 76.00 | 75.78 | 74.53 | 54.50 | 71.23 | 69.33 | 69.71 | 72.33 | 60.33 | 63.09 | 85.20 | 83.33 | 82.91 | 84.57 | 80.00 | 83.00 | 76.83 | 79.50 | 81.09 | 75.78 | 80.55 | 79.20 | 78.50 | 81.64 | 84.00 | 71.83 | 81.27 | 77.20 | 75.17 | 56.67 | 83.14 | 83.50 | 82.15 | 81.11 | 78.22 | 81.29 | 79.14 | 81.43 | 76.75 | 73.75 | 75.25 | 75.00 | 83.57 | 76.31 | 78.62 | 81.09 | 79.00 | 84.31 | 77.71 | 80.73 | 77.73 | 76.75 | 66.82 |
11 | claude-opus-4-1-20250805 | 76.10 | 80.73 | 80.67 | 76.18 | 80.00 | 74.67 | 74.91 | 71.00 | 78.00 | 77.67 | 81.40 | 73.25 | 72.00 | 82.60 | 74.00 | 72.67 | 72.00 | 70.00 | 76.00 | 74.75 | 72.00 | 79.00 | 76.20 | 75.82 | 79.67 | 78.17 | 78.18 | 76.75 | 76.73 | 73.33 | 77.33 | 74.15 | 75.18 | 76.86 | 72.75 | 71.71 | 68.40 | 69.27 | 80.40 | 78.15 | 75.07 | 77.78 | 77.33 | 72.89 | 78.73 | 75.00 | 74.83 | 74.86 | 70.00 | 72.89 | 70.93 | 68.50 | 76.46 | 71.67 | 70.00 | 59.00 | 76.00 | 73.09 | 82.40 | 75.33 | 82.91 | 69.43 | 81.00 | 83.25 | 80.83 | 79.50 | 78.36 | 75.33 | 80.55 | 79.87 | 78.67 | 79.64 | 80.89 | 79.50 | 81.09 | 76.00 | 74.17 | 67.33 | 78.00 | 79.00 | 80.92 | 83.11 | 78.44 | 79.43 | 74.00 | 80.43 | 74.00 | 68.25 | 73.00 | 64.67 | 78.86 | 76.15 | 77.54 | 79.45 | 72.33 | 82.62 | 76.57 | 77.45 | 74.00 | 75.75 | 69.53 |
12 | claude-opus-4-20250514-thinking | 74.75 | 79.82 | 76.67 | 76.00 | 81.00 | 76.00 | 72.73 | 72.50 | 78.67 | 76.50 | 74.40 | 74.25 | 71.20 | 76.80 | 70.17 | 69.11 | 75.25 | 65.25 | 75.60 | 76.25 | 74.00 | 76.20 | 74.60 | 76.00 | 76.67 | 76.83 | 75.82 | 76.00 | 74.18 | 71.17 | 77.00 | 73.69 | 71.65 | 72.14 | 73.25 | 73.71 | 67.90 | 73.27 | 78.40 | 76.62 | 75.20 | 78.67 | 77.78 | 65.11 | 76.36 | 70.00 | 73.67 | 75.71 | 70.29 | 74.89 | 71.07 | 68.75 | 74.00 | 72.33 | 67.14 | 64.00 | 75.00 | 73.64 | 80.80 | 73.67 | 82.36 | 81.14 | 81.00 | 83.25 | 78.83 | 78.00 | 75.82 | 73.33 | 79.09 | 74.00 | 78.17 | 78.36 | 81.33 | 76.50 | 80.36 | 73.00 | 70.83 | 57.33 | 78.29 | 80.00 | 78.77 | 82.00 | 78.22 | 77.71 | 72.57 | 77.43 | 73.25 | 65.50 | 68.50 | 65.33 | 79.71 | 74.46 | 76.92 | 75.45 | 70.00 | 79.38 | 70.86 | 75.27 | 74.40 | 74.25 | 68.82 |
13 | claude-opus-4-20250514 | 74.22 | 79.45 | 77.50 | 69.64 | 74.50 | 76.67 | 73.27 | 71.67 | 77.50 | 74.83 | 76.60 | 70.00 | 71.80 | 70.60 | 70.67 | 68.44 | 66.00 | 67.00 | 73.20 | 70.75 | 71.50 | 75.60 | 74.40 | 73.45 | 76.00 | 76.00 | 76.91 | 76.75 | 76.00 | 71.83 | 74.33 | 72.62 | 72.24 | 72.71 | 71.75 | 67.71 | 66.30 | 70.73 | 78.00 | 76.31 | 76.53 | 78.22 | 77.11 | 71.78 | 76.91 | 71.50 | 73.17 | 74.86 | 65.71 | 68.89 | 70.93 | 73.25 | 74.15 | 70.00 | 68.00 | 65.33 | 74.00 | 74.55 | 81.20 | 74.33 | 82.00 | 78.57 | 79.00 | 82.50 | 78.33 | 73.00 | 76.18 | 72.44 | 78.36 | 75.60 | 77.00 | 78.18 | 78.44 | 76.83 | 80.55 | 75.60 | 67.67 | 67.33 | 80.29 | 78.50 | 78.77 | 80.89 | 78.44 | 78.86 | 70.57 | 78.29 | 72.25 | 68.50 | 67.75 | 63.33 | 77.29 | 74.92 | 75.54 | 79.27 | 72.00 | 78.15 | 73.43 | 72.36 | 74.93 | 76.50 | 68.00 |
14 | claude-sonnet-4-20250514 | 74.13 | 80.36 | 81.50 | 73.82 | 81.50 | 72.00 | 71.09 | 70.50 | 77.67 | 76.67 | 77.80 | 72.00 | 69.40 | 76.20 | 70.00 | 70.22 | 75.00 | 71.00 | 76.80 | 75.25 | 69.00 | 76.60 | 74.20 | 73.82 | 76.33 | 76.67 | 78.18 | 77.50 | 75.82 | 73.33 | 78.00 | 73.08 | 72.00 | 76.00 | 71.25 | 65.71 | 67.20 | 67.64 | 77.60 | 72.31 | 74.40 | 78.00 | 75.11 | 73.56 | 77.09 | 68.83 | 72.00 | 65.14 | 69.14 | 71.56 | 69.73 | 69.75 | 72.77 | 64.17 | 69.43 | 64.33 | 76.33 | 74.55 | 79.20 | 72.67 | 78.00 | 80.57 | 75.50 | 79.75 | 77.33 | 70.50 | 75.09 | 71.56 | 77.45 | 76.00 | 78.17 | 74.55 | 79.56 | 77.33 | 78.73 | 71.00 | 69.50 | 64.67 | 78.29 | 79.00 | 80.62 | 72.00 | 77.33 | 78.00 | 69.71 | 78.71 | 70.25 | 69.50 | 69.25 | 63.00 | 79.86 | 73.69 | 74.77 | 79.09 | 73.00 | 81.54 | 74.29 | 72.36 | 74.40 | 73.50 | 67.18 |
15 | claude-sonnet-4-20250514-thinking | 73.92 | 79.82 | 76.83 | 76.00 | 82.00 | 74.67 | 71.82 | 69.83 | 76.50 | 72.00 | 77.40 | 73.25 | 72.20 | 78.00 | 71.50 | 69.33 | 72.00 | 70.25 | 79.60 | 75.00 | 73.00 | 78.00 | 74.80 | 76.18 | 77.67 | 77.00 | 78.55 | 74.75 | 76.55 | 72.50 | 76.33 | 71.85 | 71.65 | 73.57 | 73.00 | 70.00 | 67.00 | 70.91 | 77.00 | 74.31 | 74.13 | 77.78 | 77.33 | 64.00 | 77.82 | 68.17 | 73.67 | 68.00 | 67.14 | 68.22 | 71.07 | 71.00 | 68.00 | 66.50 | 68.57 | 52.33 | 66.33 | 73.64 | 78.80 | 76.00 | 77.82 | 78.00 | 79.50 | 82.00 | 77.17 | 70.50 | 76.00 | 69.78 | 78.73 | 73.87 | 79.17 | 76.91 | 78.22 | 76.83 | 80.18 | 74.60 | 69.17 | 65.33 | 76.86 | 80.50 | 80.00 | 79.78 | 76.00 | 78.43 | 70.86 | 76.57 | 74.25 | 73.50 | 70.75 | 64.33 | 74.71 | 64.92 | 74.46 | 77.64 | 71.33 | 81.38 | 74.00 | 75.64 | 72.40 | 73.00 | 67.18 |
16 | qwen3-235b-a22b-thinking | 73.12 | 81.09 | 80.00 | 74.91 | 74.50 | 75.33 | 76.91 | 73.17 | 69.33 | 73.33 | 70.60 | 72.50 | 74.40 | 68.00 | 69.17 | 64.00 | 69.25 | 66.00 | 75.20 | 76.50 | 74.00 | 72.60 | 73.20 | 73.09 | 77.00 | 73.33 | 71.82 | 73.25 | 75.27 | 72.83 | 71.67 | 69.38 | 73.88 | 73.29 | 73.25 | 69.71 | 65.20 | 61.82 | 74.60 | 76.92 | 76.40 | 74.22 | 70.67 | 73.56 | 73.27 | 77.00 | 72.50 | 71.43 | 73.14 | 71.11 | 67.60 | 76.25 | 72.92 | 73.50 | 66.86 | 67.33 | 75.33 | 66.73 | 76.00 | 87.00 | 82.00 | 83.14 | 81.00 | 80.25 | 82.17 | 71.50 | 75.27 | 69.78 | 72.00 | 70.00 | 72.33 | 72.73 | 71.33 | 68.17 | 65.82 | 69.40 | 72.67 | 70.67 | 81.14 | 79.75 | 77.54 | 68.89 | 76.89 | 78.86 | 74.57 | 72.43 | 72.75 | 73.50 | 74.75 | 71.67 | 80.29 | 75.38 | 73.23 | 76.73 | 70.67 | 73.85 | 72.57 | 76.00 | 70.67 | 65.75 | 66.82 |
17 | claude-3-7-sonnet-20250219-thinking | 72.36 | 80.00 | 71.33 | 73.45 | 77.00 | 72.67 | 71.64 | 70.00 | 74.83 | 71.50 | 70.40 | 70.00 | 70.60 | 70.20 | 67.00 | 66.89 | 67.00 | 60.50 | 74.00 | 72.25 | 74.00 | 76.80 | 74.40 | 73.64 | 77.33 | 78.50 | 74.55 | 69.25 | 69.27 | 72.83 | 73.67 | 73.08 | 73.41 | 67.86 | 73.50 | 58.29 | 66.60 | 60.00 | 78.20 | 76.00 | 75.73 | 77.78 | 78.44 | 73.33 | 72.36 | 70.33 | 70.67 | 71.43 | 68.29 | 73.56 | 68.27 | 72.75 | 71.08 | 68.83 | 67.71 | 66.00 | 74.33 | 62.36 | 80.40 | 74.00 | 79.27 | 75.14 | 76.00 | 78.50 | 78.83 | 69.00 | 78.55 | 65.33 | 75.27 | 75.33 | 75.83 | 73.64 | 65.56 | 76.17 | 67.64 | 66.00 | 70.00 | 60.00 | 78.00 | 77.00 | 78.77 | 74.67 | 72.89 | 74.86 | 69.43 | 71.29 | 73.75 | 72.75 | 66.75 | 67.33 | 78.00 | 73.85 | 74.77 | 73.27 | 71.00 | 80.77 | 72.57 | 72.18 | 75.20 | 70.00 | 65.41 |
18 | claude-3-7-sonnet-20250219 | 71.70 | 79.45 | 69.83 | 74.36 | 79.75 | 74.00 | 69.27 | 67.83 | 73.33 | 71.33 | 78.20 | 73.50 | 73.20 | 69.00 | 67.00 | 66.00 | 66.75 | 62.00 | 75.20 | 70.25 | 65.50 | 73.80 | 76.00 | 72.00 | 74.50 | 76.50 | 73.45 | 70.75 | 68.73 | 72.50 | 73.67 | 71.23 | 74.47 | 70.29 | 72.75 | 70.00 | 63.80 | 61.45 | 78.40 | 72.92 | 74.67 | 76.22 | 76.89 | 70.22 | 70.00 | 69.50 | 68.50 | 67.71 | 68.00 | 71.56 | 67.20 | 70.75 | 72.77 | 67.33 | 71.14 | 57.67 | 67.33 | 63.27 | 74.40 | 74.00 | 79.27 | 76.57 | 73.50 | 74.75 | 76.83 | 73.00 | 78.55 | 65.33 | 65.45 | 72.67 | 75.17 | 73.82 | 67.11 | 75.83 | 66.36 | 63.60 | 68.17 | 59.33 | 78.57 | 76.75 | 78.62 | 75.56 | 70.22 | 75.86 | 69.14 | 72.14 | 72.50 | 71.50 | 68.75 | 67.67 | 76.86 | 70.77 | 74.62 | 75.45 | 71.33 | 81.23 | 73.71 | 70.55 | 73.60 | 71.25 | 64.35 |
19 | qwq-32b | 71.22 | 77.45 | 74.83 | 73.45 | 75.75 | 78.00 | 75.82 | 69.83 | 69.33 | 69.00 | 66.80 | 72.75 | 71.80 | 67.40 | 67.17 | 65.78 | 65.75 | 68.50 | 74.00 | 72.00 | 68.00 | 71.80 | 74.40 | 73.82 | 72.00 | 72.00 | 72.18 | 70.75 | 68.36 | 70.17 | 68.67 | 69.54 | 70.47 | 69.57 | 69.00 | 62.86 | 65.30 | 60.55 | 75.00 | 77.38 | 76.27 | 75.33 | 75.11 | 70.89 | 71.45 | 74.17 | 70.67 | 69.71 | 69.14 | 69.11 | 69.20 | 68.25 | 70.92 | 70.67 | 65.71 | 66.00 | 61.67 | 63.64 | 73.20 | 82.67 | 70.18 | 77.71 | 78.00 | 74.75 | 76.33 | 67.00 | 72.73 | 72.89 | 66.91 | 72.40 | 72.00 | 62.00 | 66.67 | 65.17 | 70.18 | 60.60 | 67.17 | 62.00 | 80.00 | 77.50 | 75.69 | 71.56 | 78.22 | 76.43 | 76.57 | 71.00 | 71.75 | 73.50 | 70.00 | 70.00 | 80.86 | 75.54 | 73.85 | 73.64 | 71.33 | 75.23 | 74.00 | 74.36 | 70.93 | 68.50 | 65.65 |
20 | chatgpt-4o-latest | 70.97 | 75.64 | 70.83 | 71.64 | 74.00 | 72.00 | 66.55 | 65.17 | 68.50 | 70.50 | 69.60 | 68.00 | 70.40 | 65.40 | 67.83 | 64.67 | 70.00 | 67.00 | 77.60 | 72.25 | 72.00 | 71.20 | 72.40 | 71.45 | 71.33 | 72.83 | 70.55 | 70.25 | 70.91 | 70.00 | 73.33 | 70.15 | 70.71 | 66.71 | 68.25 | 67.71 | 66.70 | 64.73 | 77.20 | 71.38 | 72.40 | 70.44 | 75.11 | 73.11 | 72.18 | 74.17 | 69.33 | 74.29 | 66.86 | 74.22 | 68.93 | 71.00 | 68.00 | 69.83 | 66.86 | 69.00 | 62.67 | 61.27 | 76.80 | 71.67 | 77.64 | 78.86 | 75.50 | 77.25 | 75.50 | 73.50 | 72.55 | 75.56 | 69.64 | 70.67 | 73.67 | 72.18 | 68.67 | 71.17 | 71.27 | 64.40 | 68.17 | 68.67 | 77.43 | 76.75 | 71.08 | 75.11 | 75.78 | 74.00 | 67.43 | 71.43 | 67.50 | 70.25 | 68.00 | 60.33 | 76.86 | 73.69 | 73.69 | 74.91 | 74.00 | 74.62 | 72.57 | 75.45 | 71.73 | 70.00 | 65.41 |
21 | deepseek-r1 | 70.54 | 78.36 | 72.17 | 72.73 | 70.75 | 78.67 | 73.09 | 72.17 | 67.50 | 68.00 | 66.40 | 66.25 | 71.40 | 64.20 | 67.33 | 62.44 | 64.00 | 67.50 | 74.00 | 73.75 | 72.00 | 66.80 | 72.60 | 72.91 | 71.67 | 69.00 | 68.55 | 69.00 | 64.00 | 69.50 | 65.33 | 65.85 | 70.35 | 67.29 | 69.25 | 62.57 | 66.60 | 53.45 | 74.40 | 78.31 | 75.47 | 74.89 | 74.67 | 71.78 | 67.82 | 72.00 | 68.33 | 70.00 | 69.14 | 68.00 | 69.33 | 78.75 | 69.54 | 71.33 | 65.43 | 68.67 | 57.67 | 57.64 | 70.40 | 83.00 | 80.73 | 78.29 | 78.50 | 68.00 | 77.83 | 77.50 | 77.64 | 78.22 | 66.00 | 66.80 | 72.17 | 65.64 | 66.22 | 65.00 | 71.64 | 59.60 | 69.17 | 70.00 | 76.29 | 77.75 | 75.38 | 74.44 | 74.67 | 74.71 | 72.86 | 68.29 | 68.50 | 68.25 | 75.50 | 70.00 | 80.57 | 75.54 | 72.92 | 74.18 | 71.33 | 73.69 | 71.43 | 73.64 | 69.87 | 66.25 | 68.94 |
22 | o4-mini-2025-04-16 | 70.52 | 80.18 | 78.67 | 75.09 | 77.75 | 74.00 | 74.00 | 70.67 | 73.67 | 75.83 | 74.00 | 73.50 | 75.20 | 65.00 | 71.67 | 67.56 | 66.75 | 62.75 | 76.80 | 74.25 | 73.50 | 73.20 | 71.60 | 73.45 | 70.33 | 67.00 | 73.27 | 71.00 | 71.27 | 70.33 | 72.67 | 70.62 | 67.88 | 69.29 | 60.75 | 71.71 | 58.50 | 64.36 | 74.00 | 73.08 | 74.27 | 70.22 | 72.44 | 71.56 | 71.09 | 74.83 | 69.33 | 64.57 | 71.43 | 73.78 | 67.20 | 71.00 | 64.46 | 68.17 | 72.86 | 68.67 | 59.33 | 58.18 | 78.00 | 69.33 | 75.82 | 73.43 | 71.50 | 64.00 | 73.17 | 60.00 | 68.91 | 70.00 | 67.82 | 71.60 | 69.33 | 68.18 | 66.89 | 67.83 | 63.82 | 64.00 | 63.00 | 40.67 | 77.71 | 78.50 | 74.15 | 76.00 | 72.00 | 75.43 | 72.86 | 71.29 | 75.50 | 69.75 | 66.75 | 71.33 | 74.86 | 71.69 | 71.85 | 74.18 | 72.00 | 72.92 | 71.43 | 75.27 | 70.27 | 69.75 | 64.71 |
23 | qwen3-235b-a22b-nothinking | 70.47 | 76.18 | 73.50 | 72.36 | 76.00 | 72.67 | 68.00 | 66.17 | 70.50 | 70.33 | 69.00 | 62.50 | 66.20 | 65.80 | 68.50 | 63.78 | 67.00 | 64.75 | 76.00 | 71.00 | 68.50 | 71.20 | 73.00 | 71.64 | 71.33 | 70.33 | 70.36 | 72.75 | 73.64 | 68.17 | 71.33 | 70.77 | 70.59 | 67.86 | 69.75 | 69.43 | 65.00 | 63.64 | 73.80 | 70.77 | 69.73 | 75.11 | 75.33 | 71.33 | 72.91 | 71.83 | 71.00 | 72.29 | 72.29 | 72.44 | 68.80 | 68.50 | 67.85 | 67.00 | 63.71 | 67.00 | 64.00 | 62.00 | 75.60 | 66.00 | 73.27 | 76.86 | 71.50 | 75.75 | 75.83 | 80.00 | 73.09 | 71.33 | 70.91 | 72.27 | 71.50 | 72.73 | 68.67 | 70.00 | 70.00 | 67.60 | 69.17 | 66.67 | 75.71 | 77.00 | 75.38 | 68.22 | 71.56 | 75.00 | 68.86 | 73.14 | 71.75 | 69.75 | 66.00 | 70.00 | 74.57 | 73.08 | 70.00 | 73.64 | 70.33 | 74.77 | 68.29 | 71.09 | 71.07 | 67.50 | 62.24 |
24 | grok-3 | 69.80 | 68.55 | 73.00 | 67.27 | 75.25 | 47.33 | 71.27 | 67.67 | 71.00 | 70.17 | 65.40 | 70.00 | 67.00 | 71.00 | 68.00 | 62.00 | 66.00 | 68.25 | 69.60 | 71.25 | 67.00 | 73.00 | 72.40 | 66.73 | 67.00 | 74.00 | 69.09 | 70.50 | 68.55 | 69.83 | 70.33 | 72.15 | 70.94 | 64.43 | 67.00 | 68.57 | 65.90 | 60.91 | 73.40 | 72.46 | 72.40 | 75.33 | 76.89 | 73.78 | 76.18 | 76.17 | 71.00 | 74.29 | 67.43 | 68.00 | 70.40 | 61.50 | 68.77 | 67.67 | 66.00 | 67.33 | 72.67 | 64.36 | 64.00 | 69.33 | 79.82 | 80.29 | 58.00 | 78.25 | 76.83 | 77.00 | 66.55 | 65.33 | 68.73 | 73.87 | 67.00 | 63.45 | 71.11 | 74.00 | 72.18 | 62.00 | 66.33 | 64.67 | 67.43 | 70.75 | 78.77 | 75.78 | 73.33 | 75.71 | 73.14 | 71.71 | 72.25 | 67.75 | 65.75 | 69.00 | 75.14 | 63.69 | 69.23 | 71.45 | 73.33 | 70.77 | 58.86 | 69.64 | 68.67 | 63.75 | 62.94 |
25 | gpt-4.1-2025-04-14 | 69.00 | 73.09 | 66.67 | 71.64 | 73.25 | 67.33 | 67.09 | 67.17 | 69.83 | 70.17 | 69.80 | 69.50 | 67.00 | 62.40 | 70.50 | 63.11 | 65.25 | 63.25 | 77.20 | 69.50 | 72.00 | 69.40 | 69.60 | 68.73 | 67.33 | 73.00 | 68.91 | 68.25 | 69.64 | 68.00 | 67.33 | 67.54 | 67.76 | 59.86 | 70.25 | 61.43 | 64.00 | 60.18 | 75.20 | 73.08 | 71.87 | 71.11 | 73.33 | 71.11 | 66.55 | 70.17 | 67.17 | 69.14 | 64.29 | 71.78 | 67.60 | 66.00 | 70.00 | 69.00 | 69.14 | 68.67 | 58.67 | 61.64 | 70.80 | 66.67 | 76.36 | 73.43 | 75.00 | 74.75 | 67.83 | 71.50 | 71.27 | 73.33 | 67.45 | 69.47 | 69.33 | 69.64 | 70.67 | 66.67 | 69.09 | 62.60 | 68.33 | 72.00 | 73.14 | 74.50 | 71.69 | 74.00 | 73.78 | 73.86 | 64.57 | 68.86 | 69.00 | 62.50 | 65.00 | 66.33 | 72.29 | 71.54 | 72.92 | 72.18 | 71.00 | 75.54 | 70.00 | 68.36 | 70.40 | 71.50 | 60.47 |
26 | deepseek-v3-250324 | 67.74 | 74.55 | 71.00 | 70.00 | 68.75 | 70.00 | 70.55 | 71.50 | 68.67 | 63.50 | 60.80 | 66.50 | 69.40 | 59.20 | 65.00 | 62.22 | 61.00 | 66.75 | 72.40 | 70.00 | 69.00 | 67.40 | 68.20 | 67.45 | 67.33 | 65.83 | 65.09 | 63.75 | 64.91 | 65.50 | 61.33 | 63.23 | 68.71 | 65.57 | 68.25 | 62.29 | 61.40 | 50.91 | 72.40 | 72.62 | 73.07 | 70.00 | 74.00 | 71.78 | 66.55 | 72.83 | 70.00 | 67.71 | 64.57 | 60.89 | 69.60 | 60.25 | 64.15 | 66.00 | 58.57 | 60.33 | 57.33 | 56.55 | 78.00 | 77.00 | 76.00 | 77.43 | 74.50 | 74.00 | 72.83 | 75.00 | 73.09 | 77.78 | 63.09 | 68.53 | 69.00 | 55.82 | 65.56 | 65.67 | 58.36 | 62.20 | 65.50 | 75.33 | 70.29 | 74.50 | 72.77 | 70.89 | 73.11 | 70.57 | 72.57 | 65.57 | 66.75 | 61.25 | 69.75 | 67.33 | 74.57 | 74.15 | 69.69 | 70.00 | 68.33 | 71.54 | 71.14 | 72.18 | 69.20 | 66.75 | 65.41 |
27 | o3-mini | 64.88 | 68.73 | 70.67 | 67.64 | 73.25 | 62.67 | 64.55 | 64.17 | 66.17 | 65.50 | 66.00 | 65.50 | 63.40 | 62.60 | 60.17 | 61.78 | 60.50 | 58.50 | 74.00 | 65.50 | 61.50 | 68.00 | 69.00 | 64.73 | 66.83 | 69.67 | 67.09 | 66.75 | 69.64 | 63.33 | 64.33 | 66.77 | 63.76 | 60.14 | 60.00 | 62.29 | 58.70 | 58.55 | 72.20 | 64.62 | 68.80 | 67.33 | 63.78 | 67.33 | 69.09 | 63.00 | 62.33 | 63.43 | 62.57 | 64.22 | 62.53 | 61.50 | 66.00 | 62.67 | 57.14 | 62.67 | 55.67 | 60.55 | 67.20 | 66.00 | 66.91 | 70.86 | 66.50 | 67.25 | 64.67 | 63.00 | 64.91 | 63.78 | 62.18 | 67.20 | 64.33 | 68.36 | 69.56 | 66.00 | 59.45 | 54.00 | 53.17 | 35.33 | 71.14 | 73.00 | 72.77 | 73.78 | 68.00 | 75.00 | 65.71 | 68.71 | 69.00 | 69.00 | 58.25 | 60.33 | 65.71 | 65.69 | 65.69 | 69.64 | 64.67 | 64.46 | 62.86 | 66.91 | 59.87 | 60.25 | 58.59 |
28 | qwen-max | 64.68 | 70.00 | 67.83 | 67.45 | 68.25 | 68.67 | 66.91 | 67.50 | 66.83 | 62.50 | 60.80 | 61.50 | 60.80 | 58.20 | 59.33 | 60.67 | 55.50 | 63.50 | 66.80 | 65.25 | 66.00 | 63.80 | 64.20 | 64.18 | 63.33 | 64.33 | 64.91 | 63.50 | 60.73 | 63.50 | 62.67 | 61.54 | 63.88 | 61.00 | 61.00 | 64.86 | 58.70 | 51.27 | 65.20 | 67.23 | 67.73 | 67.78 | 65.33 | 69.33 | 63.82 | 66.67 | 61.83 | 67.71 | 63.43 | 62.44 | 61.87 | 65.75 | 59.08 | 61.17 | 61.71 | 58.33 | 58.33 | 50.73 | 73.20 | 68.33 | 68.91 | 73.14 | 68.50 | 73.25 | 66.50 | 72.50 | 68.55 | 70.89 | 66.18 | 64.53 | 65.83 | 66.00 | 64.00 | 60.50 | 64.00 | 63.80 | 57.00 | 69.33 | 71.14 | 72.50 | 70.92 | 69.33 | 71.33 | 66.00 | 68.57 | 63.71 | 63.50 | 66.50 | 64.25 | 63.33 | 72.71 | 71.23 | 70.00 | 68.73 | 66.67 | 64.00 | 64.57 | 66.00 | 65.87 | 63.25 | 60.94 |
29 | gpt-4.5-preview | 64.31 | 70.91 | 66.67 | 66.36 | 64.25 | 69.33 | 67.27 | 67.50 | 61.17 | 62.00 | 62.60 | 61.00 | 63.40 | 59.20 | 62.17 | 61.33 | 56.50 | 62.00 | 75.20 | 68.50 | 66.50 | 62.40 | 65.20 | 64.73 | 61.83 | 61.17 | 56.91 | 63.75 | 59.82 | 62.83 | 60.33 | 62.15 | 62.82 | 60.43 | 60.50 | 62.57 | 59.90 | 46.00 | 63.00 | 70.15 | 69.73 | 70.22 | 70.44 | 68.22 | 63.09 | 67.50 | 61.83 | 73.14 | 67.43 | 65.33 | 62.67 | 68.00 | 62.00 | 63.17 | 62.29 | 60.67 | 56.67 | 49.82 | 70.00 | 65.67 | 72.91 | 70.00 | 71.50 | 69.00 | 68.67 | 71.00 | 69.27 | 67.11 | 64.91 | 64.80 | 60.50 | 59.82 | 50.67 | 65.00 | 57.45 | 53.60 | 60.33 | 64.00 | 70.57 | 71.25 | 70.92 | 69.11 | 70.22 | 67.14 | 64.86 | 62.00 | 64.25 | 70.75 | 61.75 | 63.00 | 70.71 | 69.85 | 68.46 | 67.27 | 66.33 | 66.92 | 70.86 | 68.00 | 64.67 | 62.25 | 60.47 |
30 | gemini-2.0-flash | 64.14 | 70.73 | 74.33 | 69.27 | 63.75 | 66.00 | 58.91 | 62.17 | 64.17 | 64.17 | 63.60 | 65.75 | 58.80 | 62.40 | 61.50 | 56.44 | 57.25 | 60.75 | 70.80 | 62.50 | 67.00 | 69.40 | 66.60 | 61.64 | 64.83 | 62.50 | 62.73 | 61.25 | 66.73 | 62.33 | 59.67 | 70.46 | 59.18 | 60.71 | 56.25 | 58.00 | 57.80 | 54.91 | 69.60 | 65.08 | 69.07 | 68.00 | 68.89 | 66.89 | 69.82 | 64.17 | 68.83 | 64.00 | 60.29 | 55.11 | 64.67 | 56.25 | 63.38 | 59.50 | 53.14 | 60.33 | 59.67 | 54.73 | 70.00 | 65.00 | 68.91 | 65.14 | 64.50 | 69.25 | 63.50 | 64.50 | 68.36 | 66.00 | 65.27 | 64.53 | 64.00 | 64.36 | 63.11 | 60.83 | 65.82 | 61.20 | 56.17 | 66.67 | 65.14 | 74.00 | 69.38 | 73.78 | 68.67 | 69.00 | 64.57 | 64.71 | 65.25 | 72.00 | 60.25 | 53.67 | 64.29 | 66.46 | 67.38 | 67.09 | 63.33 | 65.54 | 67.14 | 64.73 | 64.00 | 66.75 | 62.71 |
31 | o1-preview-2024-09-12 | 61.46 | 68.00 | 57.83 | 64.73 | 69.00 | 48.00 | 64.18 | 65.33 | 68.17 | 54.83 | 61.60 | 61.00 | 59.00 | 54.00 | 58.33 | 58.89 | 60.75 | 56.25 | 70.40 | 63.25 | 62.50 | 63.80 | 63.00 | 62.55 | 58.67 | 61.83 | 63.27 | 59.75 | 64.18 | 58.67 | 60.00 | 58.31 | 49.53 | 56.00 | 56.00 | 61.43 | 54.90 | 50.18 | 65.40 | 65.38 | 64.40 | 65.33 | 64.00 | 63.33 | 57.45 | 54.17 | 62.17 | 70.57 | 55.14 | 57.33 | 61.33 | 61.25 | 58.31 | 57.17 | 56.57 | 56.33 | 66.33 | 51.09 | 65.60 | 62.67 | 67.09 | 65.43 | 67.00 | 67.25 | 57.17 | 61.00 | 62.00 | 66.00 | 62.55 | 67.47 | 60.83 | 62.73 | 59.56 | 60.67 | 62.91 | 55.40 | 57.17 | 62.00 | 69.14 | 70.50 | 70.15 | 62.00 | 65.78 | 67.43 | 66.29 | 61.86 | 61.00 | 61.50 | 58.25 | 49.00 | 66.14 | 69.85 | 64.31 | 62.18 | 62.67 | 68.00 | 66.57 | 62.73 | 60.80 | 62.00 | 58.00 |
32 | deepseek-v3 | 56.32 | 65.27 | 60.17 | 61.64 | 58.75 | 64.00 | 59.27 | 61.17 | 57.17 | 54.17 | 50.80 | 51.75 | 55.60 | 52.60 | 54.00 | 53.11 | 48.50 | 52.00 | 61.20 | 58.50 | 63.50 | 58.40 | 57.60 | 58.00 | 54.00 | 52.33 | 54.00 | 54.50 | 51.82 | 57.33 | 48.33 | 49.85 | 55.06 | 54.57 | 49.50 | 56.00 | 47.90 | 45.09 | 53.40 | 61.85 | 59.33 | 58.00 | 59.33 | 61.56 | 49.27 | 57.67 | 55.17 | 60.86 | 59.43 | 53.56 | 57.20 | 56.00 | 53.23 | 53.50 | 49.71 | 57.67 | 43.00 | 46.00 | 64.80 | 61.67 | 58.91 | 62.86 | 66.50 | 55.75 | 53.83 | 60.00 | 60.91 | 61.33 | 58.00 | 56.13 | 55.33 | 54.91 | 46.67 | 50.67 | 50.36 | 49.40 | 48.50 | 54.00 | 62.86 | 68.25 | 63.38 | 60.89 | 60.22 | 57.29 | 60.29 | 52.71 | 57.50 | 62.75 | 58.75 | 54.67 | 59.14 | 66.15 | 64.92 | 57.09 | 63.00 | 57.08 | 59.14 | 58.55 | 57.07 | 58.50 | 58.94 |
33 | gemini-1.5-pro | 55.94 | 63.64 | 62.50 | 62.36 | 55.00 | 50.00 | 56.55 | 61.83 | 54.83 | 49.33 | 48.60 | 53.00 | 55.40 | 50.80 | 54.33 | 57.33 | 40.25 | 53.50 | 69.20 | 60.00 | 58.00 | 58.40 | 54.60 | 53.09 | 52.00 | 52.83 | 49.82 | 47.75 | 50.00 | 53.17 | 47.33 | 52.31 | 51.06 | 48.43 | 43.00 | 49.14 | 49.80 | 37.82 | 59.00 | 55.08 | 58.00 | 61.11 | 60.00 | 55.56 | 52.36 | 57.33 | 56.17 | 57.43 | 58.00 | 48.00 | 55.20 | 54.25 | 55.54 | 49.17 | 55.14 | 45.33 | 58.67 | 43.64 | 62.80 | 68.33 | 66.55 | 59.14 | 61.00 | 63.75 | 64.33 | 63.50 | 62.00 | 63.78 | 62.36 | 55.20 | 51.00 | 60.36 | 49.11 | 52.67 | 51.09 | 56.60 | 53.00 | 59.33 | 60.29 | 66.75 | 59.69 | 59.56 | 59.33 | 59.86 | 65.71 | 56.57 | 58.00 | 58.75 | 56.00 | 53.00 | 64.57 | 68.15 | 59.69 | 59.64 | 62.67 | 54.92 | 62.57 | 59.27 | 55.73 | 50.00 | 59.18 |
34 | qwen-2.5-72b-instruct | 54.19 | 63.09 | 56.17 | 58.00 | 62.00 | 51.33 | 58.36 | 56.00 | 53.00 | 55.83 | 53.00 | 55.00 | 56.00 | 56.80 | 53.17 | 53.56 | 44.75 | 51.50 | 60.00 | 60.50 | 60.00 | 56.60 | 53.00 | 50.73 | 53.83 | 54.17 | 52.36 | 56.50 | 53.45 | 52.33 | 50.00 | 51.54 | 52.82 | 50.71 | 49.50 | 47.71 | 45.20 | 46.18 | 54.80 | 59.08 | 60.67 | 57.56 | 54.44 | 60.44 | 50.36 | 50.83 | 57.17 | 58.00 | 55.14 | 55.56 | 52.67 | 54.75 | 49.69 | 52.00 | 45.43 | 46.67 | 52.33 | 52.36 | 60.80 | 58.00 | 52.00 | 52.57 | 58.00 | 52.25 | 54.67 | 52.50 | 54.91 | 56.44 | 51.45 | 53.20 | 50.17 | 48.00 | 51.56 | 45.83 | 51.09 | 50.20 | 43.00 | 48.67 | 63.14 | 62.50 | 64.31 | 59.56 | 61.56 | 58.86 | 59.71 | 53.43 | 57.50 | 62.25 | 53.50 | 55.33 | 55.29 | 58.62 | 58.92 | 54.55 | 52.67 | 54.92 | 56.57 | 51.64 | 51.73 | 56.75 | 54.24 |
35 | claude-3-5-sonnet-20241022 | 53.25 | 61.82 | 52.83 | 61.64 | 50.50 | 64.67 | 58.91 | 60.00 | 54.83 | 49.50 | 52.40 | 45.00 | 50.80 | 46.00 | 40.17 | 46.67 | 45.25 | 44.75 | 59.20 | 62.25 | 59.50 | 56.20 | 52.20 | 56.73 | 50.50 | 49.00 | 51.45 | 46.00 | 45.45 | 53.33 | 46.00 | 53.85 | 47.06 | 51.43 | 46.00 | 46.86 | 51.10 | 40.91 | 48.60 | 52.00 | 52.67 | 54.89 | 54.00 | 54.89 | 53.09 | 54.17 | 50.50 | 55.71 | 57.43 | 46.00 | 52.13 | 57.75 | 43.69 | 48.33 | 50.57 | 47.33 | 55.33 | 39.09 | 58.00 | 64.00 | 63.09 | 52.00 | 64.00 | 61.00 | 58.00 | 54.00 | 59.09 | 65.11 | 61.09 | 50.00 | 42.17 | 53.64 | 37.11 | 46.50 | 48.36 | 48.60 | 47.50 | 57.33 | 58.86 | 57.00 | 58.15 | 54.67 | 58.89 | 55.14 | 66.00 | 53.00 | 53.75 | 49.50 | 52.75 | 46.67 | 64.57 | 68.15 | 65.54 | 54.18 | 63.67 | 55.38 | 66.57 | 58.55 | 55.33 | 51.50 | 58.47 |
36 | gpt-4o-mini | 52.36 | 62.36 | 53.83 | 59.64 | 52.25 | 62.00 | 58.91 | 57.83 | 51.50 | 51.67 | 49.80 | 50.25 | 51.80 | 48.00 | 50.50 | 50.00 | 42.75 | 46.00 | 54.80 | 56.75 | 58.50 | 53.00 | 51.40 | 53.45 | 49.83 | 50.17 | 50.18 | 50.75 | 50.00 | 52.33 | 49.00 | 48.00 | 49.18 | 45.00 | 47.50 | 47.71 | 47.60 | 39.82 | 50.40 | 55.85 | 54.80 | 52.89 | 52.00 | 56.89 | 47.64 | 48.67 | 48.67 | 50.00 | 46.57 | 51.11 | 54.27 | 54.00 | 47.69 | 47.00 | 49.71 | 48.33 | 37.67 | 44.55 | 58.00 | 62.00 | 58.00 | 55.43 | 55.50 | 57.75 | 55.33 | 53.00 | 57.27 | 56.89 | 52.36 | 50.80 | 52.17 | 51.45 | 46.67 | 48.00 | 48.18 | 48.80 | 43.17 | 48.00 | 63.43 | 58.25 | 54.92 | 59.33 | 55.33 | 52.86 | 60.86 | 50.43 | 54.25 | 55.25 | 54.00 | 53.00 | 56.14 | 62.15 | 60.92 | 58.91 | 58.67 | 49.69 | 60.57 | 53.27 | 55.33 | 56.00 | 54.00 |
37 | mistral-large-instruct-2411 | 50.86 | 59.64 | 52.33 | 58.73 | 52.75 | 61.33 | 56.73 | 56.00 | 55.17 | 46.17 | 49.40 | 51.25 | 48.80 | 50.00 | 48.83 | 45.78 | 42.25 | 43.00 | 56.40 | 53.75 | 56.00 | 52.20 | 50.80 | 50.73 | 51.00 | 52.33 | 51.09 | 52.00 | 52.73 | 49.67 | 49.00 | 48.92 | 47.88 | 48.00 | 46.50 | 48.00 | 45.50 | 40.55 | 49.40 | 55.23 | 54.40 | 52.22 | 50.89 | 55.11 | 46.91 | 50.67 | 52.17 | 53.43 | 50.00 | 48.00 | 51.60 | 51.50 | 48.15 | 46.33 | 46.29 | 48.33 | 49.67 | 45.82 | 52.40 | 57.67 | 49.82 | 51.71 | 54.00 | 47.00 | 52.00 | 46.50 | 50.91 | 50.67 | 48.36 | 50.40 | 46.67 | 50.55 | 45.78 | 42.00 | 47.27 | 45.20 | 42.17 | 46.00 | 60.86 | 56.00 | 57.08 | 57.56 | 51.33 | 51.29 | 59.71 | 49.00 | 51.25 | 61.50 | 52.25 | 46.00 | 48.57 | 58.77 | 61.69 | 51.45 | 57.67 | 47.54 | 56.29 | 50.73 | 49.60 | 53.00 | 54.12 |
38 | llama-4-maverick-17b-128e | 48.68 | 57.82 | 51.83 | 50.00 | 48.50 | 61.33 | 51.45 | 50.33 | 46.83 | 44.83 | 44.40 | 43.75 | 47.00 | 45.60 | 42.33 | 42.89 | 38.75 | 44.50 | 56.80 | 56.50 | 59.50 | 47.80 | 48.00 | 51.27 | 44.83 | 43.17 | 42.91 | 45.25 | 46.55 | 47.00 | 42.67 | 48.62 | 47.41 | 45.57 | 43.00 | 45.43 | 44.80 | 39.82 | 47.60 | 46.77 | 50.93 | 50.44 | 48.44 | 51.33 | 42.73 | 48.50 | 48.00 | 51.43 | 58.86 | 46.44 | 48.67 | 54.25 | 44.15 | 45.00 | 46.00 | 43.00 | 51.00 | 44.55 | 51.60 | 57.67 | 52.55 | 48.00 | 49.50 | 48.50 | 51.00 | 49.00 | 54.36 | 51.33 | 49.64 | 46.00 | 43.83 | 46.55 | 40.00 | 46.17 | 42.36 | 44.20 | 45.33 | 50.67 | 51.43 | 50.50 | 56.77 | 51.56 | 51.78 | 50.29 | 56.00 | 46.29 | 51.50 | 50.25 | 49.50 | 48.00 | 52.00 | 61.54 | 57.08 | 51.27 | 57.67 | 51.08 | 56.00 | 49.27 | 50.13 | 53.25 | 52.59 |
39 | llama-4-scout-17b-16e-instruct | 47.84 | 57.64 | 52.33 | 52.36 | 46.50 | 56.67 | 54.55 | 51.67 | 43.67 | 44.83 | 48.00 | 44.50 | 44.80 | 44.00 | 43.67 | 44.89 | 41.00 | 43.50 | 60.40 | 53.25 | 50.00 | 51.40 | 47.80 | 49.45 | 45.50 | 45.67 | 47.27 | 45.25 | 47.27 | 48.83 | 46.67 | 48.15 | 46.00 | 47.00 | 41.00 | 45.14 | 46.20 | 37.64 | 41.60 | 49.38 | 53.07 | 50.67 | 48.00 | 51.78 | 46.00 | 47.33 | 46.50 | 44.86 | 52.29 | 48.22 | 48.93 | 50.00 | 46.62 | 43.67 | 44.86 | 46.33 | 48.00 | 41.45 | 51.60 | 51.00 | 48.00 | 47.14 | 49.50 | 46.75 | 45.17 | 50.50 | 47.45 | 42.67 | 48.55 | 47.87 | 43.50 | 46.55 | 40.67 | 44.17 | 43.45 | 42.40 | 42.00 | 44.67 | 52.00 | 49.50 | 52.15 | 52.44 | 48.00 | 51.00 | 59.43 | 44.57 | 52.75 | 47.75 | 50.25 | 45.00 | 47.43 | 56.77 | 57.23 | 48.00 | 53.33 | 42.92 | 54.86 | 48.91 | 50.53 | 50.50 | 50.12 |
40 | claude-3-5-haiku-20241022 | 47.57 | 59.45 | 54.50 | 55.27 | 44.50 | 60.00 | 53.09 | 57.17 | 43.83 | 43.33 | 42.40 | 42.00 | 45.60 | 42.00 | 42.33 | 43.78 | 39.25 | 42.50 | 56.00 | 58.25 | 58.50 | 48.40 | 46.80 | 48.91 | 44.83 | 43.67 | 44.18 | 42.50 | 42.36 | 50.17 | 45.00 | 48.92 | 44.35 | 47.14 | 39.25 | 41.43 | 44.60 | 36.73 | 44.20 | 47.69 | 46.93 | 48.00 | 47.11 | 52.44 | 46.55 | 45.00 | 45.50 | 47.71 | 44.86 | 40.00 | 48.00 | 51.50 | 40.92 | 43.33 | 43.71 | 46.33 | 51.67 | 36.73 | 48.40 | 64.00 | 51.45 | 48.00 | 57.00 | 43.25 | 52.17 | 54.50 | 48.73 | 58.22 | 47.64 | 42.27 | 44.50 | 44.36 | 35.56 | 36.50 | 41.64 | 40.80 | 41.00 | 42.67 | 48.86 | 50.00 | 49.85 | 48.44 | 50.89 | 42.86 | 56.29 | 47.71 | 46.00 | 47.75 | 57.50 | 38.67 | 55.14 | 64.92 | 59.08 | 47.45 | 60.67 | 46.62 | 65.14 | 50.73 | 49.60 | 49.00 | 54.94 |
41 | qwen-2.5-7b-instruct | 45.33 | 58.55 | 51.83 | 52.18 | 47.25 | 50.00 | 53.09 | 52.17 | 45.00 | 46.00 | 41.40 | 43.75 | 47.60 | 45.80 | 42.33 | 46.44 | 39.75 | 39.50 | 48.80 | 48.00 | 55.00 | 53.60 | 46.60 | 48.91 | 43.83 | 50.50 | 45.45 | 43.75 | 43.82 | 45.83 | 44.67 | 42.46 | 43.29 | 43.29 | 45.75 | 40.00 | 42.00 | 38.36 | 47.80 | 52.00 | 49.87 | 47.33 | 40.22 | 49.11 | 42.91 | 44.83 | 43.33 | 39.43 | 48.86 | 44.67 | 47.73 | 44.75 | 44.46 | 42.33 | 48.00 | 42.67 | 42.67 | 40.55 | 51.60 | 57.67 | 33.82 | 41.71 | 45.00 | 33.00 | 38.17 | 43.00 | 38.55 | 56.00 | 40.73 | 39.47 | 40.00 | 38.36 | 33.78 | 29.33 | 38.91 | 40.80 | 38.50 | 41.33 | 57.14 | 52.75 | 48.00 | 51.33 | 50.22 | 48.43 | 56.57 | 45.57 | 51.25 | 56.50 | 44.75 | 46.00 | 44.14 | 53.23 | 51.69 | 44.36 | 51.67 | 40.62 | 47.71 | 45.45 | 43.33 | 46.00 | 45.88 |
42 | llama-3.3-70b-instruct | 43.97 | 52.00 | 50.00 | 47.27 | 40.50 | 46.67 | 50.55 | 49.00 | 41.17 | 39.67 | 40.00 | 42.75 | 38.40 | 40.00 | 36.83 | 45.33 | 35.50 | 40.00 | 44.80 | 52.50 | 53.00 | 44.40 | 42.40 | 47.45 | 44.33 | 42.17 | 41.45 | 41.00 | 43.45 | 44.83 | 45.67 | 44.31 | 39.41 | 42.86 | 38.25 | 43.71 | 43.80 | 35.82 | 40.20 | 42.00 | 44.93 | 42.89 | 45.56 | 47.56 | 44.18 | 41.33 | 36.00 | 42.29 | 45.71 | 41.56 | 46.27 | 44.75 | 40.00 | 39.50 | 44.29 | 46.67 | 39.00 | 33.27 | 44.40 | 56.67 | 46.55 | 42.00 | 48.00 | 46.25 | 40.50 | 46.50 | 44.36 | 35.11 | 45.27 | 43.07 | 40.17 | 43.09 | 37.78 | 42.67 | 38.73 | 42.20 | 34.67 | 35.33 | 50.29 | 43.75 | 48.62 | 51.56 | 49.33 | 42.57 | 53.43 | 44.00 | 49.00 | 44.50 | 46.75 | 44.00 | 44.14 | 54.92 | 52.46 | 46.55 | 53.67 | 41.85 | 51.71 | 46.55 | 46.40 | 47.25 | 50.82 |
43 | qwen3-0.6b-thinking | 42.00 | 52.55 | 55.00 | 53.82 | 44.75 | 50.67 | 54.55 | 53.50 | 37.67 | 43.83 | 38.20 | 45.00 | 47.40 | 43.80 | 42.67 | 41.78 | 40.50 | 43.25 | 43.60 | 44.25 | 50.50 | 44.00 | 48.80 | 47.45 | 46.33 | 43.83 | 46.36 | 50.25 | 44.00 | 42.33 | 42.00 | 35.85 | 39.88 | 41.57 | 44.00 | 38.86 | 39.30 | 37.27 | 39.20 | 52.15 | 50.80 | 40.22 | 36.00 | 45.33 | 36.36 | 38.67 | 40.17 | 33.14 | 35.43 | 45.56 | 43.87 | 45.50 | 43.54 | 45.33 | 41.71 | 38.00 | 38.33 | 42.73 | 40.80 | 55.67 | 38.36 | 38.00 | 40.00 | 30.00 | 38.33 | 37.50 | 34.55 | 49.78 | 31.09 | 37.07 | 34.50 | 31.45 | 32.00 | 28.33 | 27.09 | 37.80 | 34.83 | 42.67 | 55.43 | 40.75 | 45.08 | 43.56 | 47.33 | 37.43 | 53.43 | 43.00 | 44.00 | 44.50 | 47.75 | 37.67 | 44.14 | 42.46 | 41.85 | 38.36 | 44.67 | 30.77 | 43.14 | 44.00 | 41.07 | 40.00 | 39.88 |
44 | qwen3-0.6b-nothinking | 38.53 | 46.91 | 49.67 | 46.36 | 40.25 | 48.00 | 47.45 | 45.67 | 32.83 | 41.67 | 36.20 | 42.00 | 43.80 | 42.00 | 42.50 | 38.00 | 36.75 | 42.75 | 46.80 | 43.50 | 51.50 | 40.60 | 41.60 | 45.64 | 40.50 | 41.17 | 39.82 | 48.00 | 38.55 | 40.83 | 41.00 | 33.23 | 36.82 | 37.14 | 39.25 | 33.71 | 34.70 | 35.27 | 38.40 | 45.23 | 44.93 | 37.78 | 34.44 | 45.33 | 32.55 | 35.50 | 39.17 | 30.57 | 34.57 | 40.89 | 39.73 | 39.25 | 38.00 | 43.33 | 39.14 | 32.67 | 34.33 | 40.36 | 36.00 | 40.00 | 33.64 | 36.00 | 39.00 | 28.25 | 34.17 | 33.00 | 31.09 | 38.44 | 29.64 | 34.13 | 31.67 | 31.45 | 31.78 | 27.33 | 28.18 | 31.20 | 32.50 | 36.67 | 51.43 | 37.75 | 39.85 | 36.44 | 38.44 | 35.86 | 42.57 | 39.00 | 42.50 | 41.00 | 40.25 | 38.67 | 38.14 | 42.31 | 42.92 | 35.64 | 39.67 | 29.85 | 39.71 | 40.18 | 38.27 | 40.00 | 39.53 |
45 | llama-3.1-8b-instruct | 36.52 | 46.91 | 45.17 | 41.45 | 37.25 | 40.67 | 43.45 | 42.00 | 33.17 | 36.33 | 34.20 | 33.75 | 36.40 | 35.00 | 32.67 | 37.78 | 31.25 | 29.00 | 39.20 | 42.25 | 49.00 | 43.60 | 37.80 | 39.45 | 36.67 | 34.33 | 38.91 | 36.50 | 37.09 | 38.00 | 38.33 | 35.23 | 31.53 | 35.71 | 32.00 | 39.43 | 35.20 | 29.82 | 35.40 | 38.15 | 33.07 | 36.89 | 34.00 | 38.22 | 32.91 | 32.83 | 36.17 | 32.00 | 30.29 | 35.33 | 33.87 | 28.75 | 34.00 | 31.67 | 36.57 | 30.67 | 35.00 | 29.09 | 36.00 | 42.33 | 30.91 | 35.14 | 34.50 | 32.75 | 29.83 | 36.50 | 30.55 | 37.78 | 38.00 | 36.93 | 35.00 | 32.73 | 32.00 | 31.83 | 30.36 | 34.20 | 26.33 | 32.67 | 38.29 | 38.25 | 34.31 | 40.89 | 39.56 | 37.29 | 44.57 | 39.14 | 43.25 | 43.50 | 40.50 | 35.33 | 37.57 | 46.77 | 45.23 | 35.45 | 47.00 | 33.38 | 46.57 | 38.91 | 37.73 | 45.00 | 44.12 |
46 | llama-3.2-3b-instruct | 34.52 | 44.36 | 41.83 | 44.91 | 29.50 | 34.67 | 42.00 | 39.33 | 29.83 | 33.83 | 33.60 | 32.50 | 33.80 | 32.20 | 33.50 | 39.33 | 30.00 | 32.75 | 34.00 | 40.50 | 42.50 | 38.80 | 32.80 | 38.36 | 36.33 | 34.33 | 36.36 | 35.75 | 36.18 | 37.67 | 37.67 | 30.62 | 30.82 | 33.57 | 30.75 | 37.71 | 35.50 | 31.09 | 29.00 | 31.85 | 33.07 | 29.56 | 33.11 | 38.67 | 29.45 | 26.83 | 37.50 | 30.86 | 27.14 | 32.00 | 33.60 | 27.75 | 30.92 | 28.83 | 40.00 | 34.67 | 34.67 | 28.73 | 33.20 | 32.00 | 31.09 | 31.71 | 31.50 | 30.00 | 31.50 | 36.00 | 25.27 | 32.89 | 37.64 | 32.80 | 31.33 | 30.91 | 30.00 | 30.33 | 30.36 | 31.60 | 26.17 | 23.33 | 44.57 | 34.75 | 35.54 | 43.11 | 37.11 | 31.29 | 44.00 | 37.00 | 39.75 | 42.25 | 35.50 | 36.33 | 33.86 | 39.23 | 45.85 | 34.36 | 47.00 | 27.38 | 44.29 | 35.09 | 36.40 | 40.00 | 41.29 |
47 | llama-3.2-1b-instruct | 26.17 | 37.64 | 36.67 | 35.27 | 24.75 | 30.00 | 34.00 | 32.83 | 23.17 | 28.00 | 30.80 | 28.50 | 25.40 | 26.80 | 27.50 | 32.89 | 23.25 | 25.25 | 28.40 | 26.00 | 29.00 | 29.60 | 24.80 | 31.82 | 31.83 | 27.17 | 33.27 | 28.25 | 27.45 | 31.67 | 32.67 | 19.54 | 24.24 | 24.14 | 21.00 | 26.86 | 25.70 | 23.45 | 19.80 | 22.00 | 26.67 | 23.78 | 24.67 | 23.33 | 24.36 | 21.00 | 28.00 | 19.71 | 20.29 | 26.89 | 20.27 | 18.75 | 22.00 | 21.00 | 33.14 | 23.33 | 24.67 | 25.09 | 26.80 | 18.33 | 26.36 | 22.29 | 25.50 | 17.75 | 23.17 | 17.00 | 16.91 | 23.56 | 28.36 | 26.00 | 25.17 | 22.55 | 23.11 | 19.67 | 22.91 | 22.60 | 21.17 | 14.67 | 35.43 | 23.00 | 24.15 | 28.67 | 25.56 | 23.57 | 26.86 | 30.43 | 30.00 | 34.75 | 25.75 | 26.33 | 23.43 | 30.77 | 34.46 | 24.55 | 35.67 | 20.46 | 30.57 | 24.91 | 26.53 | 30.75 | 32.71 |
10 | claude-3-7-sonnet-20250219-thinking | 81.34 | 81.23 | 83.84 | 81.28 | 83.42 | 80.71 | 79.69 |
1 | qwen3-235b-a22b-thinking-2507 | 81.34 | 81.23 | 83.84 | 81.28 | 83.42 | 80.71 | 79.69 |
2 | gpt-5-2025-08-07 | 81.21 | 80.47 | 80.96 | 81.17 | 83.11 | 79.93 | 74.69 |
3 | kimi-k2-instruct | 79.04 | 79.15 | 82.47 | 79.01 | 82.50 | 78.38 | 72.26 |
4 | doubao-seed-1-6-thinking-250715 | 78.92 | 78.92 | 81.04 | 78.78 | 81.59 | 77.50 | 68.72 |
5 | qwen3-235b-a22b-instruct-2507 | 78.66 | 78.88 | 82.11 | 78.77 | 83.33 | 77.79 | 75.27 |
6 | doubao-seed-1-6-thinking-250615 | 77.91 | 77.52 | 79.62 | 77.97 | 81.31 | 76.19 | 68.54 |
7 | o3-2025-04-16 | 77.68 | 77.64 | 80.10 | 77.43 | 82.29 | 78.14 | 75.58 |
8 | deepseek-r1-0528 | 77.49 | 77.82 | 80.34 | 77.36 | 81.14 | 77.64 | 72.30 |
9 | gemini-2.5-pro | 77.35 | 77.71 | 80.84 | 77.55 | 82.78 | 77.05 | 72.92 |
10 | doubao-seed-1-6-250615 | 77.03 | 77.41 | 79.56 | 76.51 | 80.70 | 76.86 | 71.81 |
11 | claude-opus-4-1-20250805 | 76.10 | 76.78 | 79.70 | 75.82 | 81.08 | 76.23 | 74.03 |
12 | claude-opus-4-20250514-thinking | 74.75 | 75.87 | 78.41 | 74.67 | 80.70 | 75.52 | 74.12 |
13 | claude-opus-4-20250514 | 74.22 | 75.07 | 78.01 | 73.72 | 79.75 | 74.86 | 72.21 |
14 | claude-sonnet-4-20250514 | 74.13 | 74.47 | 77.03 | 73.82 | 79.84 | 74.16 | 71.59 |
15 | claude-sonnet-4-20250514-thinking | 73.92 | 74.44 | 77.55 | 73.37 | 79.92 | 74.34 | 72.26 |
16 | qwen3-235b-a22b-thinking | 73.12 | 73.71 | 76.45 | 72.64 | 77.65 | 72.38 | 67.43 |
17 | claude-3-7-sonnet-20250219-thinking | 72.36 | 72.88 | 75.70 | 71.78 | 77.24 | 72.65 | 69.29 |
18 | claude-3-7-sonnet-20250219 | 71.70 | 71.92 | 74.04 | 71.17 | 76.75 | 71.78 | 67.17 |
19 | qwq-32b | 71.22 | 72.17 | 74.78 | 71.11 | 77.50 | 69.50 | 63.81 |
20 | chatgpt-4o-latest | 70.97 | 72.14 | 75.58 | 71.00 | 78.47 | 71.10 | 69.29 |
21 | deepseek-r1 | 70.54 | 71.85 | 75.04 | 70.20 | 74.76 | 69.60 | 60.27 |
22 | o4-mini-2025-04-16 | 70.52 | 70.59 | 72.83 | 70.40 | 76.75 | 71.02 | 67.61 |
23 | qwen3-235b-a22b-nothinking | 70.47 | 71.73 | 75.44 | 70.69 | 78.22 | 70.19 | 67.70 |
24 | grok-3 | 69.80 | 70.80 | 73.84 | 69.54 | 77.08 | 69.95 | 67.83 |
25 | gpt-4.1-2025-04-14 | 69.00 | 70.11 | 73.19 | 69.49 | 77.40 | 70.41 | 68.23 |
26 | deepseek-v3-250324 | 67.74 | 69.36 | 73.21 | 67.31 | 73.56 | 66.51 | 59.03 |
27 | o3-mini | 64.88 | 64.81 | 67.93 | 65.40 | 75.58 | 65.88 | 67.35 |
28 | qwen-max | 64.68 | 66.22 | 70.26 | 64.56 | 72.49 | 65.03 | 62.21 |
29 | gpt-4.5-preview | 64.31 | 65.51 | 69.62 | 64.76 | 72.56 | 63.67 | 59.91 |
30 | gemini-2.0-flash | 64.14 | 64.54 | 68.65 | 63.95 | 71.53 | 65.02 | 61.42 |
31 | o1-preview-2024-09-12 | 61.46 | 62.83 | 67.39 | 61.11 | 70.23 | 63.27 | 59.91 |
32 | deepseek-v3 | 56.32 | 57.60 | 61.99 | 56.25 | 65.99 | 57.19 | 53.27 |
33 | gemini-1.5-pro | 55.94 | 57.43 | 62.09 | 55.03 | 63.17 | 57.16 | 50.22 |
34 | qwen-2.5-72b-instruct | 54.19 | 54.99 | 57.57 | 54.94 | 64.48 | 54.43 | 49.78 |
35 | claude-3-5-sonnet-20241022 | 53.25 | 54.86 | 59.10 | 52.08 | 58.43 | 55.21 | 50.40 |
36 | gpt-4o-mini | 52.36 | 53.60 | 57.57 | 52.30 | 62.90 | 53.98 | 50.09 |
37 | mistral-large-instruct-2411 | 50.86 | 51.47 | 54.02 | 51.16 | 60.47 | 51.52 | 46.77 |
38 | llama-4-maverick-17b-128e | 48.68 | 49.88 | 53.23 | 48.37 | 56.03 | 49.59 | 45.58 |
39 | llama-4-scout-17b-16e-instruct | 47.84 | 48.44 | 51.10 | 47.90 | 55.68 | 48.54 | 44.29 |
40 | claude-3-5-haiku-20241022 | 47.57 | 49.08 | 53.73 | 46.74 | 53.05 | 48.07 | 43.27 |
41 | qwen-2.5-7b-instruct | 45.33 | 45.25 | 47.53 | 45.73 | 54.07 | 44.59 | 40.93 |
42 | llama-3.3-70b-instruct | 43.97 | 43.91 | 45.84 | 43.83 | 51.39 | 44.98 | 41.19 |
43 | qwen3-0.6b-thinking | 42.00 | 41.69 | 43.88 | 42.47 | 49.59 | 40.99 | 40.40 |
44 | qwen3-0.6b-nothinking | 38.53 | 38.52 | 40.72 | 39.22 | 46.07 | 37.50 | 36.90 |
45 | llama-3.1-8b-instruct | 36.52 | 36.10 | 37.41 | 36.65 | 42.68 | 37.23 | 33.58 |
46 | llama-3.2-3b-instruct | 34.52 | 33.51 | 34.54 | 34.25 | 39.98 | 34.49 | 31.81 |
47 | llama-3.2-1b-instruct | 26.17 | 24.71 | 25.46 | 26.27 | 28.69 | 25.35 | 23.63 |
Requirements: Style / Format / Length
Requirement_R: Includes all queries related to a specific requirement along with their entire set of criteria.
Requirement_C: Includes all queries related to a specific requirement but only the criteria pertinent to that specific requirement.
title={WritingBench: A Comprehensive Benchmark for Generative Writing},
author={Yuning Wu and Jiahao Mei and Ming Yan and Chenliang Li and Shaopeng Lai and Yuran Ren and Zijia Wang and Ji Zhang and Mengyue Wu and Qin Jin and Fei Huang},
year={2025},
url={https://arxiv.org/abs/2503.05244},
}```