๐ WritingBench: A Comprehensive Benchmark for Generative Writing
Evaluating LLMs' writing capabilities across 1,000 real-world queries, spanning 6 primary domains and 100 fine-grained subdomains.
๐ Last Updated: 2025-07-18 | ๐ Scored by: Claude-3.7-Sonnet
10 | doubao-seed-1-6-thinking-250615 | Mistral AI | 86.24 |
10 | doubao-seed-1-6-thinking-250615 | 86.24 | 86.42 | 85.97 | 85.54 | 87.19 | 86.52 | 85.95 |
1 | kimi-k2-instruct | 86.24 | 86.42 | 85.97 | 85.54 | 87.19 | 86.52 | 85.95 |
2 | doubao-seed-1-6-thinking-250615 | 85.58 | 86.17 | 85.44 | 84.42 | 85.57 | 86.77 | 85.84 |
3 | o3-2025-04-16 | 85.27 | 84.81 | 85.20 | 83.89 | 85.88 | 85.82 | 86.80 |
4 | doubao-seed-1-6-250615 | 83.33 | 84.49 | 82.56 | 78.64 | 85.65 | 85.57 | 85.22 |
5 | deepseek-r1-0528 | 83.22 | 83.15 | 81.48 | 81.55 | 85.68 | 84.14 | 84.44 |
6 | gemini-2.5-pro-preview-06-05 | 83.05 | 83.21 | 81.47 | 83.00 | 84.52 | 84.49 | 82.14 |
7 | gemini-2.5-pro-preview-03-25 | 81.09 | 81.04 | 79.47 | 80.90 | 82.96 | 81.91 | 80.77 |
8 | qwen3-235b-a22b-thinking | 80.48 | 79.86 | 78.72 | 79.73 | 82.48 | 81.06 | 81.97 |
9 | claude-3-7-sonnet-thinking | 78.83 | 78.66 | 77.68 | 77.61 | 80.10 | 79.12 | 80.78 |
10 | claude-3-7-sonnet | 78.48 | 78.24 | 77.93 | 76.51 | 79.37 | 79.26 | 80.88 |
11 | o4-mini-2025-04-16 | 78.44 | 79.47 | 76.72 | 77.93 | 77.13 | 81.08 | 80.27 |
12 | deepseek-r1 | 77.04 | 75.87 | 73.58 | 76.74 | 78.50 | 78.50 | 81.34 |
13 | chatgpt-4o-latest | 77.03 | 76.05 | 75.55 | 76.14 | 78.59 | 77.68 | 79.31 |
14 | gpt-4.1 | 76.60 | 76.48 | 75.21 | 76.13 | 77.73 | 77.23 | 77.59 |
15 | chatgpt-4o-2025-03-27 | 75.46 | 74.40 | 73.42 | 74.38 | 77.91 | 75.86 | 78.08 |
16 | grok-3 | 75.27 | 74.72 | 74.08 | 76.19 | 75.23 | 77.17 | 74.91 |
17 | o3-mini | 74.95 | 75.08 | 75.10 | 73.97 | 75.42 | 77.51 | 73.19 |
18 | deepseek-v3-2025-03-24 | 74.53 | 73.75 | 72.42 | 73.62 | 76.14 | 74.61 | 78.09 |
19 | gpt-4.5-preview | 71.99 | 71.82 | 69.96 | 71.14 | 72.03 | 73.86 | 75.22 |
20 | gemini-2.0-flash | 71.88 | 72.77 | 70.23 | 70.93 | 71.70 | 73.64 | 73.66 |
21 | qwen-max | 71.82 | 71.40 | 69.43 | 70.32 | 73.09 | 73.98 | 74.98 |
22 | o1-preview | 68.57 | 68.54 | 67.01 | 66.57 | 69.53 | 70.31 | 71.41 |
23 | qwen-2.5-72b-instruct | 63.63 | 65.69 | 61.13 | 63.29 | 61.56 | 66.74 | 65.81 |
24 | gemini-1.5-pro | 61.94 | 61.54 | 57.85 | 59.92 | 64.28 | 65.24 | 66.09 |
25 | mistral-large-instruct | 60.26 | 62.00 | 59.03 | 59.12 | 56.84 | 63.41 | 63.92 |
26 | gpt-4o-mini | 60.04 | 59.98 | 56.77 | 57.47 | 61.18 | 63.28 | 65.05 |
27 | claude-3-5-sonnet | 59.70 | 59.05 | 57.68 | 56.32 | 59.36 | 62.00 | 67.70 |
28 | qwen-2.5-7b-instruct | 56.02 | 58.87 | 55.39 | 55.15 | 50.96 | 60.34 | 58.20 |
29 | llama-4-scout-17b-16e-instruct | 55.82 | 56.30 | 54.30 | 54.23 | 54.20 | 58.16 | 60.50 |
30 | qwen3-0.6b-thinking | 55.62 | 58.86 | 56.39 | 55.85 | 50.16 | 59.08 | 54.55 |
31 | llama-4-maverick-17b-128e | 55.14 | 54.65 | 52.41 | 53.65 | 54.73 | 57.48 | 61.14 |
32 | claude-3-5-haiku | 54.52 | 55.77 | 53.20 | 51.50 | 51.90 | 56.14 | 62.09 |
33 | qwen3-0.6b-nothinking | 50.94 | 55.31 | 52.32 | 50.74 | 45.08 | 52.70 | 50.17 |
34 | llama-3.3-70b-instruct | 50.43 | 50.67 | 49.25 | 47.90 | 48.52 | 52.92 | 56.56 |
35 | llama-3.1-8b-instruct | 44.17 | 45.51 | 44.35 | 40.76 | 40.50 | 47.46 | 49.84 |
36 | llama-3.2-3b-instruct | 42.38 | 44.50 | 42.03 | 39.52 | 38.46 | 45.28 | 47.80 |
37 | llama-3.2-1b-instruct | 33.69 | 38.56 | 34.18 | 30.14 | 29.54 | 34.83 | 37.03 |
10 | doubao-seed-1-6-thinking-250615 | 86.24 | 91.33 | 91.64 | 84.91 | 88.17 | 88.20 | 87.33 | 87.75 | 87.17 | 88.33 | 88.00 | 88.25 | 81.78 | 82.25 | 84.00 | 82.36 | 85.00 | 84.50 | 87.00 | 87.60 | 90.75 | 84.55 | 85.80 | 86.31 | 89.20 | 88.73 | 87.64 | 85.50 | 85.43 | 89.50 | 87.17 | 88.14 | 87.75 | 85.00 | 84.71 | 80.50 | 83.70 | 77.45 | 86.67 | 83.33 | 88.18 | 90.89 | 84.92 | 89.50 | 89.00 | 87.83 | 89.00 | 82.00 | 88.33 | 79.87 | 87.87 | 85.56 | 86.29 | 83.83 | 83.38 | 90.29 | 70.00 | 82.36 | 88.00 | 86.00 | 93.60 | 89.82 | 81.78 | 90.00 | 87.33 | 90.36 | 86.00 | 86.55 | 89.50 | 86.50 | 90.25 | 85.67 | 87.00 | 89.50 | 81.40 | 90.44 | 86.00 | 84.55 | 90.50 | 86.67 | 88.67 | 86.77 | 89.14 | 88.43 | 79.25 | 82.50 | 87.25 | 82.57 | 87.00 | 87.33 | 84.29 | 82.67 | 82.00 | 87.23 | 89.82 | 89.57 | 90.77 | 86.75 | 85.09 | 79.06 | 88.00 |
1 | kimi-k2-instruct | 86.24 | 91.33 | 91.64 | 84.91 | 88.17 | 88.20 | 87.33 | 87.75 | 87.17 | 88.33 | 88.00 | 88.25 | 81.78 | 82.25 | 84.00 | 82.36 | 85.00 | 84.50 | 87.00 | 87.60 | 90.75 | 84.55 | 85.80 | 86.31 | 89.20 | 88.73 | 87.64 | 85.50 | 85.43 | 89.50 | 87.17 | 88.14 | 87.75 | 85.00 | 84.71 | 80.50 | 83.70 | 77.45 | 86.67 | 83.33 | 88.18 | 90.89 | 84.92 | 89.50 | 89.00 | 87.83 | 89.00 | 82.00 | 88.33 | 79.87 | 87.87 | 85.56 | 86.29 | 83.83 | 83.38 | 90.29 | 70.00 | 82.36 | 88.00 | 86.00 | 93.60 | 89.82 | 81.78 | 90.00 | 87.33 | 90.36 | 86.00 | 86.55 | 89.50 | 86.50 | 90.25 | 85.67 | 87.00 | 89.50 | 81.40 | 90.44 | 86.00 | 84.55 | 90.50 | 86.67 | 88.67 | 86.77 | 89.14 | 88.43 | 79.25 | 82.50 | 87.25 | 82.57 | 87.00 | 87.33 | 84.29 | 82.67 | 82.00 | 87.23 | 89.82 | 89.57 | 90.77 | 86.75 | 85.09 | 79.06 | 88.00 |
2 | doubao-seed-1-6-thinking-250615 | 85.58 | 88.67 | 88.00 | 83.45 | 89.83 | 86.80 | 86.83 | 88.00 | 87.33 | 87.83 | 87.80 | 86.75 | 85.33 | 83.75 | 80.67 | 82.73 | 87.00 | 85.75 | 84.00 | 89.60 | 86.75 | 85.27 | 87.40 | 86.77 | 87.20 | 87.27 | 88.00 | 84.83 | 84.29 | 84.50 | 86.67 | 88.00 | 84.25 | 82.00 | 83.53 | 83.00 | 83.50 | 82.73 | 87.11 | 84.00 | 89.64 | 89.33 | 86.15 | 87.33 | 87.40 | 87.33 | 69.25 | 85.71 | 86.67 | 85.07 | 84.80 | 85.33 | 85.43 | 83.00 | 77.85 | 86.57 | 75.00 | 81.45 | 90.50 | 92.33 | 90.40 | 86.91 | 87.78 | 87.71 | 86.00 | 86.55 | 87.07 | 81.64 | 90.00 | 87.50 | 84.00 | 87.33 | 87.67 | 82.83 | 80.60 | 89.11 | 78.55 | 79.09 | 92.00 | 89.33 | 88.00 | 86.77 | 88.57 | 88.29 | 84.50 | 86.75 | 84.75 | 80.86 | 86.86 | 81.00 | 87.14 | 82.67 | 85.85 | 85.23 | 85.09 | 88.71 | 92.46 | 86.00 | 83.82 | 82.00 | 84.93 |
3 | o3-2025-04-16 | 85.27 | 90.00 | 90.36 | 87.82 | 88.33 | 87.60 | 88.17 | 85.75 | 86.67 | 87.33 | 72.20 | 88.00 | 84.00 | 82.25 | 80.50 | 81.64 | 78.40 | 84.75 | 86.00 | 80.00 | 88.75 | 84.91 | 86.40 | 86.77 | 88.20 | 86.73 | 87.45 | 86.33 | 86.86 | 87.00 | 85.17 | 84.43 | 87.75 | 84.33 | 84.82 | 72.75 | 83.40 | 83.82 | 86.22 | 83.78 | 87.27 | 87.33 | 85.54 | 88.67 | 88.00 | 87.83 | 87.00 | 83.14 | 88.33 | 83.73 | 79.33 | 85.56 | 84.00 | 84.17 | 81.85 | 85.43 | 69.67 | 70.36 | 88.00 | 88.00 | 90.80 | 90.36 | 79.33 | 90.29 | 83.33 | 88.00 | 86.93 | 86.55 | 90.50 | 85.33 | 88.00 | 86.83 | 87.67 | 86.00 | 72.60 | 86.67 | 81.64 | 86.18 | 89.75 | 88.44 | 88.67 | 86.00 | 88.57 | 88.00 | 83.75 | 81.25 | 86.00 | 81.71 | 87.57 | 73.00 | 86.00 | 84.67 | 85.69 | 89.85 | 90.00 | 89.00 | 91.08 | 87.75 | 86.91 | 78.24 | 87.33 |
4 | doubao-seed-1-6-250615 | 83.33 | 84.67 | 88.36 | 84.00 | 89.00 | 84.80 | 85.00 | 85.00 | 87.00 | 86.50 | 85.40 | 85.50 | 84.67 | 78.25 | 79.00 | 80.36 | 85.20 | 81.50 | 83.00 | 87.20 | 82.75 | 84.18 | 83.60 | 84.92 | 82.20 | 84.91 | 84.36 | 81.33 | 80.29 | 82.83 | 85.50 | 83.57 | 81.75 | 80.67 | 80.71 | 81.25 | 78.60 | 81.64 | 86.67 | 74.67 | 86.36 | 85.78 | 84.46 | 85.33 | 87.40 | 85.00 | 57.50 | 75.71 | 87.67 | 82.93 | 60.53 | 82.44 | 82.57 | 75.50 | 76.92 | 73.71 | 69.33 | 71.82 | 89.50 | 90.00 | 90.00 | 87.45 | 83.78 | 89.71 | 63.33 | 88.18 | 86.13 | 86.91 | 86.00 | 85.83 | 87.25 | 85.00 | 80.50 | 84.33 | 84.20 | 87.78 | 84.55 | 85.64 | 89.00 | 88.44 | 86.89 | 85.38 | 87.14 | 86.14 | 81.00 | 85.50 | 83.00 | 82.86 | 85.86 | 84.00 | 87.14 | 84.00 | 84.31 | 85.23 | 87.09 | 87.71 | 89.38 | 84.75 | 86.55 | 78.12 | 85.60 |
5 | deepseek-r1-0528 | 83.22 | 86.00 | 87.27 | 83.09 | 87.00 | 84.00 | 83.00 | 86.00 | 83.67 | 85.17 | 80.20 | 85.00 | 85.11 | 81.25 | 81.33 | 79.82 | 79.20 | 77.00 | 82.00 | 87.20 | 85.25 | 84.73 | 83.00 | 81.85 | 86.40 | 83.27 | 81.82 | 80.83 | 83.43 | 81.67 | 84.50 | 80.86 | 82.25 | 80.33 | 79.29 | 76.75 | 79.10 | 72.18 | 85.11 | 82.00 | 85.64 | 84.89 | 84.15 | 86.50 | 85.60 | 82.83 | 82.25 | 80.57 | 82.00 | 81.60 | 81.87 | 77.78 | 82.86 | 77.83 | 80.31 | 79.43 | 68.33 | 73.64 | 89.50 | 85.33 | 89.20 | 87.45 | 85.78 | 91.43 | 89.33 | 80.73 | 85.73 | 86.73 | 88.00 | 87.33 | 89.00 | 85.50 | 87.33 | 84.83 | 83.20 | 81.33 | 85.45 | 80.55 | 88.50 | 86.89 | 85.78 | 85.54 | 86.29 | 85.57 | 79.25 | 82.25 | 81.50 | 77.14 | 84.00 | 84.00 | 81.71 | 83.33 | 82.46 | 84.62 | 87.27 | 84.86 | 88.62 | 83.50 | 86.18 | 80.82 | 84.93 |
6 | gemini-2.5-pro-preview-06-05 | 83.05 | 83.33 | 86.18 | 85.09 | 86.67 | 83.60 | 85.83 | 86.00 | 85.83 | 84.33 | 85.00 | 85.25 | 81.33 | 82.25 | 71.67 | 76.73 | 85.00 | 81.25 | 82.00 | 88.80 | 82.75 | 80.91 | 79.00 | 83.54 | 85.60 | 83.82 | 83.64 | 80.33 | 83.14 | 80.17 | 84.17 | 83.71 | 81.50 | 76.00 | 79.53 | 77.25 | 78.00 | 80.18 | 85.78 | 83.56 | 86.73 | 84.67 | 82.31 | 83.83 | 87.00 | 84.00 | 81.50 | 85.14 | 83.00 | 80.53 | 81.47 | 82.67 | 81.71 | 82.33 | 82.31 | 79.14 | 86.00 | 78.91 | 84.00 | 80.00 | 86.80 | 87.64 | 77.33 | 87.14 | 84.67 | 86.55 | 82.40 | 85.64 | 86.50 | 87.67 | 89.50 | 85.00 | 85.33 | 79.17 | 84.40 | 85.11 | 85.27 | 82.55 | 87.75 | 86.00 | 84.44 | 88.92 | 85.14 | 87.86 | 80.75 | 82.75 | 82.25 | 72.86 | 84.86 | 82.67 | 80.29 | 80.33 | 80.15 | 82.62 | 85.82 | 82.86 | 86.77 | 82.50 | 80.18 | 79.65 | 81.73 |
7 | gemini-2.5-pro-preview-03-25 | 81.09 | 82.00 | 81.09 | 81.82 | 86.00 | 80.80 | 83.00 | 84.00 | 85.17 | 84.00 | 84.60 | 83.75 | 78.44 | 78.00 | 72.50 | 72.73 | 82.00 | 77.75 | 77.00 | 86.40 | 78.75 | 79.64 | 82.60 | 86.15 | 83.60 | 80.00 | 83.82 | 77.17 | 77.71 | 79.83 | 82.83 | 81.57 | 76.75 | 75.00 | 78.24 | 74.50 | 73.40 | 76.00 | 83.78 | 83.33 | 86.55 | 85.33 | 81.54 | 81.83 | 84.20 | 80.67 | 77.25 | 82.86 | 80.00 | 80.00 | 79.60 | 79.56 | 78.57 | 79.50 | 80.46 | 72.29 | 85.67 | 75.09 | 83.50 | 78.67 | 86.80 | 87.09 | 79.11 | 85.14 | 78.67 | 84.18 | 83.33 | 84.00 | 86.50 | 85.33 | 86.00 | 84.00 | 82.50 | 82.67 | 77.40 | 84.89 | 81.82 | 77.64 | 88.50 | 84.44 | 82.00 | 76.15 | 83.14 | 85.71 | 81.50 | 79.00 | 79.25 | 74.86 | 84.57 | 81.33 | 76.00 | 79.00 | 80.46 | 79.38 | 84.73 | 83.86 | 86.77 | 80.25 | 79.09 | 77.76 | 79.07 |
8 | qwen3-235b-a22b-thinking | 80.48 | 83.33 | 83.45 | 83.82 | 81.67 | 83.40 | 78.83 | 80.25 | 77.67 | 79.83 | 77.60 | 79.50 | 78.22 | 80.75 | 80.33 | 79.64 | 75.40 | 74.75 | 77.50 | 81.20 | 83.75 | 81.27 | 78.80 | 78.62 | 82.00 | 78.18 | 80.55 | 79.17 | 79.71 | 81.67 | 79.17 | 79.43 | 78.75 | 74.00 | 78.12 | 80.25 | 76.30 | 68.73 | 82.00 | 80.89 | 79.82 | 74.00 | 83.23 | 79.50 | 82.40 | 81.83 | 84.50 | 79.14 | 80.67 | 77.60 | 80.53 | 78.44 | 79.43 | 80.83 | 78.92 | 78.00 | 80.67 | 72.73 | 87.00 | 92.33 | 85.60 | 86.55 | 83.33 | 88.00 | 84.00 | 84.55 | 81.07 | 80.36 | 83.50 | 84.33 | 85.25 | 87.00 | 79.50 | 84.50 | 75.60 | 76.44 | 78.36 | 74.36 | 85.75 | 73.56 | 83.33 | 80.62 | 85.43 | 82.57 | 80.50 | 84.50 | 78.75 | 77.14 | 79.71 | 82.00 | 84.00 | 81.33 | 82.15 | 83.08 | 82.55 | 86.14 | 82.15 | 78.25 | 83.09 | 78.47 | 80.80 |
9 | claude-3-7-sonnet-thinking | 78.83 | 80.67 | 81.27 | 79.82 | 76.50 | 81.80 | 79.67 | 81.50 | 79.67 | 81.00 | 80.60 | 80.25 | 78.00 | 73.75 | 73.17 | 74.73 | 78.60 | 78.00 | 80.50 | 80.00 | 78.25 | 79.45 | 82.40 | 79.85 | 80.40 | 79.45 | 79.27 | 78.50 | 71.14 | 80.67 | 83.67 | 77.57 | 75.00 | 76.67 | 74.94 | 78.00 | 72.60 | 70.00 | 82.00 | 78.00 | 79.82 | 81.33 | 81.08 | 76.00 | 79.80 | 73.17 | 80.25 | 78.86 | 76.67 | 77.07 | 76.27 | 79.33 | 76.57 | 76.17 | 75.69 | 74.29 | 80.67 | 72.91 | 83.50 | 80.33 | 85.20 | 85.82 | 73.78 | 78.86 | 75.33 | 83.09 | 83.20 | 79.64 | 73.50 | 84.83 | 81.00 | 80.33 | 84.50 | 78.50 | 76.60 | 73.33 | 78.55 | 74.73 | 83.25 | 80.67 | 80.67 | 80.00 | 80.86 | 81.00 | 75.50 | 78.00 | 80.50 | 73.71 | 78.14 | 73.67 | 78.86 | 79.00 | 80.15 | 83.23 | 81.09 | 83.14 | 86.62 | 80.50 | 76.55 | 75.88 | 82.13 |
10 | claude-3-7-sonnet | 78.48 | 83.33 | 83.27 | 80.55 | 74.67 | 81.60 | 82.00 | 83.75 | 79.50 | 80.50 | 83.40 | 78.50 | 74.67 | 73.00 | 68.00 | 71.64 | 79.40 | 76.50 | 78.50 | 81.60 | 78.25 | 78.91 | 80.20 | 81.85 | 81.60 | 79.64 | 78.91 | 77.67 | 76.29 | 77.83 | 82.50 | 78.57 | 77.00 | 78.67 | 77.65 | 78.00 | 70.60 | 71.82 | 82.44 | 77.11 | 77.82 | 78.00 | 78.77 | 73.83 | 79.60 | 75.00 | 79.00 | 73.71 | 70.00 | 75.73 | 77.20 | 76.44 | 74.29 | 77.00 | 79.69 | 72.86 | 77.00 | 70.55 | 82.00 | 80.00 | 83.20 | 83.64 | 76.89 | 79.14 | 72.00 | 84.00 | 81.87 | 72.55 | 77.50 | 84.33 | 80.50 | 79.17 | 83.67 | 77.33 | 73.80 | 74.00 | 79.27 | 76.73 | 84.50 | 82.00 | 78.22 | 79.08 | 82.29 | 82.00 | 76.00 | 76.75 | 78.00 | 72.00 | 79.57 | 77.33 | 80.57 | 77.00 | 79.38 | 82.46 | 82.00 | 83.86 | 87.38 | 81.75 | 78.55 | 75.06 | 81.07 |
11 | o4-mini-2025-04-16 | 78.44 | 82.67 | 82.91 | 82.36 | 82.83 | 82.60 | 80.67 | 81.50 | 79.00 | 80.67 | 79.40 | 82.25 | 80.67 | 73.50 | 76.83 | 76.73 | 72.40 | 74.25 | 78.00 | 82.80 | 81.25 | 82.00 | 79.80 | 78.15 | 80.20 | 80.73 | 79.27 | 78.17 | 81.71 | 77.50 | 71.83 | 76.71 | 76.75 | 78.33 | 74.82 | 67.50 | 69.50 | 72.36 | 79.78 | 78.44 | 79.82 | 81.78 | 79.38 | 79.33 | 81.60 | 81.50 | 80.00 | 72.86 | 79.67 | 76.67 | 79.20 | 83.11 | 77.14 | 77.33 | 75.54 | 80.57 | 66.33 | 65.64 | 83.00 | 76.33 | 88.00 | 83.64 | 74.44 | 79.71 | 58.67 | 78.91 | 80.13 | 78.18 | 61.00 | 79.17 | 71.25 | 80.00 | 78.17 | 74.50 | 73.40 | 77.78 | 76.91 | 72.18 | 84.25 | 82.89 | 82.89 | 79.85 | 85.43 | 83.43 | 77.50 | 77.25 | 81.00 | 76.00 | 80.71 | 80.33 | 81.14 | 80.00 | 78.77 | 80.92 | 82.91 | 83.43 | 84.31 | 76.75 | 80.55 | 72.82 | 82.40 |
12 | deepseek-r1 | 77.04 | 84.67 | 82.00 | 80.00 | 77.33 | 79.00 | 74.83 | 74.50 | 73.67 | 76.50 | 72.00 | 74.50 | 71.78 | 76.00 | 78.67 | 76.73 | 70.80 | 69.25 | 79.50 | 80.80 | 80.50 | 79.27 | 73.80 | 75.23 | 78.40 | 72.91 | 70.55 | 75.33 | 72.86 | 74.50 | 74.33 | 73.43 | 70.50 | 69.00 | 72.71 | 73.25 | 71.80 | 61.09 | 81.33 | 77.11 | 74.91 | 79.11 | 82.92 | 74.17 | 78.00 | 78.00 | 85.75 | 73.71 | 79.67 | 77.20 | 79.20 | 74.00 | 76.29 | 77.50 | 75.38 | 76.29 | 67.00 | 64.18 | 84.50 | 89.67 | 80.80 | 85.27 | 83.33 | 83.71 | 87.33 | 83.27 | 76.40 | 79.82 | 81.50 | 79.17 | 74.50 | 80.67 | 76.00 | 78.67 | 72.60 | 73.33 | 70.55 | 68.36 | 82.00 | 80.89 | 81.78 | 79.23 | 80.00 | 80.00 | 73.25 | 81.50 | 74.50 | 75.14 | 75.57 | 77.67 | 84.29 | 82.00 | 82.77 | 80.15 | 80.18 | 85.57 | 80.46 | 77.00 | 81.27 | 81.18 | 79.73 |
13 | chatgpt-4o-latest | 77.03 | 82.00 | 78.73 | 79.45 | 77.00 | 77.60 | 76.67 | 78.75 | 77.50 | 75.50 | 76.60 | 78.25 | 76.44 | 78.25 | 70.00 | 69.82 | 74.60 | 71.00 | 78.50 | 85.60 | 75.25 | 78.18 | 76.60 | 77.38 | 79.60 | 75.82 | 76.00 | 76.50 | 78.00 | 74.00 | 75.83 | 75.00 | 74.00 | 74.00 | 74.82 | 75.75 | 72.30 | 68.36 | 79.11 | 78.67 | 76.91 | 80.22 | 76.62 | 74.00 | 79.00 | 79.17 | 79.25 | 77.71 | 76.00 | 76.67 | 74.67 | 77.33 | 75.71 | 76.17 | 75.69 | 72.00 | 71.33 | 66.55 | 81.50 | 74.33 | 82.40 | 82.91 | 82.44 | 84.57 | 82.67 | 79.82 | 78.80 | 78.73 | 76.50 | 80.00 | 81.25 | 78.00 | 82.00 | 76.00 | 75.20 | 74.22 | 75.64 | 70.73 | 82.00 | 80.22 | 80.22 | 78.77 | 78.86 | 80.14 | 75.00 | 78.50 | 73.75 | 67.14 | 76.57 | 77.33 | 79.14 | 77.33 | 80.31 | 78.92 | 82.55 | 82.00 | 80.46 | 78.00 | 78.91 | 75.76 | 78.80 |
14 | gpt-4.1 | 76.60 | 78.00 | 77.82 | 79.27 | 73.67 | 77.00 | 78.83 | 79.25 | 80.83 | 79.33 | 78.00 | 79.75 | 76.89 | 75.50 | 72.00 | 68.00 | 74.80 | 72.50 | 79.50 | 84.40 | 75.00 | 77.27 | 77.60 | 75.69 | 76.60 | 77.45 | 77.45 | 76.33 | 76.57 | 72.83 | 78.17 | 70.14 | 73.50 | 78.33 | 74.12 | 76.50 | 72.20 | 68.55 | 78.89 | 76.22 | 78.00 | 79.78 | 77.23 | 76.33 | 81.40 | 75.83 | 77.25 | 78.00 | 76.67 | 75.07 | 74.93 | 78.89 | 71.71 | 74.67 | 75.23 | 72.86 | 79.67 | 67.09 | 80.00 | 74.00 | 78.40 | 80.91 | 81.11 | 79.43 | 82.00 | 78.73 | 78.80 | 74.73 | 79.00 | 78.33 | 79.00 | 77.67 | 78.50 | 78.17 | 74.40 | 76.67 | 73.64 | 75.64 | 80.75 | 79.56 | 79.78 | 78.46 | 78.57 | 81.43 | 75.75 | 75.25 | 75.25 | 68.29 | 76.00 | 71.67 | 78.00 | 76.33 | 77.08 | 76.62 | 80.00 | 79.29 | 83.54 | 79.25 | 76.00 | 71.53 | 77.87 |
15 | chatgpt-4o-2025-03-27 | 75.46 | 77.33 | 76.36 | 76.73 | 74.33 | 75.00 | 77.33 | 74.75 | 72.50 | 74.50 | 74.20 | 74.50 | 76.44 | 78.25 | 69.83 | 69.45 | 75.20 | 71.25 | 74.00 | 83.20 | 75.00 | 75.45 | 74.40 | 76.62 | 76.60 | 75.45 | 74.00 | 73.67 | 75.71 | 75.00 | 75.67 | 71.71 | 71.25 | 73.00 | 71.65 | 71.50 | 70.00 | 63.82 | 78.67 | 76.67 | 74.36 | 78.00 | 77.38 | 75.83 | 72.80 | 76.00 | 76.50 | 77.71 | 73.67 | 74.53 | 74.67 | 72.00 | 74.57 | 75.33 | 74.62 | 68.00 | 68.00 | 64.73 | 81.00 | 77.00 | 81.20 | 82.18 | 83.11 | 79.71 | 82.67 | 79.27 | 77.33 | 77.82 | 82.50 | 79.67 | 77.25 | 78.67 | 81.67 | 74.50 | 70.80 | 72.67 | 73.27 | 75.45 | 80.50 | 78.67 | 82.44 | 78.77 | 77.71 | 75.71 | 75.00 | 74.50 | 73.25 | 71.43 | 72.71 | 66.33 | 77.71 | 79.33 | 78.31 | 76.77 | 82.00 | 80.14 | 79.85 | 73.75 | 78.18 | 76.00 | 76.93 |
16 | grok-3 | 75.27 | 55.33 | 72.18 | 72.36 | 77.33 | 77.20 | 76.83 | 77.75 | 77.17 | 78.67 | 70.20 | 77.25 | 72.89 | 78.00 | 72.50 | 71.09 | 78.00 | 72.00 | 77.50 | 74.00 | 75.50 | 74.00 | 76.60 | 79.69 | 78.20 | 74.18 | 74.73 | 73.67 | 75.43 | 71.33 | 77.17 | 73.00 | 72.75 | 71.67 | 74.47 | 72.75 | 70.20 | 68.73 | 80.44 | 79.33 | 81.27 | 79.78 | 78.15 | 77.00 | 76.20 | 79.67 | 68.00 | 77.14 | 76.67 | 77.07 | 75.73 | 74.89 | 74.00 | 74.83 | 74.31 | 69.71 | 76.33 | 69.82 | 64.50 | 76.00 | 68.80 | 82.36 | 70.00 | 83.71 | 80.67 | 72.91 | 79.87 | 72.18 | 80.00 | 72.83 | 80.25 | 77.83 | 81.33 | 75.33 | 69.00 | 76.00 | 63.45 | 75.45 | 74.25 | 81.33 | 77.78 | 81.54 | 68.86 | 80.29 | 74.00 | 76.50 | 76.75 | 75.14 | 75.43 | 79.00 | 69.71 | 79.33 | 71.23 | 74.46 | 76.73 | 80.00 | 76.31 | 71.50 | 74.73 | 72.24 | 76.80 |
17 | o3-mini | 74.95 | 76.67 | 71.64 | 76.18 | 77.33 | 77.40 | 77.67 | 79.75 | 78.50 | 73.50 | 78.60 | 77.50 | 76.67 | 71.75 | 70.00 | 68.73 | 74.80 | 71.50 | 73.00 | 81.20 | 73.75 | 75.82 | 75.60 | 80.00 | 78.60 | 78.55 | 78.73 | 74.67 | 74.00 | 76.67 | 78.33 | 74.86 | 74.25 | 74.67 | 72.82 | 71.75 | 68.50 | 71.82 | 77.33 | 76.44 | 80.36 | 75.56 | 72.77 | 72.17 | 79.00 | 72.17 | 73.25 | 75.71 | 75.33 | 72.13 | 73.60 | 76.89 | 71.43 | 74.00 | 73.54 | 68.29 | 68.33 | 70.36 | 74.50 | 76.67 | 74.40 | 77.64 | 72.89 | 79.43 | 50.00 | 75.09 | 78.53 | 76.91 | 70.00 | 77.50 | 77.25 | 73.67 | 80.50 | 74.00 | 66.60 | 79.56 | 78.73 | 72.00 | 84.00 | 81.11 | 76.67 | 79.54 | 76.86 | 83.29 | 76.00 | 69.75 | 76.25 | 68.57 | 76.86 | 73.67 | 72.57 | 74.67 | 71.69 | 71.38 | 77.64 | 72.71 | 78.92 | 74.25 | 74.91 | 67.53 | 72.53 |
18 | deepseek-v3-2025-03-24 | 74.53 | 80.67 | 76.91 | 76.18 | 75.50 | 79.00 | 72.50 | 71.75 | 72.67 | 73.83 | 68.60 | 74.25 | 71.78 | 76.00 | 75.67 | 75.45 | 68.20 | 67.25 | 79.00 | 80.40 | 75.75 | 76.18 | 74.40 | 71.69 | 73.00 | 72.36 | 72.55 | 72.33 | 71.71 | 70.83 | 73.50 | 73.00 | 67.25 | 65.67 | 74.24 | 75.50 | 72.00 | 62.18 | 77.56 | 76.67 | 73.82 | 79.56 | 79.08 | 76.83 | 76.20 | 78.00 | 65.00 | 74.57 | 71.00 | 77.20 | 74.27 | 71.78 | 72.00 | 73.00 | 70.62 | 68.00 | 65.67 | 61.82 | 82.50 | 82.67 | 82.80 | 81.09 | 85.11 | 79.14 | 88.00 | 80.91 | 76.53 | 75.09 | 81.50 | 76.83 | 78.75 | 77.17 | 76.50 | 75.17 | 72.00 | 70.89 | 64.91 | 59.82 | 79.00 | 75.56 | 79.33 | 77.08 | 73.71 | 74.14 | 71.25 | 75.00 | 71.25 | 74.00 | 71.43 | 73.67 | 79.14 | 80.67 | 81.85 | 78.00 | 75.09 | 79.86 | 77.85 | 76.75 | 76.00 | 75.88 | 78.93 |
19 | gpt-4.5-preview | 71.99 | 80.00 | 75.64 | 75.82 | 74.00 | 75.00 | 71.83 | 71.00 | 68.50 | 71.67 | 71.00 | 70.50 | 72.89 | 75.25 | 70.00 | 70.18 | 68.00 | 64.25 | 74.00 | 79.60 | 75.50 | 75.09 | 69.80 | 73.23 | 73.00 | 67.27 | 70.18 | 72.67 | 70.57 | 70.67 | 69.17 | 68.71 | 68.50 | 68.00 | 68.94 | 67.75 | 67.60 | 58.91 | 76.89 | 74.00 | 73.82 | 76.22 | 72.62 | 67.83 | 66.80 | 72.33 | 78.25 | 74.86 | 73.00 | 72.13 | 72.13 | 71.78 | 70.86 | 71.17 | 70.46 | 67.43 | 63.67 | 57.45 | 78.50 | 72.00 | 75.20 | 79.82 | 72.00 | 74.86 | 75.33 | 77.45 | 74.67 | 76.00 | 75.00 | 68.50 | 72.50 | 74.33 | 76.50 | 70.17 | 63.20 | 61.56 | 67.27 | 63.27 | 76.75 | 75.33 | 78.44 | 77.85 | 76.57 | 74.43 | 75.25 | 70.25 | 69.50 | 70.57 | 68.86 | 72.00 | 77.43 | 76.33 | 75.54 | 74.15 | 76.91 | 77.14 | 76.62 | 73.25 | 74.73 | 72.82 | 74.27 |
20 | gemini-2.0-flash | 71.88 | 78.67 | 72.73 | 76.00 | 79.17 | 70.80 | 75.33 | 69.00 | 76.33 | 73.67 | 76.60 | 74.75 | 69.56 | 72.50 | 67.00 | 63.64 | 74.40 | 68.50 | 74.00 | 75.20 | 71.50 | 69.27 | 75.20 | 80.62 | 75.80 | 72.18 | 73.64 | 70.50 | 68.86 | 71.67 | 68.00 | 72.14 | 66.50 | 66.67 | 66.00 | 62.75 | 64.10 | 66.18 | 75.56 | 75.11 | 77.82 | 72.67 | 72.62 | 75.67 | 72.40 | 68.50 | 66.25 | 71.43 | 73.33 | 72.13 | 73.07 | 63.11 | 68.29 | 69.83 | 72.00 | 63.14 | 69.67 | 60.36 | 74.50 | 72.00 | 76.40 | 76.91 | 74.67 | 72.86 | 76.67 | 74.73 | 75.73 | 72.00 | 68.00 | 73.50 | 75.00 | 67.83 | 70.00 | 66.50 | 69.40 | 68.89 | 67.64 | 67.09 | 82.25 | 80.67 | 73.33 | 75.85 | 70.29 | 76.14 | 76.00 | 71.00 | 71.25 | 66.29 | 70.14 | 65.67 | 75.43 | 72.33 | 74.00 | 73.08 | 75.09 | 72.57 | 75.54 | 75.50 | 70.55 | 74.47 | 72.27 |
21 | qwen-max | 71.82 | 79.33 | 74.18 | 76.36 | 73.33 | 70.20 | 72.00 | 74.25 | 72.17 | 70.83 | 68.40 | 68.25 | 72.67 | 73.75 | 72.33 | 68.18 | 68.20 | 62.50 | 76.50 | 75.60 | 72.50 | 73.45 | 72.00 | 71.38 | 69.60 | 73.27 | 68.36 | 69.67 | 71.43 | 66.83 | 71.33 | 70.57 | 65.75 | 67.33 | 67.65 | 66.75 | 66.90 | 61.09 | 74.89 | 76.67 | 74.55 | 70.67 | 73.38 | 69.83 | 70.20 | 71.83 | 75.00 | 74.00 | 69.33 | 70.93 | 70.00 | 69.33 | 71.14 | 68.50 | 66.46 | 66.00 | 69.33 | 56.73 | 77.00 | 74.33 | 78.40 | 75.09 | 78.67 | 74.86 | 78.67 | 75.09 | 73.33 | 74.36 | 74.00 | 72.67 | 75.75 | 71.17 | 71.33 | 67.67 | 70.40 | 69.11 | 71.45 | 71.27 | 77.00 | 76.44 | 76.89 | 76.46 | 74.29 | 73.71 | 76.00 | 73.75 | 68.00 | 69.43 | 70.86 | 75.00 | 77.14 | 77.67 | 76.62 | 75.38 | 76.18 | 77.71 | 73.08 | 72.50 | 73.45 | 73.18 | 73.87 |
22 | o1-preview | 68.57 | 52.67 | 73.09 | 71.27 | 63.67 | 67.20 | 65.17 | 74.00 | 75.00 | 70.00 | 68.20 | 68.25 | 67.56 | 69.25 | 70.33 | 68.18 | 63.20 | 66.75 | 72.00 | 76.40 | 71.50 | 72.00 | 74.40 | 68.77 | 68.60 | 70.91 | 71.45 | 68.17 | 70.29 | 64.33 | 67.83 | 64.57 | 66.00 | 67.67 | 56.12 | 64.00 | 63.00 | 62.55 | 72.67 | 68.89 | 65.27 | 70.89 | 70.77 | 69.33 | 69.60 | 60.17 | 68.00 | 77.43 | 65.67 | 68.00 | 66.00 | 62.44 | 61.43 | 64.67 | 63.08 | 63.43 | 74.00 | 55.82 | 76.50 | 74.33 | 73.20 | 71.82 | 77.33 | 70.57 | 74.67 | 66.00 | 74.80 | 71.82 | 68.00 | 70.17 | 72.25 | 60.50 | 71.33 | 64.50 | 64.40 | 68.00 | 65.45 | 67.27 | 77.75 | 70.00 | 72.67 | 75.08 | 71.71 | 75.14 | 68.75 | 67.00 | 66.50 | 66.86 | 67.71 | 55.67 | 77.14 | 76.00 | 74.77 | 70.77 | 70.73 | 69.14 | 75.85 | 71.25 | 72.00 | 67.29 | 67.60 |
23 | qwen-2.5-72b-instruct | 63.63 | 67.33 | 66.55 | 69.09 | 65.00 | 69.20 | 67.83 | 69.00 | 67.00 | 65.00 | 64.20 | 66.25 | 64.89 | 63.75 | 63.83 | 63.64 | 67.40 | 56.00 | 66.00 | 67.20 | 68.25 | 62.91 | 63.40 | 65.85 | 60.80 | 61.27 | 61.45 | 61.17 | 62.00 | 60.67 | 61.17 | 62.43 | 63.00 | 58.33 | 57.76 | 56.50 | 57.20 | 56.00 | 66.67 | 67.56 | 64.00 | 65.11 | 68.00 | 65.00 | 61.40 | 61.50 | 65.50 | 64.29 | 58.00 | 61.07 | 64.40 | 66.00 | 66.00 | 61.33 | 61.08 | 56.86 | 59.67 | 60.18 | 72.00 | 69.33 | 65.60 | 64.55 | 64.67 | 58.29 | 60.67 | 60.18 | 64.53 | 64.55 | 60.00 | 63.17 | 60.00 | 60.50 | 58.83 | 57.17 | 62.20 | 60.00 | 53.64 | 60.36 | 72.50 | 69.56 | 70.44 | 70.31 | 64.57 | 66.71 | 70.00 | 62.00 | 64.25 | 67.43 | 59.00 | 66.67 | 70.86 | 68.00 | 71.08 | 67.54 | 65.27 | 64.71 | 66.77 | 67.75 | 61.45 | 63.29 | 62.13 |
24 | gemini-1.5-pro | 61.94 | 59.33 | 66.18 | 67.64 | 67.17 | 66.20 | 58.17 | 60.25 | 62.33 | 61.50 | 57.20 | 60.25 | 61.56 | 66.75 | 63.83 | 59.82 | 57.40 | 44.25 | 63.50 | 71.60 | 67.00 | 60.73 | 64.80 | 60.92 | 60.40 | 57.27 | 58.73 | 60.33 | 58.57 | 56.67 | 56.67 | 57.29 | 53.25 | 53.00 | 56.12 | 51.50 | 53.50 | 48.73 | 66.22 | 61.78 | 60.91 | 64.00 | 58.92 | 63.33 | 61.00 | 63.67 | 63.75 | 62.86 | 51.67 | 59.47 | 59.87 | 53.56 | 60.57 | 58.83 | 59.23 | 56.86 | 63.00 | 48.91 | 68.50 | 76.33 | 68.00 | 69.82 | 70.67 | 63.14 | 74.67 | 66.18 | 65.33 | 67.64 | 68.00 | 60.83 | 65.50 | 66.17 | 62.00 | 61.17 | 62.80 | 56.00 | 59.27 | 53.45 | 73.00 | 66.44 | 66.22 | 67.23 | 65.43 | 65.57 | 62.00 | 66.00 | 62.00 | 68.29 | 60.14 | 62.33 | 70.86 | 71.33 | 72.31 | 64.62 | 66.91 | 67.86 | 63.23 | 59.50 | 64.00 | 68.00 | 60.80 |
25 | mistral-large-instruct | 60.26 | 70.67 | 65.82 | 68.73 | 64.17 | 61.40 | 59.67 | 61.25 | 66.00 | 60.17 | 63.40 | 59.50 | 58.89 | 57.25 | 63.17 | 62.73 | 61.40 | 49.75 | 64.50 | 60.80 | 63.75 | 60.73 | 61.40 | 61.85 | 60.20 | 60.36 | 62.00 | 58.83 | 61.71 | 58.83 | 59.00 | 58.43 | 58.50 | 57.33 | 56.71 | 59.00 | 54.70 | 52.55 | 61.33 | 62.67 | 61.45 | 59.78 | 59.54 | 60.50 | 57.40 | 60.67 | 60.25 | 61.43 | 55.67 | 59.87 | 57.87 | 58.89 | 60.00 | 59.17 | 58.92 | 53.43 | 59.33 | 53.09 | 66.00 | 68.67 | 60.80 | 56.91 | 62.00 | 60.29 | 60.00 | 59.45 | 59.33 | 56.55 | 56.50 | 59.00 | 51.50 | 57.00 | 54.00 | 53.33 | 53.80 | 53.56 | 54.36 | 48.55 | 67.25 | 69.11 | 60.44 | 66.46 | 65.43 | 62.14 | 68.25 | 62.50 | 58.25 | 65.71 | 58.00 | 59.67 | 70.00 | 68.67 | 70.15 | 69.54 | 63.27 | 57.86 | 60.15 | 62.75 | 62.18 | 65.65 | 58.27 |
26 | gpt-4o-mini | 60.04 | 72.67 | 64.73 | 67.27 | 63.50 | 59.80 | 61.00 | 56.50 | 57.83 | 61.33 | 61.20 | 56.00 | 60.44 | 57.25 | 61.67 | 56.36 | 56.60 | 48.00 | 61.50 | 58.40 | 63.25 | 62.55 | 60.00 | 60.31 | 58.60 | 58.36 | 58.36 | 59.67 | 59.43 | 54.83 | 54.83 | 53.29 | 56.75 | 53.00 | 54.94 | 53.75 | 54.10 | 47.64 | 60.44 | 62.22 | 60.55 | 58.44 | 58.00 | 57.83 | 56.60 | 57.00 | 62.25 | 59.14 | 55.67 | 60.40 | 59.47 | 57.33 | 53.43 | 57.67 | 53.38 | 56.86 | 46.67 | 50.91 | 66.50 | 67.67 | 64.00 | 65.82 | 66.00 | 61.14 | 64.00 | 64.00 | 61.47 | 62.91 | 62.50 | 62.33 | 66.25 | 59.17 | 61.33 | 55.67 | 57.20 | 57.11 | 58.18 | 52.73 | 66.75 | 66.44 | 63.11 | 63.38 | 66.29 | 62.29 | 65.25 | 62.25 | 60.50 | 66.00 | 57.29 | 66.00 | 71.71 | 68.67 | 69.08 | 65.38 | 65.45 | 63.43 | 62.15 | 63.25 | 62.55 | 64.71 | 63.60 |
27 | claude-3-5-sonnet | 59.70 | 75.33 | 66.18 | 67.64 | 60.67 | 59.60 | 60.33 | 53.00 | 60.17 | 49.50 | 60.40 | 53.25 | 56.67 | 53.75 | 64.17 | 62.18 | 55.40 | 49.75 | 64.50 | 62.80 | 71.00 | 64.00 | 63.20 | 62.46 | 60.00 | 58.36 | 54.18 | 62.33 | 56.00 | 55.83 | 53.67 | 57.86 | 52.25 | 53.00 | 53.18 | 52.75 | 56.20 | 48.73 | 62.67 | 59.33 | 62.18 | 60.44 | 54.00 | 55.67 | 52.60 | 59.33 | 64.25 | 60.57 | 58.00 | 54.53 | 56.67 | 52.89 | 63.14 | 56.33 | 49.08 | 56.29 | 52.00 | 44.91 | 72.50 | 73.67 | 61.60 | 66.91 | 75.56 | 54.86 | 67.33 | 66.55 | 56.67 | 67.09 | 59.00 | 52.17 | 63.25 | 58.67 | 54.17 | 52.67 | 57.20 | 41.78 | 57.27 | 51.64 | 65.00 | 62.22 | 63.56 | 63.54 | 63.43 | 62.71 | 57.00 | 62.50 | 59.00 | 68.57 | 59.57 | 56.67 | 70.57 | 72.33 | 74.77 | 71.08 | 62.18 | 69.71 | 65.38 | 64.00 | 65.82 | 68.59 | 62.00 |
28 | qwen-2.5-7b-instruct | 56.02 | 63.33 | 66.73 | 63.82 | 62.33 | 62.00 | 59.50 | 57.25 | 58.50 | 55.83 | 54.80 | 54.00 | 58.22 | 55.50 | 60.83 | 58.36 | 57.60 | 49.75 | 65.00 | 58.80 | 59.50 | 61.64 | 63.40 | 56.15 | 54.60 | 57.64 | 54.18 | 56.83 | 54.57 | 53.83 | 56.17 | 52.57 | 54.50 | 53.33 | 51.76 | 54.75 | 52.30 | 49.45 | 58.00 | 61.78 | 56.00 | 50.00 | 56.92 | 51.67 | 57.60 | 57.33 | 55.50 | 52.57 | 54.33 | 55.07 | 56.53 | 56.89 | 56.57 | 53.50 | 53.38 | 53.14 | 57.67 | 49.45 | 59.50 | 68.33 | 58.00 | 44.73 | 65.11 | 50.57 | 55.33 | 48.55 | 52.13 | 56.00 | 50.00 | 53.50 | 41.50 | 49.00 | 42.17 | 51.67 | 52.00 | 46.89 | 46.36 | 47.09 | 64.50 | 63.78 | 60.67 | 58.62 | 61.14 | 61.00 | 65.25 | 59.50 | 60.00 | 60.86 | 54.43 | 58.67 | 62.86 | 64.33 | 67.08 | 60.00 | 59.27 | 54.71 | 50.62 | 60.25 | 57.82 | 56.12 | 54.93 |
29 | llama-4-scout-17b-16e-instruct | 55.82 | 67.33 | 62.36 | 61.09 | 60.50 | 55.60 | 53.67 | 55.00 | 52.67 | 52.33 | 56.80 | 53.25 | 55.56 | 58.75 | 58.50 | 61.09 | 51.20 | 45.75 | 65.00 | 63.20 | 63.75 | 58.00 | 57.60 | 61.54 | 57.40 | 53.45 | 57.64 | 54.00 | 52.00 | 53.00 | 49.17 | 54.86 | 50.25 | 51.33 | 50.94 | 47.00 | 52.70 | 45.09 | 55.33 | 58.00 | 57.45 | 55.56 | 53.23 | 53.00 | 49.80 | 54.33 | 57.75 | 58.00 | 54.00 | 56.40 | 57.73 | 53.33 | 58.57 | 52.17 | 53.08 | 49.14 | 48.67 | 47.64 | 63.50 | 62.67 | 60.00 | 57.27 | 53.11 | 54.29 | 57.33 | 57.82 | 56.93 | 60.73 | 60.00 | 53.17 | 56.25 | 52.00 | 52.33 | 48.00 | 51.60 | 46.89 | 52.00 | 45.82 | 58.75 | 62.22 | 56.67 | 60.92 | 54.29 | 60.71 | 57.50 | 62.75 | 57.00 | 62.86 | 50.57 | 54.67 | 67.43 | 64.33 | 68.77 | 61.38 | 56.91 | 56.00 | 54.46 | 61.75 | 59.82 | 61.41 | 58.67 |
30 | qwen3-0.6b-thinking | 55.62 | 67.33 | 63.64 | 63.27 | 67.00 | 63.40 | 59.17 | 59.50 | 52.67 | 55.00 | 54.00 | 56.50 | 53.56 | 61.00 | 58.33 | 61.09 | 55.80 | 53.00 | 66.00 | 56.40 | 62.75 | 62.36 | 58.20 | 52.62 | 62.40 | 60.18 | 58.73 | 56.00 | 54.00 | 59.17 | 55.83 | 55.00 | 60.00 | 53.67 | 52.71 | 57.75 | 51.00 | 48.18 | 54.22 | 60.22 | 49.27 | 49.78 | 64.00 | 55.33 | 53.20 | 55.67 | 61.75 | 47.71 | 55.33 | 56.93 | 60.93 | 57.33 | 50.00 | 58.33 | 60.15 | 51.43 | 47.67 | 53.45 | 58.50 | 61.00 | 54.00 | 52.73 | 60.89 | 55.71 | 60.67 | 49.64 | 52.80 | 49.82 | 51.50 | 48.33 | 42.50 | 54.50 | 41.17 | 49.50 | 52.40 | 42.44 | 45.45 | 40.91 | 63.75 | 57.78 | 61.78 | 61.69 | 66.57 | 54.71 | 56.25 | 58.25 | 58.75 | 59.71 | 58.29 | 53.00 | 59.71 | 59.67 | 56.00 | 56.00 | 52.36 | 57.86 | 46.92 | 52.00 | 58.00 | 53.88 | 52.27 |
31 | llama-4-maverick-17b-128e | 55.14 | 74.67 | 62.55 | 58.36 | 59.00 | 53.80 | 53.83 | 51.75 | 52.83 | 52.17 | 52.60 | 50.50 | 52.44 | 57.50 | 55.83 | 55.82 | 50.00 | 45.00 | 65.00 | 59.60 | 63.25 | 58.00 | 53.00 | 59.69 | 53.80 | 50.00 | 52.36 | 53.33 | 53.14 | 50.17 | 45.17 | 52.29 | 49.75 | 48.33 | 50.00 | 48.00 | 51.30 | 45.27 | 56.89 | 59.33 | 54.36 | 54.67 | 49.23 | 55.33 | 51.40 | 53.67 | 62.75 | 58.57 | 48.67 | 55.73 | 54.13 | 50.22 | 61.71 | 51.50 | 47.85 | 50.00 | 56.00 | 47.64 | 58.50 | 65.00 | 58.80 | 57.64 | 62.67 | 53.14 | 74.00 | 59.64 | 55.07 | 58.55 | 57.50 | 51.50 | 52.75 | 55.17 | 55.33 | 54.67 | 52.00 | 44.22 | 48.36 | 43.45 | 61.00 | 56.89 | 58.44 | 62.92 | 53.14 | 56.71 | 59.00 | 59.50 | 54.75 | 62.29 | 49.71 | 58.67 | 67.71 | 63.33 | 67.69 | 60.46 | 57.64 | 58.00 | 59.54 | 62.00 | 58.91 | 61.76 | 59.47 |
32 | claude-3-5-haiku | 54.52 | 70.67 | 63.45 | 64.55 | 62.33 | 53.60 | 53.17 | 48.75 | 49.67 | 52.50 | 54.60 | 50.00 | 56.44 | 52.00 | 63.00 | 59.45 | 51.00 | 45.75 | 64.00 | 58.80 | 68.25 | 59.09 | 56.60 | 56.77 | 55.00 | 51.64 | 50.18 | 57.00 | 49.43 | 52.83 | 49.50 | 55.86 | 49.50 | 49.33 | 49.18 | 47.50 | 50.10 | 45.64 | 53.11 | 58.67 | 54.91 | 55.11 | 49.23 | 51.50 | 49.00 | 52.17 | 57.25 | 53.14 | 52.33 | 51.73 | 50.80 | 48.00 | 51.43 | 47.17 | 48.31 | 51.14 | 61.00 | 43.64 | 63.00 | 66.00 | 52.00 | 55.27 | 69.33 | 53.71 | 62.00 | 56.00 | 50.13 | 53.64 | 57.50 | 50.67 | 49.50 | 55.00 | 45.17 | 45.50 | 52.80 | 40.44 | 45.27 | 42.36 | 57.00 | 57.78 | 60.00 | 58.92 | 49.43 | 53.86 | 55.50 | 62.00 | 50.75 | 61.71 | 53.86 | 53.00 | 70.86 | 70.67 | 69.38 | 64.62 | 59.09 | 57.00 | 59.69 | 56.50 | 57.64 | 66.47 | 56.40 |
33 | qwen3-0.6b-nothinking | 50.94 | 62.67 | 58.55 | 58.55 | 62.67 | 61.00 | 57.67 | 55.25 | 48.67 | 53.50 | 53.80 | 54.25 | 49.33 | 57.50 | 51.33 | 55.82 | 55.00 | 47.75 | 65.50 | 56.00 | 60.75 | 58.00 | 53.80 | 49.23 | 54.60 | 56.55 | 54.91 | 52.17 | 51.71 | 52.33 | 52.67 | 50.57 | 56.75 | 54.00 | 47.88 | 53.50 | 45.40 | 44.36 | 51.11 | 59.11 | 46.91 | 44.89 | 50.77 | 53.17 | 50.40 | 52.33 | 50.75 | 42.29 | 47.33 | 51.20 | 54.13 | 52.89 | 45.43 | 54.67 | 53.23 | 47.14 | 43.67 | 50.55 | 57.00 | 45.00 | 49.20 | 44.36 | 51.33 | 48.00 | 50.67 | 44.00 | 47.60 | 43.09 | 46.00 | 48.50 | 40.25 | 46.67 | 39.17 | 45.33 | 48.20 | 42.44 | 42.36 | 36.73 | 54.75 | 50.44 | 52.44 | 53.23 | 61.71 | 50.29 | 56.00 | 49.50 | 56.75 | 48.29 | 51.14 | 51.00 | 55.14 | 51.67 | 52.62 | 53.69 | 48.55 | 48.86 | 43.23 | 50.50 | 52.18 | 52.35 | 46.40 |
34 | llama-3.3-70b-instruct | 50.43 | 54.00 | 56.18 | 56.18 | 57.67 | 46.80 | 46.17 | 44.75 | 47.00 | 46.83 | 53.00 | 48.25 | 52.22 | 53.75 | 55.00 | 55.64 | 46.80 | 38.75 | 64.00 | 48.00 | 62.00 | 54.36 | 53.80 | 52.31 | 47.20 | 46.73 | 48.36 | 52.17 | 52.86 | 49.33 | 44.00 | 49.29 | 45.00 | 48.00 | 43.18 | 43.50 | 50.10 | 43.45 | 48.67 | 50.67 | 54.55 | 49.56 | 45.54 | 41.17 | 44.20 | 48.67 | 52.50 | 48.57 | 54.00 | 51.87 | 47.60 | 46.00 | 54.86 | 44.33 | 44.15 | 50.29 | 52.00 | 39.27 | 54.00 | 64.33 | 52.40 | 51.45 | 46.22 | 50.00 | 49.33 | 51.09 | 48.93 | 52.00 | 53.00 | 47.33 | 50.00 | 46.33 | 50.83 | 40.33 | 48.80 | 42.44 | 45.27 | 42.73 | 52.00 | 56.44 | 54.00 | 56.15 | 51.14 | 49.29 | 52.50 | 54.75 | 51.50 | 58.00 | 49.00 | 54.00 | 64.57 | 63.67 | 62.92 | 57.08 | 53.64 | 50.00 | 49.38 | 56.75 | 55.09 | 60.71 | 54.80 |
35 | llama-3.1-8b-instruct | 44.17 | 52.67 | 51.82 | 50.18 | 53.67 | 43.80 | 43.50 | 42.50 | 43.33 | 42.00 | 41.60 | 39.75 | 48.00 | 41.75 | 52.00 | 48.18 | 41.80 | 35.00 | 61.00 | 44.40 | 53.25 | 48.73 | 50.80 | 46.15 | 43.60 | 43.64 | 45.27 | 45.67 | 49.71 | 45.17 | 38.50 | 44.57 | 42.25 | 44.00 | 37.29 | 40.00 | 44.10 | 36.55 | 44.00 | 47.56 | 42.36 | 37.78 | 43.38 | 42.00 | 40.20 | 40.17 | 36.75 | 42.29 | 40.33 | 41.33 | 39.07 | 41.78 | 36.57 | 38.33 | 39.69 | 43.14 | 49.67 | 33.09 | 42.50 | 51.33 | 44.00 | 38.18 | 46.00 | 40.57 | 44.67 | 39.64 | 43.07 | 44.36 | 46.50 | 42.33 | 39.50 | 37.00 | 40.00 | 34.50 | 45.40 | 36.67 | 35.64 | 33.45 | 47.50 | 49.56 | 47.78 | 46.46 | 42.29 | 44.86 | 54.25 | 49.00 | 49.25 | 52.57 | 45.00 | 44.33 | 57.14 | 59.33 | 55.85 | 51.08 | 43.27 | 43.29 | 42.46 | 53.00 | 48.00 | 56.12 | 46.27 |
36 | llama-3.2-3b-instruct | 42.38 | 44.00 | 47.64 | 50.55 | 50.83 | 41.40 | 41.67 | 39.75 | 40.00 | 42.33 | 44.60 | 40.25 | 50.22 | 47.25 | 51.00 | 47.09 | 38.80 | 34.25 | 52.50 | 39.60 | 49.00 | 46.00 | 47.40 | 40.46 | 40.40 | 42.18 | 43.09 | 42.00 | 46.00 | 43.33 | 38.83 | 40.00 | 40.75 | 41.00 | 36.82 | 37.75 | 44.00 | 38.73 | 37.11 | 45.56 | 40.00 | 40.22 | 39.85 | 42.83 | 36.00 | 37.67 | 34.75 | 40.57 | 45.67 | 41.47 | 39.73 | 40.44 | 32.29 | 35.83 | 40.15 | 46.57 | 41.33 | 34.91 | 40.00 | 40.33 | 39.20 | 39.82 | 43.78 | 40.86 | 38.00 | 34.55 | 40.67 | 44.55 | 45.00 | 40.33 | 37.25 | 37.33 | 36.50 | 34.17 | 39.20 | 35.78 | 34.00 | 34.73 | 41.00 | 52.44 | 46.00 | 46.00 | 46.29 | 40.00 | 49.00 | 45.75 | 43.50 | 52.86 | 41.86 | 44.67 | 56.57 | 60.00 | 52.46 | 52.31 | 42.18 | 41.00 | 34.62 | 50.25 | 43.82 | 56.24 | 44.80 |
37 | llama-3.2-1b-instruct | 33.69 | 40.00 | 42.18 | 46.18 | 45.50 | 34.00 | 36.33 | 33.25 | 30.17 | 34.83 | 42.80 | 38.25 | 41.11 | 37.75 | 43.83 | 44.55 | 33.40 | 28.00 | 44.50 | 38.00 | 37.50 | 42.18 | 37.00 | 26.92 | 30.00 | 38.55 | 34.00 | 39.00 | 40.86 | 38.00 | 32.67 | 32.57 | 32.00 | 39.33 | 28.59 | 26.75 | 33.30 | 28.36 | 31.33 | 28.44 | 32.18 | 32.00 | 31.54 | 34.83 | 26.00 | 26.17 | 23.75 | 25.14 | 32.67 | 25.47 | 34.80 | 34.44 | 25.43 | 26.33 | 31.08 | 42.00 | 28.67 | 30.55 | 34.50 | 25.33 | 31.20 | 32.00 | 32.22 | 27.43 | 22.67 | 23.09 | 33.33 | 36.73 | 21.50 | 30.50 | 22.00 | 32.83 | 26.50 | 31.17 | 31.20 | 28.89 | 26.55 | 29.27 | 29.25 | 35.33 | 33.33 | 31.54 | 41.71 | 31.71 | 43.75 | 33.25 | 36.25 | 34.86 | 36.71 | 34.00 | 43.43 | 49.00 | 40.62 | 41.69 | 32.55 | 30.14 | 26.31 | 39.50 | 32.00 | 45.65 | 33.73 |
10 | doubao-seed-1-6-thinking-250615 | 86.24 | 86.24 | 89.50 | 86.60 | 91.19 | 86.19 | 84.91 |
1 | kimi-k2-instruct | 86.24 | 86.24 | 89.50 | 86.60 | 91.19 | 86.19 | 84.91 |
2 | doubao-seed-1-6-thinking-250615 | 85.58 | 85.87 | 87.99 | 85.78 | 90.59 | 85.17 | 84.03 |
3 | o3-2025-04-16 | 85.27 | 85.14 | 87.45 | 85.24 | 90.98 | 86.31 | 87.21 |
4 | doubao-seed-1-6-250615 | 83.33 | 84.24 | 86.61 | 83.00 | 88.45 | 84.25 | 84.78 |
5 | deepseek-r1-0528 | 83.22 | 84.24 | 87.27 | 83.72 | 89.35 | 83.83 | 82.70 |
6 | gemini-2.5-pro-preview-06-05 | 83.05 | 83.58 | 86.49 | 83.86 | 90.45 | 83.35 | 83.98 |
7 | gemini-2.5-pro-preview-03-25 | 81.09 | 81.78 | 84.98 | 81.73 | 89.16 | 81.78 | 82.74 |
8 | qwen3-235b-a22b-thinking | 80.48 | 81.61 | 85.38 | 80.66 | 86.95 | 80.86 | 80.93 |
9 | claude-3-7-sonnet-thinking | 78.83 | 79.67 | 83.53 | 79.16 | 86.22 | 79.92 | 81.50 |
10 | claude-3-7-sonnet | 78.48 | 79.43 | 82.51 | 78.84 | 86.14 | 79.23 | 80.49 |
11 | o4-mini-2025-04-16 | 78.44 | 78.95 | 82.41 | 78.59 | 86.30 | 79.33 | 79.20 |
12 | deepseek-r1 | 77.04 | 78.82 | 83.47 | 77.13 | 84.11 | 77.03 | 73.89 |
13 | chatgpt-4o-latest | 77.03 | 78.33 | 82.73 | 77.84 | 87.12 | 77.47 | 80.00 |
14 | gpt-4.1 | 76.60 | 77.73 | 81.91 | 77.44 | 86.99 | 78.23 | 80.62 |
15 | chatgpt-4o-2025-03-27 | 75.46 | 76.82 | 81.57 | 75.82 | 85.46 | 76.13 | 76.73 |
16 | grok-3 | 75.27 | 76.39 | 80.32 | 75.63 | 84.58 | 76.10 | 79.69 |
17 | o3-mini | 74.95 | 75.22 | 79.44 | 76.11 | 86.58 | 77.53 | 83.58 |
18 | deepseek-v3-2025-03-24 | 74.53 | 76.30 | 81.24 | 74.64 | 82.92 | 74.44 | 72.70 |
19 | gpt-4.5-preview | 71.99 | 73.14 | 78.17 | 72.71 | 82.62 | 71.85 | 72.83 |
20 | gemini-2.0-flash | 71.88 | 72.59 | 77.41 | 72.20 | 82.29 | 73.20 | 74.51 |
21 | qwen-max | 71.82 | 73.47 | 78.13 | 72.08 | 83.25 | 72.62 | 75.27 |
22 | o1-preview | 68.57 | 70.09 | 75.10 | 68.49 | 79.78 | 70.91 | 73.81 |
23 | qwen-2.5-72b-instruct | 63.63 | 64.86 | 69.50 | 64.66 | 76.97 | 64.26 | 64.73 |
24 | gemini-1.5-pro | 61.94 | 63.78 | 69.72 | 61.50 | 72.15 | 63.23 | 60.62 |
25 | mistral-large-instruct | 60.26 | 60.95 | 65.50 | 61.24 | 73.74 | 60.80 | 60.31 |
26 | gpt-4o-mini | 60.04 | 61.66 | 67.63 | 60.30 | 73.52 | 61.58 | 61.95 |
27 | claude-3-5-sonnet | 59.70 | 61.42 | 67.37 | 59.18 | 67.96 | 61.23 | 59.16 |
28 | qwen-2.5-7b-instruct | 56.02 | 56.43 | 61.10 | 56.99 | 69.28 | 56.05 | 56.95 |
29 | llama-4-scout-17b-16e-instruct | 55.82 | 56.79 | 61.97 | 55.98 | 66.52 | 56.48 | 55.27 |
30 | qwen3-0.6b-thinking | 55.62 | 56.09 | 62.05 | 56.61 | 68.98 | 55.05 | 59.38 |
31 | llama-4-maverick-17b-128e | 55.14 | 56.89 | 61.79 | 55.02 | 65.30 | 55.90 | 54.78 |
32 | claude-3-5-haiku | 54.52 | 56.13 | 62.37 | 54.32 | 63.35 | 54.30 | 51.81 |
33 | qwen3-0.6b-nothinking | 50.94 | 51.10 | 56.37 | 52.37 | 64.74 | 49.78 | 53.36 |
34 | llama-3.3-70b-instruct | 50.43 | 50.71 | 54.94 | 50.38 | 60.23 | 51.08 | 49.78 |
35 | llama-3.1-8b-instruct | 44.17 | 44.05 | 47.23 | 44.49 | 52.90 | 44.58 | 43.23 |
36 | llama-3.2-3b-instruct | 42.38 | 42.10 | 45.14 | 42.31 | 50.41 | 42.15 | 40.75 |
37 | llama-3.2-1b-instruct | 33.69 | 32.50 | 35.30 | 33.88 | 39.04 | 32.88 | 32.61 |
Requirements: Style / Format / Length
Requirement_R: Includes all queries related to a specific requirement along with their entire set of criteria.
Requirement_C: Includes all queries related to a specific requirement but only the criteria pertinent to that specific requirement.
title={WritingBench: A Comprehensive Benchmark for Generative Writing},
author={Yuning Wu and Jiahao Mei and Ming Yan and Chenliang Li and Shaopeng Lai and Yuran Ren and Zijia Wang and Ji Zhang and Mengyue Wu and Qin Jin and Fei Huang},
year={2025},
url={https://arxiv.org/abs/2503.05244},
}```