Roo Code tests each frontier model against a suite of hundreds of exercises across 5 programming languages with varying difficulty. These results can help you find the right price-to-intelligence ratio for your use case.
Want to see the results for a model we haven't tested yet? Ping us in Discord.
| Model | Metrics | Scores | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Name Context Window | Price In / Out | Duration | Tokens In / Out | Cost USD | Total | |||||
Claude Sonnet 4.5 0 | $0.00 / $0.00 | 3h 26m 50s | 30M / 430K | $38.43 | 100% | 100% | 100% | 100% | 100% | 100% |
GPT-5 Mini 0 | $0.00 / $0.00 | 5h 46m 33s | 14M / 977K | $3.34 | 100% | 98% | 100% | 100% | 97% | 99% |
Claude Opus 4.1 0 | $0.00 / $0.00 | 7h 3m 6s | 27M / 490K | $140.14 | 97% | 96% | 98% | 100% | 100% | 98% |
GPT-5 (Medium) 0 | $0.00 / $0.00 | 8h 40m 10s | 14M / 1M | $23.19 | 97% | 98% | 100% | 100% | 93% | 98% |
Claude Sonnet 4 0 | $0.00 / $0.00 | 5h 35m 31s | 39M / 644K | $39.61 | 94% | 100% | 98% | 100% | 97% | 98% |
Gemini 2.5 Pro 0 | $0.00 / $0.00 | 6h 17m 23s | 43M / 1M | $57.80 | 97% | 91% | 96% | 100% | 97% | 96% |
GPT-5 (Low) 0 | $0.00 / $0.00 | 5h 50m 41s | 16M / 862K | $16.18 | 100% | 96% | 86% | 100% | 100% | 95% |
Claude 3.7 Sonnet 0 | $0.00 / $0.00 | 5h 53m 33s | 38M / 894K | $37.58 | 92% | 98% | 94% | 100% | 93% | 95% |
Kimi K2 0905 (Groq) 0 | $0.00 / $0.00 | 3h 44m 51s | 13M / 619K | $15.25 | 94% | 91% | 96% | 97% | 93% | 94% |
Claude Opus 4 0 | $0.00 / $0.00 | 7h 50m 29s | 30M / 485K | $172.29 | 92% | 91% | 94% | 94% | 100% | 94% |
GPT-4.1 0 | $0.00 / $0.00 | 4h 39m 51s | 37M / 624K | $38.64 | 92% | 91% | 90% | 94% | 90% | 91% |
GPT-5 (Minimal) 0 | $0.00 / $0.00 | 5h 18m 41s | 23M / 453K | $14.45 | 94% | 82% | 92% | 94% | 90% | 90% |
Grok Code Fast 1 0 | $0.00 / $0.00 | 4h 52m 24s | 59M / 2M | $6.82 | 92% | 91% | 88% | 94% | 83% | 90% |
Gemini 2.5 Flash 0 | $0.00 / $0.00 | 3h 39m 38s | 61M / 1M | $14.15 | 89% | 91% | 92% | 85% | 90% | 90% |
Claude 3.5 Sonnet 0 | $0.00 / $0.00 | 3h 37m 58s | 19M / 323K | $24.98 | 94% | 91% | 92% | 88% | 80% | 90% |
Grok 3 0 | $0.00 / $0.00 | 5h 14m 20s | 40M / 890K | $74.40 | 97% | 89% | 90% | 91% | 77% | 89% |
Kimi K2 0905 0 | $0.00 / $0.00 | 8h 26m 13s | 36M / 491K | $28.14 | 83% | 82% | 96% | 91% | 90% | 89% |
Sonoma Sky 0 | $0.00 / $0.00 | 6h 40m 9s | 24M / 330K | $0.00 | 83% | 87% | 90% | 88% | 77% | 86% |
Qwen 3 Max 0 | $0.00 / $0.00 | 7h 59m 42s | 27M / 587K | $36.14 | 84% | 91% | 79% | 76% | 69% | 86% |
Z.AI: GLM 4.5 0 | $0.00 / $0.00 | 7h 2m 33s | 46M / 809K | $27.16 | 83% | 87% | 88% | 82% | 87% | 86% |
Qwen 3 Coder 0 | $0.00 / $0.00 | 7h 56m 14s | 51M / 828K | $27.63 | 86% | 80% | 82% | 85% | 87% | 84% |
Kimi K2 0711 0 | $0.00 / $0.00 | 7h 52m 24s | 27M / 433K | $12.39 | 81% | 80% | 88% | 82% | 83% | 83% |
GPT-4.1 Mini 0 | $0.00 / $0.00 | 5h 17m 57s | 47M / 715K | $8.81 | 81% | 84% | 94% | 76% | 70% | 83% |
o4 Mini (High) 0 | $0.00 / $0.00 | 14h 44m 26s | 13M / 3M | $25.70 | 75% | 82% | 86% | 79% | 67% | 79% |
Sonoma Dusk 0 | $0.00 / $0.00 | 7h 12m 38s | 89M / 1M | $0.00 | 86% | 53% | 84% | 91% | 83% | 78% |
GPT-5 Nano 0 | $0.00 / $0.00 | 9h 13m 34s | 16M / 3M | $1.61 | 86% | 73% | 76% | 79% | 77% | 78% |
DeepSeek V3 0 | $0.00 / $0.00 | 7h 12m 41s | 30M / 524K | $12.82 | 83% | 76% | 82% | 76% | 67% | 77% |
o3 Mini (High) 0 | $0.00 / $0.00 | 13h 1m 13s | 12M / 2M | $20.36 | 67% | 78% | 72% | 88% | 73% | 75% |
Qwen 3 Next 0 | $0.00 / $0.00 | 7h 29m 11s | 77M / 1M | $13.67 | 78% | 69% | 80% | 76% | 57% | 73% |
Grok 4 0 | $0.00 / $0.00 | 11h 27m 59s | 14M / 2M | $44.99 | 78% | 67% | 66% | 82% | 70% | 72% |
Z.AI: GLM 4.5 Air 0 | $0.00 / $0.00 | 10h 49m 5s | 59M / 856K | $10.86 | 58% | 58% | 60% | 41% | 50% | 54% |
Llama 4 Maverick 0 | $0.00 / $0.00 | 7h 41m 14s | 101M / 1M | $18.86 | 47% | 49% | 52% | 53% | 60% | 52% |