Skip to content

Cost and output

Every output with its grade, what the run cost, the tokens it used and how long the model worked.

Runs made by hand in a lab’s own app are counted whole: the helper agents the app started on the lab’s other models are in the cost, the tokens and the time. Tokens include cached input read again at each step, and a cost is those tokens at the lab’s list prices.

Cost by test

What each model’s run of a test cost, cheapest first. Bars compare the runs of one test: every chart has its own scale.

Zombie

T01 · Games
  1. Gemini 4 Argon$0.67
  2. GPT-6.1 Sol$1.18
  3. Claude Haiku 5.5$1.87
  4. Gemini 4 Carbon$2.22
  5. Gemini 4 Hydrogen$2.38
  6. Gemini 4 Neon$5.19
  7. GPT-6.2 Astra$5.21
  8. GPT-6.1 Terra$8.17
  9. GPT-6.2 Sol$9.76
  10. GPT-6 Astra$28.51
  11. DeepSeek V4.1 FlashCost unknown

Spider-Man

T02 · Games
  1. Gemini 4 Argon$1.09
  2. GPT-6.1 Sol$1.78
  3. Gemini 4 Carbon$1.79
  4. Gemini 4 Hydrogen$2.52
  5. Gemini 4 Neon$4.05
  6. GPT-6.1 Terra$5.32
  7. GPT-6.2 Astra$7.41
  8. GPT-6.2 Sol$9.15
  9. GPT-6 Astra$31.82
  10. DeepSeek V4.1 FlashCost unknown
  1. Gemini 4 Argon$1.01
  2. GPT-6.1 Terra$1.15
  3. Gemini 4 Hydrogen$1.16
  4. GPT-6.1 Sol$1.24
  5. Gemini 4 Carbon$1.27
  6. Gemini 4 Neon$2.22
  7. GPT-6.2 Sol$4.01
  8. GPT-6.2 Astra$4.09
  9. Claude Haiku 5.5$12.67
  10. GPT-6 Astra$16.29
  11. Claude Opus 5.5$115.96
  12. Claude Sonnet 5.5$225.99
  13. Claude Fable 5.2$237.18
  14. DeepSeek V4.1 FlashCost unknown

Bora Bora

T04 · Rendering
  1. Gemini 4 Argon$0.95
  2. Gemini 4 Hydrogen$1.63
  3. Gemini 4 Carbon$4.01
  4. Gemini 4 Neon$4.41
  5. GPT-6.2 Astra$6.32
  6. GPT-6.2 Sol$16.87
  7. GPT-6 Astra$17.20
  8. Claude Fable 5.2$115.69
  9. Claude Sonnet 5.5$263.73
  10. Claude Opus 5.5$1376.70

Every output

Every model on every public test: the output, the editor’s grade, the cost of the run in US dollars, the tokens used and how long the model worked.
Claude Fable 5.2Claude Haiku 5.5Claude Opus 5.5Claude Sonnet 5.5DeepSeek V4.1 FlashGemini 4 ArgonGemini 4 CarbonGemini 4 HydrogenGemini 4 NeonGPT-6 AstraGPT-6.1 SolGPT-6.1 TerraGPT-6.2 AstraGPT-6.2 Sol
All tests4 tests
Claude Fable 5.2

$352.87612M tokens5 h 22 min

Mean gradeNo mean yet

2 of 4 tests · 1 graded

Claude Haiku 5.5

$14.5433.0M tokens1 h 26 min

Mean gradeNo mean yet

2 of 4 tests · 0 graded

Claude Opus 5.5

$1492.664.0B tokens16 h 54 min

Mean gradeNo mean yet

2 of 4 tests · 0 graded

Claude Sonnet 5.5

$489.721.3B tokens10 h 48 min

Mean gradeNo mean yet

2 of 4 tests · 0 graded

DeepSeek V4.1 Flash

Cost, tokens and time unknown

Mean gradeNo mean yet

3 of 4 tests · 0 graded · 3 costs unknown · 3 token counts unknown · 3 times unknown

Gemini 4 Argon

$3.722.8M tokens41 min 6 s

Mean gradeNo mean yet

4 of 4 tests · 0 graded

Gemini 4 Carbon

$9.2919.6M tokens1 h 52 min

Mean gradeNo mean yet

4 of 4 tests · 0 graded

Gemini 4 Hydrogen

$7.6922.6M tokens58 min 44 s

Mean gradeNo mean yet

4 of 4 tests · 0 graded

Gemini 4 Neon

$15.8714.9M tokens1 h 23 min

Mean gradeNo mean yet

4 of 4 tests · 0 graded

GPT-6 Astra

$93.8265.3M tokens1 h 39 min

Mean gradeNo mean yet

4 of 4 tests · 0 graded

GPT-6.1 Sol

$4.2018.1M tokens1 h 29 min

Mean gradeNo mean yet

3 of 4 tests · 0 graded

GPT-6.1 Terra

$14.6422.0M tokens22 min 40 s

Mean gradeNo mean yet

3 of 4 tests · 0 graded

GPT-6.2 Astra

$23.0325.7M tokens2 h 18 min

Mean gradeNo mean yet

4 of 4 tests · 0 graded

GPT-6.2 Sol

$39.7944.5M tokens1 h 19 min

Mean gradeNo mean yet

4 of 4 tests · 0 graded

ZombieT01 · Games
Not run yetClaude Fable 5.2
Claude Haiku 5.5$1.8717.0M tokens33 min 58 sNot graded yet
Not run yetClaude Opus 5.5
Not run yetClaude Sonnet 5.5
DeepSeek V4.1 FlashCost, tokens and time unknownNot graded yetGemini 4 Argon$0.67269k tokens8 min 37 sNot graded yetGemini 4 Carbon$2.223.6M tokens21 min 26 sNot graded yetGemini 4 Hydrogen$2.386.0M tokens15 min 47 sNot graded yetGemini 4 Neon$5.194.0M tokens23 min 33 sNot graded yetGPT-6 Astra$28.5119.3M tokens26 min 9 sNot graded yetGPT-6.1 Sol$1.184.0M tokens28 min 34 sNot graded yetGPT-6.1 Terra$8.1711.7M tokens10 min 41 sNot graded yetGPT-6.2 Astra$5.215.1M tokens37 min 34 sNot graded yetGPT-6.2 Sol$9.769.5M tokens18 min 29 sNot graded yet
Spider-ManT02 · Games
Not run yetClaude Fable 5.2
Not run yetClaude Haiku 5.5
Not run yetClaude Opus 5.5
Not run yetClaude Sonnet 5.5
DeepSeek V4.1 FlashCost, tokens and time unknownNot graded yetGemini 4 Argon$1.09742k tokens13 min 35 sNot graded yetGemini 4 Carbon$1.792.5M tokens26 min 17 sNot graded yetGemini 4 Hydrogen$2.5210.2M tokens16 min 54 sNot graded yetGemini 4 Neon$4.054.5M tokens21 min 18 sNot graded yetGPT-6 Astra$31.8222.7M tokens26 min 48 sNot graded yetGPT-6.1 Sol$1.788.0M tokens37 min 27 sNot graded yetGPT-6.1 Terra$5.327.5M tokens7 min 48 sNot graded yetGPT-6.2 Astra$7.417.5M tokens41 min 45 sNot graded yetGPT-6.2 Sol$9.1510.1M tokens25 min 7 sNot graded yet
Pizzeria New YorkT03 · WebClaude Fable 5.2$237.18456M tokens3 h 42 min8.30Claude Haiku 5.5$12.6716.0M tokens52 min 8 sNot graded yetClaude Opus 5.5$115.96119M tokens4 h 1 minNot graded yetClaude Sonnet 5.5$225.99543M tokens4 h 13 minNot graded yetDeepSeek V4.1 FlashCost, tokens and time unknownNot graded yetGemini 4 Argon$1.01818k tokens10 min 51 sNot graded yetGemini 4 Carbon$1.271.3M tokens17 minNot graded yetGemini 4 Hydrogen$1.162.4M tokens12 min 26 sNot graded yetGemini 4 Neon$2.221.6M tokens17 min 48 sNot graded yetGPT-6 Astra$16.2911.2M tokens23 min 58 sNot graded yetGPT-6.1 Sol$1.246.1M tokens23 min 22 sNot graded yetGPT-6.1 Terra$1.152.9M tokens4 min 11 sNot graded yetGPT-6.2 Astra$4.094.9M tokens30 min 53 sNot graded yetGPT-6.2 Sol$4.015.6M tokens15 min 37 sNot graded yet
Bora BoraT04 · RenderingClaude Fable 5.2$115.69156M tokens1 h 40 minNot graded yet
Not run yetClaude Haiku 5.5
Claude Opus 5.5$1376.703.9B tokens12 h 53 minNot graded yetClaude Sonnet 5.5$263.73750M tokens6 h 35 minNot graded yet
Not run yetDeepSeek V4.1 Flash
Gemini 4 Argon$0.95941k tokens8 min 3 sNot graded yetGemini 4 Carbon$4.0112.2M tokens47 min 20 sNot graded yetGemini 4 Hydrogen$1.634.1M tokens13 min 37 sNot graded yetGemini 4 Neon$4.414.8M tokens19 min 55 sNot graded yetGPT-6 Astra$17.2012.1M tokens21 min 46 sNot graded yet
Not run yetGPT-6.1 Sol
Not run yetGPT-6.1 Terra
GPT-6.2 Astra$6.328.2M tokens27 min 30 sNot graded yetGPT-6.2 Sol$16.8719.3M tokens19 min 22 sNot graded yet

OutputBenchSame brief. Every model. Graded by hand.

Logos and names are trademarks of their owners, shown only to identify the models.