Skip to content
LegalForecastBenchIn partnership with LegalQuants
← Leaderboard

OpenAI

Eligible: the provider-reported training data/knowledge cutoff is before the earliest decision we used to test; a month-only cutoff counts as the first day of that month.Cutoff: May 18, 2026 (knowledge cutoff)

GPT-6 Luna

Micro BrierMicro Brier is the headline metric: squared error of each dismissal forecast, averaged over all claim-defendant units. Lower is better; always answering 50% scores 0.25.

0.1637

Rank 14 of 18 · On the observed cost/quality frontier: no other model is both cheaper and at least as good, and this model strictly improves over others on at least one dimension. (Note that the improvement does not have to be statistically significant.)

Equal-case Brier

0.1738

Rank 16 of 18 · On the observed cost/quality frontier: no other model is both cheaper and at least as good, and this model strictly improves over others on at least one dimension. (Note that the improvement does not have to be statistically significant.)

Unit accuracy

76.5%

Rank 15 of 18 · On the observed cost/quality frontier: no other model is both cheaper and at least as good, and this model strictly improves over others on at least one dimension. (Note that the improvement does not have to be statistically significant.)

Cost and quality

Quality metric
Headline metric
GPT-6 Sol: Micro Brier 0.1176, cost $11.01Claude Fable 5.1: Micro Brier 0.1288, cost $83.18Grok 4.6: Micro Brier 0.1337, cost $15.72Claude Opus 5: Micro Brier 0.1399, cost $45.89GPT-5.6 Sol: Micro Brier 0.1401, cost $24.91Kimi K3: Micro Brier 0.1413, cost $16.87GPT-5.6 Luna: Micro Brier 0.1432, cost $1.48Claude Opus 5.5: Micro Brier 0.1454, cost $20.59GPT-6.1 Sol: Micro Brier 0.1495, cost $10.68GPT-6 Astra: Micro Brier 0.1522, cost $59.74Gemini 3.8 Flash: Micro Brier 0.1533, cost $35.61Grok 4.7: Micro Brier 0.1583, cost $33.00Claude Sonnet 5: Micro Brier 0.1585, cost $24.68GPT-6 Luna: Micro Brier 0.1637, cost $0.62Muse Wonder: Micro Brier 0.1654, cost $57.59Claude Sonnet 5.5: Micro Brier 0.1730, cost $6.99Muse Spark 1.3: Micro Brier 0.1812, cost $8.78Gemini 3.1 Pro Preview: Micro Brier 0.1834, cost $59.04
Observed cost/quality frontier (descriptive, not significance)Other models

High-confidence forecasts

183 of 387 predictions (47.3%) were at least 90% confident either way. 19 were wrong, an error rate of 10.4%. Mean stated confidence in that subset was 95.4%.

  • Gemini 3.1 Pro Preview
    Gemini 3.1 Pro Preview: mean stated confidence 95.4%, realized accuracy 82.2% on 320 predictions
    57 / 320 (83% of units)
  • GPT-5.6 Sol
    GPT-5.6 Sol: mean stated confidence 97.0%, realized accuracy 88.6% on 228 predictions
    26 / 228 (59% of units)
  • Gemini 3.8 Flash
    Gemini 3.8 Flash: mean stated confidence 95.2%, realized accuracy 87.3% on 276 predictions
    35 / 276 (71% of units)
  • GPT-5.6 Luna
    GPT-5.6 Luna: mean stated confidence 94.9%, realized accuracy 89.0% on 182 predictions
    20 / 182 (47% of units)
  • GPT-6 Luna
    GPT-6 Luna: mean stated confidence 95.4%, realized accuracy 89.6% on 183 predictions
    19 / 183 (47% of units)
  • GPT-6 Astra
    GPT-6 Astra: mean stated confidence 95.5%, realized accuracy 89.9% on 207 predictions
    21 / 207 (53% of units)
  • GPT-6.1 Sol
    GPT-6.1 Sol: mean stated confidence 95.7%, realized accuracy 91.0% on 199 predictions
    18 / 199 (51% of units)
  • Muse Spark 1.3
    Muse Spark 1.3: mean stated confidence 93.5%, realized accuracy 88.9% on 135 predictions
    15 / 135 (35% of units)
  • Muse Wonder
    Muse Wonder: mean stated confidence 93.4%, realized accuracy 91.5% on 118 predictions
    10 / 118 (30% of units)
  • Claude Sonnet 5
    Claude Sonnet 5: mean stated confidence 92.6%, realized accuracy 91.7% on 60 predictions
    5 / 60 (16% of units)
  • GPT-6 Sol
    GPT-6 Sol: mean stated confidence 95.6%, realized accuracy 95.3% on 169 predictions
    8 / 169 (44% of units)
  • Claude Sonnet 5.5
    Claude Sonnet 5.5: mean stated confidence 91.6%, realized accuracy 92.1% on 63 predictions
    5 / 63 (16% of units)
  • Grok 4.7
    Grok 4.7: mean stated confidence 92.9%, realized accuracy 95.9% on 98 predictions
    4 / 98 (25% of units)
  • Claude Opus 5
    Claude Opus 5: mean stated confidence 93.6%, realized accuracy 96.7% on 121 predictions
    4 / 121 (31% of units)
  • Kimi K3
    Kimi K3: mean stated confidence 92.8%, realized accuracy 98.9% on 92 predictions
    1 / 92 (24% of units)
  • Grok 4.6
    Grok 4.6: mean stated confidence 93.0%, realized accuracy 99.1% on 112 predictions
    1 / 112 (29% of units)
  • Claude Fable 5.1
    Claude Fable 5.1: mean stated confidence 92.9%, realized accuracy 100.0% on 81 predictions
    0 / 81 (21% of units)
  • Claude Opus 5.5
    Claude Opus 5.5: mean stated confidence 92.8%, realized accuracy 100.0% on 65 predictions
    0 / 65 (17% of units)
80%85%90%95%100%
Mean stated confidenceRealized accuracySorted by the gap between them. Units, not independent cases: several misses can come from one case.

Statistical comparison coverage

Included in the paired significance analysis.

Analysis and reproducible inputs →

Statistically distinguishable differences

  • Better than GPT-4.1 on Micro Brier, Equal-case Brier, and Unit accuracy.
Comparison data →