Skip to content
LegalForecastBenchIn partnership with LegalQuants
← Leaderboard

OpenAI

Eligible: the provider-reported training data/knowledge cutoff is before the earliest decision we used to test; a month-only cutoff counts as the first day of that month.Cutoff: February 16, 2026 (knowledge cutoff)

GPT-5.6 Sol

Micro BrierMicro Brier is the headline metric: squared error of each dismissal forecast, averaged over all claim-defendant units. Lower is better; always answering 50% scores 0.25.

0.1401

Rank 5 of 18

Equal-case Brier

0.1332

Rank 2 of 18

Unit accuracy

80.9%

Rank 3 of 18

Cost and quality

Quality metric
Headline metric
GPT-6 Sol: Micro Brier 0.1176, cost $11.01Claude Fable 5.1: Micro Brier 0.1288, cost $83.18Grok 4.6: Micro Brier 0.1337, cost $15.72Claude Opus 5: Micro Brier 0.1399, cost $45.89GPT-5.6 Sol: Micro Brier 0.1401, cost $24.91Kimi K3: Micro Brier 0.1413, cost $16.87GPT-5.6 Luna: Micro Brier 0.1432, cost $1.48Claude Opus 5.5: Micro Brier 0.1454, cost $20.59GPT-6.1 Sol: Micro Brier 0.1495, cost $10.68GPT-6 Astra: Micro Brier 0.1522, cost $59.74Gemini 3.8 Flash: Micro Brier 0.1533, cost $35.61Grok 4.7: Micro Brier 0.1583, cost $33.00Claude Sonnet 5: Micro Brier 0.1585, cost $24.68GPT-6 Luna: Micro Brier 0.1637, cost $0.62Muse Wonder: Micro Brier 0.1654, cost $57.59Claude Sonnet 5.5: Micro Brier 0.1730, cost $6.99Muse Spark 1.3: Micro Brier 0.1812, cost $8.78Gemini 3.1 Pro Preview: Micro Brier 0.1834, cost $59.04
Observed cost/quality frontier (descriptive, not significance)Other models

High-confidence forecasts

228 of 387 predictions (58.9%) were at least 90% confident either way. 26 were wrong, an error rate of 11.4%. Mean stated confidence in that subset was 97.0%.

  • Gemini 3.1 Pro Preview
    Gemini 3.1 Pro Preview: mean stated confidence 95.4%, realized accuracy 82.2% on 320 predictions
    57 / 320 (83% of units)
  • GPT-5.6 Sol
    GPT-5.6 Sol: mean stated confidence 97.0%, realized accuracy 88.6% on 228 predictions
    26 / 228 (59% of units)
  • Gemini 3.8 Flash
    Gemini 3.8 Flash: mean stated confidence 95.2%, realized accuracy 87.3% on 276 predictions
    35 / 276 (71% of units)
  • GPT-5.6 Luna
    GPT-5.6 Luna: mean stated confidence 94.9%, realized accuracy 89.0% on 182 predictions
    20 / 182 (47% of units)
  • GPT-6 Luna
    GPT-6 Luna: mean stated confidence 95.4%, realized accuracy 89.6% on 183 predictions
    19 / 183 (47% of units)
  • GPT-6 Astra
    GPT-6 Astra: mean stated confidence 95.5%, realized accuracy 89.9% on 207 predictions
    21 / 207 (53% of units)
  • GPT-6.1 Sol
    GPT-6.1 Sol: mean stated confidence 95.7%, realized accuracy 91.0% on 199 predictions
    18 / 199 (51% of units)
  • Muse Spark 1.3
    Muse Spark 1.3: mean stated confidence 93.5%, realized accuracy 88.9% on 135 predictions
    15 / 135 (35% of units)
  • Muse Wonder
    Muse Wonder: mean stated confidence 93.4%, realized accuracy 91.5% on 118 predictions
    10 / 118 (30% of units)
  • Claude Sonnet 5
    Claude Sonnet 5: mean stated confidence 92.6%, realized accuracy 91.7% on 60 predictions
    5 / 60 (16% of units)
  • GPT-6 Sol
    GPT-6 Sol: mean stated confidence 95.6%, realized accuracy 95.3% on 169 predictions
    8 / 169 (44% of units)
  • Claude Sonnet 5.5
    Claude Sonnet 5.5: mean stated confidence 91.6%, realized accuracy 92.1% on 63 predictions
    5 / 63 (16% of units)
  • Grok 4.7
    Grok 4.7: mean stated confidence 92.9%, realized accuracy 95.9% on 98 predictions
    4 / 98 (25% of units)
  • Claude Opus 5
    Claude Opus 5: mean stated confidence 93.6%, realized accuracy 96.7% on 121 predictions
    4 / 121 (31% of units)
  • Kimi K3
    Kimi K3: mean stated confidence 92.8%, realized accuracy 98.9% on 92 predictions
    1 / 92 (24% of units)
  • Grok 4.6
    Grok 4.6: mean stated confidence 93.0%, realized accuracy 99.1% on 112 predictions
    1 / 112 (29% of units)
  • Claude Fable 5.1
    Claude Fable 5.1: mean stated confidence 92.9%, realized accuracy 100.0% on 81 predictions
    0 / 81 (21% of units)
  • Claude Opus 5.5
    Claude Opus 5.5: mean stated confidence 92.8%, realized accuracy 100.0% on 65 predictions
    0 / 65 (17% of units)
80%85%90%95%100%
Mean stated confidenceRealized accuracySorted by the gap between them. Units, not independent cases: several misses can come from one case.

Statistical comparison coverage

Included in the paired significance analysis.

Analysis and reproducible inputs →

Statistically distinguishable differences

  • Better than GPT-4.1 on Micro Brier, Equal-case Brier, and Unit accuracy.
Comparison data →