Skip to content
LegalForecastBenchIn partnership with LegalQuants
← Leaderboard

Meta

Qualified: no published training data/knowledge cutoff, or the cutoff is not before the earliest decision we used to test.Cutoff: not published

Muse Spark 1.3

Micro BrierMicro Brier is the headline metric: squared error of each dismissal forecast, averaged over all claim-defendant units. Lower is better; always answering 50% scores 0.25.

0.1812

Rank 17 of 18

Equal-case Brier

0.1784

Rank 17 of 18

Unit accuracy

72.1%

Rank 18 of 18

Cost and quality

Quality metric
Headline metric
GPT-6 Sol: Micro Brier 0.1176, cost $11.01Claude Fable 5.1: Micro Brier 0.1288, cost $83.18Grok 4.6: Micro Brier 0.1337, cost $15.72Claude Opus 5: Micro Brier 0.1399, cost $45.89GPT-5.6 Sol: Micro Brier 0.1401, cost $24.91Kimi K3: Micro Brier 0.1413, cost $16.87GPT-5.6 Luna: Micro Brier 0.1432, cost $1.48Claude Opus 5.5: Micro Brier 0.1454, cost $20.59GPT-6.1 Sol: Micro Brier 0.1495, cost $10.68GPT-6 Astra: Micro Brier 0.1522, cost $59.74Gemini 3.8 Flash: Micro Brier 0.1533, cost $35.61Grok 4.7: Micro Brier 0.1583, cost $33.00Claude Sonnet 5: Micro Brier 0.1585, cost $24.68GPT-6 Luna: Micro Brier 0.1637, cost $0.62Muse Wonder: Micro Brier 0.1654, cost $57.59Claude Sonnet 5.5: Micro Brier 0.1730, cost $6.99Muse Spark 1.3: Micro Brier 0.1812, cost $8.78Gemini 3.1 Pro Preview: Micro Brier 0.1834, cost $59.04
Observed cost/quality frontier (descriptive, not significance)Other models

High-confidence forecasts

135 of 387 predictions (34.9%) were at least 90% confident either way. 15 were wrong, an error rate of 11.1%. Mean stated confidence in that subset was 93.5%.

  • Gemini 3.1 Pro Preview
    Gemini 3.1 Pro Preview: mean stated confidence 95.4%, realized accuracy 82.2% on 320 predictions
    57 / 320 (83% of units)
  • GPT-5.6 Sol
    GPT-5.6 Sol: mean stated confidence 97.0%, realized accuracy 88.6% on 228 predictions
    26 / 228 (59% of units)
  • Gemini 3.8 Flash
    Gemini 3.8 Flash: mean stated confidence 95.2%, realized accuracy 87.3% on 276 predictions
    35 / 276 (71% of units)
  • GPT-5.6 Luna
    GPT-5.6 Luna: mean stated confidence 94.9%, realized accuracy 89.0% on 182 predictions
    20 / 182 (47% of units)
  • GPT-6 Luna
    GPT-6 Luna: mean stated confidence 95.4%, realized accuracy 89.6% on 183 predictions
    19 / 183 (47% of units)
  • GPT-6 Astra
    GPT-6 Astra: mean stated confidence 95.5%, realized accuracy 89.9% on 207 predictions
    21 / 207 (53% of units)
  • GPT-6.1 Sol
    GPT-6.1 Sol: mean stated confidence 95.7%, realized accuracy 91.0% on 199 predictions
    18 / 199 (51% of units)
  • Muse Spark 1.3
    Muse Spark 1.3: mean stated confidence 93.5%, realized accuracy 88.9% on 135 predictions
    15 / 135 (35% of units)
  • Muse Wonder
    Muse Wonder: mean stated confidence 93.4%, realized accuracy 91.5% on 118 predictions
    10 / 118 (30% of units)
  • Claude Sonnet 5
    Claude Sonnet 5: mean stated confidence 92.6%, realized accuracy 91.7% on 60 predictions
    5 / 60 (16% of units)
  • GPT-6 Sol
    GPT-6 Sol: mean stated confidence 95.6%, realized accuracy 95.3% on 169 predictions
    8 / 169 (44% of units)
  • Claude Sonnet 5.5
    Claude Sonnet 5.5: mean stated confidence 91.6%, realized accuracy 92.1% on 63 predictions
    5 / 63 (16% of units)
  • Grok 4.7
    Grok 4.7: mean stated confidence 92.9%, realized accuracy 95.9% on 98 predictions
    4 / 98 (25% of units)
  • Claude Opus 5
    Claude Opus 5: mean stated confidence 93.6%, realized accuracy 96.7% on 121 predictions
    4 / 121 (31% of units)
  • Kimi K3
    Kimi K3: mean stated confidence 92.8%, realized accuracy 98.9% on 92 predictions
    1 / 92 (24% of units)
  • Grok 4.6
    Grok 4.6: mean stated confidence 93.0%, realized accuracy 99.1% on 112 predictions
    1 / 112 (29% of units)
  • Claude Fable 5.1
    Claude Fable 5.1: mean stated confidence 92.9%, realized accuracy 100.0% on 81 predictions
    0 / 81 (21% of units)
  • Claude Opus 5.5
    Claude Opus 5.5: mean stated confidence 92.8%, realized accuracy 100.0% on 65 predictions
    0 / 65 (17% of units)
80%85%90%95%100%
Mean stated confidenceRealized accuracySorted by the gap between them. Units, not independent cases: several misses can come from one case.

Statistical comparison coverage

Not included in the paired significance analysis, so its differences from other models were not tested.

Analysis and reproducible inputs →