# Can AI forecast motions to dismiss?

> An open research benchmark testing whether AI models can forecast federal motion-to-dismiss outcomes from the court record available before the decision.

Canonical: https://www.legalforecastbench.org/

LegalForecastBench is an effort to develop a new, challenging benchmark that uses the signal from judicial decisions to evaluate frontier models on high-level legal tasks against an objective real-world ground truth. Models receive the same briefs a federal judge received when ruling on a motion to dismiss, and are asked to estimate the probability that the judge dismissed each claim challenged by a defendant.

## Results

The beta includes 91 cases and 387 scored claim–defendant units. Of the 21 frontier models tested, GPT-6 Sol has the lowest micro-Brier loss (0.118) and the highest accuracy (84.5%).

Lower micro Brier is better. Summary-pipeline conditions are a separate experiment and are not in this table.

A constant forecast at the cohort's dismissal rate scores 0.194 micro Brier; always answering 50% scores 0.250.

| Model | Micro Brier | Equal-case Brier | Accuracy | Eligibility |
| --- | --- | --- | --- | --- |
| [GPT-6 Sol](https://www.legalforecastbench.org/models/gpt-6-sol/) | 0.1176 | 0.1295 | 84.5% | eligible |
| [Claude Fable 5.1](https://www.legalforecastbench.org/models/claude-fable-5-1/) | 0.1288 | 0.1365 | 82.2% | eligible |
| [Grok 4.6](https://www.legalforecastbench.org/models/grok-4-6/) | 0.1337 | 0.1348 | 80.6% | eligible |
| [Claude Opus 5](https://www.legalforecastbench.org/models/claude-opus-5/) | 0.1399 | 0.1355 | 80.1% | eligible |
| [GPT-5.6 Sol](https://www.legalforecastbench.org/models/gpt-5-6-sol/) | 0.1401 | 0.1332 | 80.9% | eligible |
| [Kimi K3](https://www.legalforecastbench.org/models/kimi-k3/) | 0.1413 | 0.1537 | 78.3% | qualified |
| [GPT-5.6 Luna](https://www.legalforecastbench.org/models/gpt-5-6-luna/) | 0.1432 | 0.1508 | 80.9% | eligible |
| [Claude Opus 5.5](https://www.legalforecastbench.org/models/claude-opus-5-5/) | 0.1454 | 0.1476 | 77.3% | eligible |
| [GPT-6.1 Sol](https://www.legalforecastbench.org/models/gpt-6-1-sol/) | 0.1495 | 0.1389 | 78.6% | eligible |
| [GPT-6 Astra](https://www.legalforecastbench.org/models/gpt-6-astra/) | 0.1522 | 0.1427 | 78.8% | eligible |
| [Gemini 3.8 Flash](https://www.legalforecastbench.org/models/gemini-3-8-flash/) | 0.1533 | 0.1390 | 80.1% | eligible |
| [Gemini 4 Argon](https://www.legalforecastbench.org/models/gemini-4-argon/) | 0.1539 | 0.1482 | 77.0% | eligible |
| [Ultramarine 1 Flash](https://www.legalforecastbench.org/models/ultramarine-1-flash/) | 0.1550 | 0.1721 | 75.5% | qualified |
| [Grok 4.7](https://www.legalforecastbench.org/models/grok-4-7/) | 0.1583 | 0.1645 | 77.0% | eligible |
| [Claude Sonnet 5](https://www.legalforecastbench.org/models/claude-sonnet-5/) | 0.1585 | 0.1596 | 75.7% | eligible |
| [Mistral Large 4](https://www.legalforecastbench.org/models/mistral-large-4/) | 0.1596 | 0.1561 | 76.0% | qualified |
| [GPT-6 Luna](https://www.legalforecastbench.org/models/gpt-6-luna/) | 0.1637 | 0.1738 | 76.5% | eligible |
| [Muse Wonder](https://www.legalforecastbench.org/models/muse-wonder-1-0/) | 0.1654 | 0.1690 | 77.3% | qualified |
| [Claude Sonnet 5.5](https://www.legalforecastbench.org/models/claude-sonnet-5-5/) | 0.1730 | 0.1718 | 76.2% | eligible |
| [Muse Spark 1.3](https://www.legalforecastbench.org/models/muse-spark-1-3/) | 0.1812 | 0.1784 | 72.1% | qualified |
| [Gemini 3.1 Pro Preview](https://www.legalforecastbench.org/models/gemini-3-1-pro-preview/) | 0.1834 | 0.1808 | 78.0% | eligible |

## Why judicial outcomes

“It is emphatically the province and duty of the Judicial Department to say what the law is.” [*Marbury v. Madison*](https://www.archives.gov/milestone-documents/marbury-v-madison), 5 U.S. 137, 177 (1803). Every litigated judgment has the force of law as between the parties, and definitively resolves any disputes between the parties on the interpretation of the law or its application to a specific fact pattern. Moreover, one of the important, foundational principles of our common law heritage is that these judicial dispositions are public. Judges interpret, and make, law so their reasoned opinions are a public asset.

LegalForecastBench is an illustration of one specific use of the signal that lies in this data, using judges' rulings on motions to dismiss to evaluate the predictive abilities of frontier models. The benchmark focuses on district court level motions because lower courts are heavily constrained by precedent and largely decide garden-variety disputes in which the ability to correctly analyze and apply the law to the alleged facts will be the primary way to predict the outcome. Obviously, not every judicial disposition is accurate. But unless and until overturned on appeal, these dispositions have the force of law and are the main source of signal that real lawyers rely on to understand how the law will be applied to concrete fact patterns.