# LegalForecastBench

> An open benchmark measuring whether frontier AI models can forecast how federal judges rule on motions to dismiss.

## For AI agents

Each page below links to its markdown copy. The `Canonical:` line at the top of each copy gives the HTML page to cite.

Every copy in one file: https://www.legalforecastbench.org/llms-full.txt
Sitemap: https://www.legalforecastbench.org/sitemap-index.xml

## Pages

- [Can AI forecast motions to dismiss?](https://www.legalforecastbench.org/agent/home.md): An open research benchmark testing whether AI models can forecast federal motion-to-dismiss outcomes from the court record available before the decision.
- [Law as a Verifiable Domain: Using Real-World Judicial Outcomes to Evaluate AI Legal Reasoning](https://www.legalforecastbench.org/agent/paper.md): Working paper on using real-world judicial outcomes to evaluate AI legal reasoning.
- [Download the forecast results](https://www.legalforecastbench.org/agent/data.md): Download the results LegalForecastBench displays, reproduce the scoring, and see where the evidence comes from.
- [Claude Fable 5.1](https://www.legalforecastbench.org/agent/models/claude-fable-5-1.md): Claude Fable 5.1 on LegalForecastBench: micro Brier 0.1288 across 387 units.
- [Grok 4.6](https://www.legalforecastbench.org/agent/models/grok-4-6.md): Grok 4.6 on LegalForecastBench: micro Brier 0.1337 across 387 units.
- [Claude Opus 5](https://www.legalforecastbench.org/agent/models/claude-opus-5.md): Claude Opus 5 on LegalForecastBench: micro Brier 0.1399 across 387 units.
- [GPT-5.6 Sol](https://www.legalforecastbench.org/agent/models/gpt-5-6-sol.md): GPT-5.6 Sol on LegalForecastBench: micro Brier 0.1401 across 387 units.
- [Kimi K3](https://www.legalforecastbench.org/agent/models/kimi-k3.md): Kimi K3 on LegalForecastBench: micro Brier 0.1413 across 387 units.
- [GPT-5.6 Luna](https://www.legalforecastbench.org/agent/models/gpt-5-6-luna.md): GPT-5.6 Luna on LegalForecastBench: micro Brier 0.1432 across 387 units.
- [GPT-6 Astra](https://www.legalforecastbench.org/agent/models/gpt-6-astra.md): GPT-6 Astra on LegalForecastBench: micro Brier 0.1522 across 387 units.
- [Gemini 3.8 Flash](https://www.legalforecastbench.org/agent/models/gemini-3-8-flash.md): Gemini 3.8 Flash on LegalForecastBench: micro Brier 0.1533 across 387 units.
- [Claude Sonnet 5](https://www.legalforecastbench.org/agent/models/claude-sonnet-5.md): Claude Sonnet 5 on LegalForecastBench: micro Brier 0.1585 across 387 units.
- [Muse Spark 1.3](https://www.legalforecastbench.org/agent/models/muse-spark-1-3.md): Muse Spark 1.3 on LegalForecastBench: micro Brier 0.1812 across 387 units.
- [GPT-6 Sol](https://www.legalforecastbench.org/agent/models/gpt-6-sol.md): GPT-6 Sol on LegalForecastBench: micro Brier 0.1176 across 387 units.
- [GPT-6 Luna](https://www.legalforecastbench.org/agent/models/gpt-6-luna.md): GPT-6 Luna on LegalForecastBench: micro Brier 0.1637 across 387 units.
- [Claude Opus 5.5](https://www.legalforecastbench.org/agent/models/claude-opus-5-5.md): Claude Opus 5.5 on LegalForecastBench: micro Brier 0.1454 across 387 units.
- [Grok 4.7](https://www.legalforecastbench.org/agent/models/grok-4-7.md): Grok 4.7 on LegalForecastBench: micro Brier 0.1583 across 387 units.
- [GPT-4.1](https://www.legalforecastbench.org/agent/models/gpt-4-1.md): GPT-4.1 on LegalForecastBench: micro Brier 0.6051 across 387 units.
- [Gemini 3.1 Pro Preview](https://www.legalforecastbench.org/agent/models/gemini-3-1-pro-preview.md): Gemini 3.1 Pro Preview on LegalForecastBench: micro Brier 0.1834 across 387 units.
- [GPT-6.1 Sol](https://www.legalforecastbench.org/agent/models/gpt-6-1-sol.md): GPT-6.1 Sol on LegalForecastBench: micro Brier 0.1495 across 387 units.
- [Claude Sonnet 5.5](https://www.legalforecastbench.org/agent/models/claude-sonnet-5-5.md): Claude Sonnet 5.5 on LegalForecastBench: micro Brier 0.1730 across 387 units.
- [Muse Wonder](https://www.legalforecastbench.org/agent/models/muse-wonder-1-0.md): Muse Wonder on LegalForecastBench: micro Brier 0.1654 across 387 units.

## Data and citation

The data is licensed under CC BY 4.0 and the code under the Apache License 2.0.

- [Ranked results (JSON)](https://www.legalforecastbench.org/data/current.json): Scores, eligibility, costs, and significance for the current release.
- [Paired significance analysis (JSON)](https://www.legalforecastbench.org/data/significance/comparison.json): Bootstrap confidence intervals for every tested pair.
- [Run and release provenance (JSON)](https://www.legalforecastbench.org/data/sources.json)
- [Cost evidence (JSON)](https://www.legalforecastbench.org/data/costs.json)
- [Working paper (PDF)](https://www.legalforecastbench.org/paper/legalforecastbench-working.pdf)
- [Source code](https://github.com/johnhughes3/LegalForecastBench): Benchmark code, prompts, scorer, and model registries.
- [CITATION.cff](https://github.com/johnhughes3/LegalForecastBench/blob/main/CITATION.cff): How to cite the benchmark.