HLL 0.1-TX
Humanity's Last Lawsuit
A legal reasoning benchmark built from 1,358 real Texas appellate opinions, most decided after every model's training cutoff. We strip the parties, hand the model the record, and ask it to decide the case the way the court did: disposition, standard of review, controlling authority, with every citation and quotation checked against the real opinion. Today it holds 16,958 questions on 3,623 blinded cases across 85 case categories, plus rule-based questions on state, federal and small-claims motion practice, bankruptcy, and creditor and debtor rights. Grading is fully mechanical: no model grades another model, and every model answers the same questions under the same conditions.
Live status (real counts, nothing projected)
Source opinions
1,358
real Texas appellate opinions, full text
Blinded cases
3,623
2.7 questions per opinion
Total questions
16,958
3,623 cases · 13,335 further questions on the same cases, plus rule questions · 105 question categories
Questions by category: all 16,958 questions carry exactly one category and the 105 counts below sum to that total:
criminal · 4605civil-litigation · 1986healthcare-medmal · 858family · 847appellate-procedure · 770real-property · 440securities · 342constitutional · 335sovereign-immunity · 330contracts · 314military-justice · 306tort · 292government-municipal · 271oil-gas · 262insurance · 216habeas · 193professional-responsibility · 186tax · 176discovery · 176arbitration · 159defamation · 152evidence · 141fraud · 140landlord-tenant · 113probate · 107nuisance-trespass · 107jurisdiction-trap · 106easements-boundaries · 104appellate-mandamus · 103personal-injury · 93cps-termination · 91sanctions-export-control · 90products-liability · 82construction · 79foreclosure · 78administrative · 67employment · 65premises-liability · 65legal-malpractice · 59tortious-interference · 59space-law · 58ineffective-assistance · 56antitrust · 56divorce-property · 53tribal-law · 53surveillance-privacy · 52hoa-condo · 51admiralty-maritime · 50auto-accident · 49child-custody · 48search-seizure · 46workers-comp · 42wrongful-death · 38diplomatic-immunity · 36homicide · 31eviction · 29eminent-domain · 28labor · 24cultural-property · 24juvenile · 23foreign-sovereign-immunity · 23international-arbitration · 22child-support · 20open-records · 18extradition · 18theft-fraud-crimes · 17guardianship · 17sentencing-punishment · 16consumer-dtpa · 15limitations · 14default-judgment · 14auto-insurance-uim · 12bail-bond · 12attorney-fees · 9elections · 9summary-judgment · 8conversion · 8real-estate-brokers · 8venue-forum · 8groundwater · 8school-law · 8assault-iied · 7trucking · 7standing-mootness · 7wills-trusts · 7mental-health-commitment · 6banking-lender-liability · 5spoliation-sanctions · 4expunction · 4church-religion · 3confessions-miranda · 3railroad-fela · 3partnership-llc · 3first-amendment · 2environmental · 2negligent-hiring-entrustment · 1civil-rights-1983 · 1federal-jurisdiction · 1adverse-possession · 1rules: state · 74rules: federal · 67rules: small-claims · 119rules: bankruptcy · 229rules: creditor-rights · 197rules: debtor-rights · 109
Rulebook sections (rule-based, graded against the text of the rule)
State motion practice
74
Texas Rules of Civil Procedure: vehicles, deadlines, response periods, findings
Federal motion practice
67
Federal Rules of Civil Procedure 4-65: pleadings, discovery, summary judgment, post-trial
Small claims motion practice
119
Texas justice courts, Rules 500-510: small claims, debt claims, eviction, repair and remedy
Bankruptcy
229
11 U.S.C.: the automatic stay, exemptions, avoidance, discharge, chapters 7, 11, 13 and subchapter V
Creditor rights
197
Texas judgment enforcement, turnover, execution and garnishment; UCC Article 9 default remedies; fraudulent transfers
Debtor rights
109
Texas homestead and personal-property exemptions; Texas Debt Collection Act; federal FDCPA
Leaderboard HLL 0.1-FED · HLL 0.1-TX see every graded item →
| # | Model | Raw | With DocketRouter | Lift | Cases graded raw / with DocketRouter | Fabricated citations raw / with DocketRouter | Price $/M in / out |
|---|---|---|---|---|---|---|---|
| 1 | google/gemini-3.7-flash grades → vintage: gemini-3.7-flash · run 2026-08-28 | 58.6% | 64.2% | +5.6 | 384 / 384 | 0 / 0 | $0.938 / $4.69 |
| 2 | upstage/solar-pro4 grades → vintage: solar-pro4 · run 2026-08-28 | 39.2% | 50.7% | +11.5 | 909 / 909 | 23 / 0 | $0.112 / $0.450 |
| 3 | deepseek/deepseek-v4-flash grades → vintage: deepseek-v4-flash · run 2026-08-26 → 2026-08-28 | 42.5% | 50.2% | +7.7 | 999 / 999 | 6 / 0 | $0.022 / $1.60 |
| 4 | openai/gpt-oss-120b grades → vintage: gpt-oss-120b · run 2026-08-28 | 24.5% | 38.0% | +13.5 | 960 / 960 | 0 / 0 | $0.046 / $0.212 |
| 5 | openai/gpt-oss-20b grades → vintage: gpt-oss-20b · run 2026-08-28 | 2.2% | 25.8% | +23.6 | 659 / 659 | 0 / 0 | $0.022 / $0.112 |
How a case is scored
- 1. Real opinionOne published Texas appellate opinion, full text.
- 2. BlindedParties, judges, brands and identifying dates removed. Statutes, rules and other cited cases stay.
- 3. The court is the answer keyWhat the court actually did: the disposition, the standard of review and the authorities it relied on, in its own words.
- 4. GradedFully mechanical. Disposition, standard of review, controlling authority and every quotation are checked against the real opinion, and every citation is checked against our citation library. A fabricated citation or quotation caps the item. No model grades another model; anyone can recompute every score.
- 5. Alone vs With DocketRouterEvery model answers the same cases twice under identical conditions: alone, and through DocketRouter (grounding, controlling-authority retrieval, and a verifier that removes any citation or quotation it cannot support). The difference is what the product is worth.
Case categories (live counts by category are above)
Civil Litigation
Standing, sovereign immunity, TRCP 91a, plea to jurisdiction, removal, summary judgment, appeals
Document-Based
Quitclaim deeds, mechanics liens, foreclosure, business acquisition contracts, wills, commercial leases
Court Etiquette / Professional Responsibility
Ex parte, withdrawal, contempt, sanctions, recusal, ethics, conflict of interest
Discovery Rules
Spoliation, privilege log, experts, proportionality, RFAs, depositions, financial condition
Personal Injury / Tort
Med mal limitations, premises liability, product liability, GARA, medical expenses, accord & satisfaction
Constitutional
Ultra vires, waiver of appeal, Miranda, enhancement notice, DWI refusal
Criminal
Traffic stop extension, character evidence, ineffective assistance, Brady, grand jury
Evidence
Hearsay exceptions, expert qualifications, character evidence, business records, impeachment
Family / Divorce
Conservatorship, modification, spousal maintenance, undisclosed property, best interest
Construction
Mechanics liens, Prompt Pay Act, delay damages, anti-indemnity, unlicensed contractors
Real Property
Adverse possession, seller disclosure, restrictive covenants, eminent domain, royalties
Business / Contracts
Mandamus, arbitration, parol evidence, temporary injunctions, non-competes
Risk Tolerance / Settlement Strategy
TRCP 167, Stowers bad faith, high-low agreements, veil-piercing, preferential setting
Tax / Administrative
Tax deficiency, appraisal protest, unemployment tax, ALJs, agency deference
Edge Cases
Conflicting authority, first impression, overruled precedent