Artificial Intelligence

OpenAI tunes Astra to cut legal hallucinations

OpenAI says updated Astra for Law leads legal benchmarks with fewer hallucinations and wider Enterprise access.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

3 min read

Lawyer reviewing AI drafted legal brief on screen
Credit: OpenAI News

OpenAI says its updated Astra for Law cuts hallucinations for lawyers and leads on three legal tests.

A bad citation can sink a brief. That is the problem OpenAI is targeting with a legal configuration, which means tuned instructions and tools on top of a base model.

Astra for Law is a configuration of GPT-6.2 Astra. OpenAI says it combines the base model with an expanded legal search index. It adds instructions for researching a question, weighing authorities and writing the answer. Since September, OpenAI says it grew that index by 34%. The index now spans over 310 million URLs, as of the November 17 announcement. It covers more than 99.9% of published U.S. precedential case law, according to OpenAI. Harvey maintains LAB as an open-source project for testing agents on realistic legal work, per its GitHub page.

On LAB-AA, which is Artificial Analysis’s version of Harvey’s test for handling legal assignments start to finish, OpenAI says Astra for Law earned full credit on 20.8% of 120 assignments. OpenAI says base GPT-6.2 Astra earned 15.0%. OpenAI says Muse Wonder Full earned 13.1%. Astra for Law averaged about 64,000 output tokens per assignment, according to OpenAI. OpenAI says Muse Wonder Full averaged 139,000. That gap matters for teams paying by usage. The limit is clear too. Even at 20.8%, four in five assignments still missed full credit, according to OpenAI’s own scores.

Hallucinations are the sharper test. OpenAI says Astra for Law averaged 0.03 material hallucinations per assignment. OpenAI says that tied GPT-6 for lowest in the comparison. OpenAI says Claude Opus 5.5 averaged 0.59. On brief writing, Julio Pereyra, Head of Legal Research at Harvey, said the new version writes briefs preferred to GPT-6 Astra nearly 90% of the time by pairwise judges. OpenAI says Astra for Law scored 1,844 to 1,817 for Opus 5.5 when models had to find cases themselves. Given the precedents, OpenAI says the scores were effectively even at 1,883 and 1,882.

On Vals AI’s Legal Research Bench, OpenAI says full-credit answers rose from about 46% to 53.8% in under two months. That was 112 of 208 questions, according to OpenAI. OpenAI says it earned 90.9% of available points including partial credit. Early users include Cooley, Ropes & Gray and Sullivan & Cromwell, according to OpenAI. John Savva, Partner at Sullivan & Cromwell, said early versions showed strong research depth and sensitivity to authority. Every ChatGPT Enterprise customer can now request access through the Trusted Access Program, according to OpenAI. Watch whether that larger rollout surfaces failures the benchmarks miss.

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsOpenAIAstra for LawGPT-6.2 AstraHarveyGrubelVals AISullivan & CromwellCooley

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.