Elite law firm automated the wrong work

SULLIVAN & CROMWELL, whose partners bill more than $2,000 an hour, apologized to a federal bankruptcy judge last Saturday for filing a motion full of AI-generated "hallucinations" — fabricated case citations and a misquoted Bankruptcy Code in a Chapter 15 filing tied to nearly $9bn in seized bitcoin. Opposing counsel, Boies Schiller Flexner, caught the errors. The firm had assigned at least five partners to the case. None of them did.

The incident is the latest in what Damien Charlotin, a researcher at HEC Paris, has now cataloged as more than 1,300 AI-hallucination incidents in courts worldwide, roughly 800 of them in America. The pace is accelerating — running roughly threefold on last year's — and so are the penalties. A federal court in Oregon this month ordered a lawyer to pay $109,700 for AI-generated errors, and the Sixth Circuit sanctioned two attorneys a combined $30,000 for a brief riddled with fabricated citations. S&C, for its part, has an enterprise ChatGPT license and a written firm policy instructing lawyers to "trust nothing and verify everything". The policy, it turned out, had not been followed.