ChatGPT fabricates trial testimony, leading to $5,000 lawyer fine

Using generative AI to condense long documents carries severe risks when accuracy is non-negotiable. The Supreme Court of New Mexico held defense attorney Stephen Aarons in direct contempt of court, fining him $5,000 and referring him to a disciplinary board after he submitted an appeal brief containing false testimony and fabricated witnesses created by ChatGPT.

While earlier court sanctions against lawyers involved fake legal precedents, this case centers on factual summarization. As detailed by Ars Technica, Aarons transcribed trial audio using an automated tool and fed the output alongside case files into ChatGPT, relying on OpenAI’s o3 model to produce a summary. Instead of merely condensing the facts, the AI model invented non-existent police testimony and hallucinated four fictitious witnesses.

Why did ChatGPT invent witnesses from real transcripts?

Large language models do not read documents like human analysts. When prompted to summarize extensive transcripts or complex discovery records, LLMs generate probabilistic text matches rather than strict database extractions. When context windows are saturated or prompts lack strict formatting boundaries, models synthesize plausible-sounding statements that have no basis in the input text.

In court, Aarons admitted he assumed ChatGPT would deliver an accurate summary because of AI’s growing adoption in technical fields. Justices rejected this defense, noting that attorneys bear full responsibility for verifying every claim they submit, regardless of whether work was performed by an assistant, an intern, or an algorithm.

What changes for professionals using AI summarization?

This incident sets a clear operational precedent for knowledge workers handling legal, compliance, or regulatory files:

• Summaries are drafts, not facts: AI summaries cannot be quoted or submitted into formal workflows without cross-referencing source material.
• Liability remains human: Disclaimers about AI capabilities do not protect against professional negligence or financial penalties.
• Double-check named entities: Names, dates, specific quotes, and numerical figures are the most frequent hallucination targets during text condensation.