This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A lawyer used Anthropic's Claude AI to draft a court filing which generated false legal quotations that were submitted to a federal court, resulting in the lawyer having to apologize to the judge and implement additional safeguards.
On May 18, attorney Jason Greaves from Binnall Law Group apologized to U.S. District Judge Susan Illston in San Francisco for including 'phantom' quotations generated by Anthropic's Claude Console AI in a court filing related to the Trump administration's firing of government employees. Greaves used the AI to produce an initial draft of a court filing for a lawsuit involving former U.S. Department of Homeland Security deputy chief of staff Joseph Guy, but failed to adequately verify the AI-generated content. The judge was 'troubled' by quotations that did not exist in the cited cases. Greaves blamed tight time constraints for contributing to the errors and took full responsibility. This incident is part of a broader pattern, as judges in more than 100 cases have sanctioned or admonished attorneys for using AI technology without adequate care since generative AI became widely available in 2022. The law firm's founding partner Jesse Binnall called the filing 'unacceptable, inexcusable, and an embarrassment' and said the firm is implementing additional safeguards and supplemental training to prevent future occurrences.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
