Pieces Technologies, a Dallas-based healthcare AI company, was found by the Texas Attorney General to have made false and misleading claims about the accuracy of its AI products used for clinical documentation, potentially harming patients who relied on the inaccurate outputs.
On September 18, the Texas Attorney General announced a settlement with Pieces Technologies, Inc., a Dallas-based healthcare AI research and development firm, resolving allegations of false and misleading claims about AI product accuracy. Pieces develops healthcare AI products for inpatient healthcare facilities, including autonomous AI-generated clinical documentation meant to be relied upon by physicians and medical staff for patient treatment. The company marketed its products as having minimal 'hallucinations' with claims of being 'highly accurate' and having a 'severe hallucination rate' of less than 0.001% or less than 1 per 100,000. These representations led four major Texas hospitals to provide patient healthcare data to Pieces to obtain AI outputs consisting of patient condition summaries and treatment plans. The Texas AG maintained that Pieces' advertisements were inaccurate and may have deceived healthcare teams about the accuracy and safety of services, potentially putting public interest at risk. The settlement requires clear disclosures in marketing, prohibits misrepresentations about product accuracy, and mandates disclosure of potentially harmful uses of products or services for five years.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.