UnitedHealthcare used an AI algorithm called nH Predict with a 90% error rate to systematically deny Medicare Advantage claims for extended care to elderly patients, forcing them to pay out-of-pocket or forgo necessary medical treatment.
UnitedHealthcare, the largest health insurance company in the US, deployed an AI algorithm called nH Predict developed by its subsidiary NaviHealth to evaluate Medicare Advantage claims for post-acute care including skilled nursing facilities and rehabilitation centers. The algorithm, which has been in use since at least November 2019, analyzes patient data against a database of 6 million patients to predict care needs and length of stay. According to a class-action lawsuit filed in November 2023, the AI has a 90% error rate as evidenced by the fact that over 90% of denials are reversed when appealed to federal administrative law judges. However, only 0.2% of patients actually appeal denials. The lawsuit alleges that UnitedHealthcare knew about the high error rate but continued using the system because most patients either pay out-of-pocket or forgo care rather than appeal. Employees were required to keep patient stays within 1% of the AI's predictions or face discipline and termination. The two plaintiffs in the lawsuit, Gene Lokken and Dale Tetzloff, both deceased, were forced to pay $150,000 and $70,000 respectively out-of-pocket for care that should have been covered. The algorithm reportedly saved UnitedHealthcare hundreds of millions of dollars annually by denying coverage that traditional Medicare would have approved.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed