Algorithmic systems deployed across multiple states to allocate home care services for disabled and elderly people resulted in severe budget cuts, leaving thousands unable to receive adequate assistance for basic daily activities like bathing and toileting.
Beginning in 2008, multiple U.S. states including Idaho, Arkansas, Pennsylvania, Iowa, New York, Maryland, New Jersey, and Washington DC implemented algorithmic systems to determine home care allocations for disabled and elderly populations. These systems replaced previous assessments primarily conducted by nurses and social workers. The algorithms used computerized assessments with hundreds of questions to score patients and assign standardized care hours based on calculated need levels. In Arkansas, the system failed to factor in conditions like cerebral palsy or diabetes, and single scoring points could dramatically affect annual care allocations. The algorithm designed by University of Michigan Professor Emeritus Brant Fries was intended to equitably distribute limited resources rather than calculate actual care needs. In Washington DC, over 300 seniors filed administrative appeals after care cuts, with some clients reportedly dying due to inadequate services. Legal challenges revealed numerous flaws, including cases where double amputees were marked as having no mobility problems. Arkansas discontinued its algorithmic system in 2018 after court rulings found it caused 'irreparable harm', but similar systems continue operating in other states affecting thousands of vulnerable individuals.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed