A Department for Work and Pensions automated algorithm wrongly flagged over 200,000 housing benefit claimants as potentially fraudulent over three years, with two-thirds of investigated cases proving to be legitimate.
The Department for Work and Pensions (DWP) deployed an automated algorithm to identify potentially fraudulent housing benefit claims by analyzing claimants' personal characteristics including age, gender, number of children, and tenancy agreement type. The system does not use artificial intelligence or machine learning but was designed to flag high-risk claims for manual review by council staff. Over a three-year period from 2020-2023, the algorithm flagged claims as potentially fraudulent, but subsequent investigations revealed that only 34-37% of flagged cases actually involved fraud or error, compared to the 64% fraud rate identified during the pilot phase. This meant that approximately 200,000 people were wrongly investigated for housing benefit fraud. The system cost £4.4 million in investigation expenses that did not result in savings. Despite the poor accuracy, the DWP reported that every pound spent on case reviews returned £2.71 in savings according to 2021/22 figures. The data was obtained by civil liberties group Big Brother Watch through Freedom of Information requests. The DWP has expanded its use of AI systems across other benefit programs including universal credit despite warnings about algorithmic bias against vulnerable claimants.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed