Legal action was launched against the French government over its discriminatory welfare fraud detection algorithm that assigns risk scores to over 30 million people, disproportionately targeting disabled people and single mothers for intrusive investigations.
Since the 2010s, the French welfare agency CNAF has used a risk-scoring algorithm to detect welfare fraud and overpayments among more than 30 million people claiming government support. The algorithm assigns each person a score between 0 and 1 based on their likelihood of receiving fraudulent payments. Analysis of older versions of the algorithm by La Quadrature du Net revealed discriminatory criteria that increase risk scores for people with disabilities receiving AAH benefits who are working, and single-parent families. On October 15, 2024, a coalition of 15 human rights groups including Amnesty International filed legal action with France's top administrative court, alleging the algorithm violates European privacy rules and French anti-discrimination laws. People with higher risk scores are subject to stressful and intrusive investigations that can result in welfare payment suspensions. The groups argue the algorithm constitutes massive surveillance and disproportionately affects marginalized communities. The CNAF has not publicly shared the current algorithm's source code, though older versions from 2014-2020 have been analyzed. This case represents the first legal challenge to a public algorithm in France and could set precedent for similar welfare fraud detection systems across Europe.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed