The UK Home Office deployed an AI-powered algorithm in 2019 to automatically triage marriage applications and flag potential 'sham marriages' for investigation, with the system showing discriminatory patterns against certain nationalities including Bulgarian, Greek, Romanian and Albanian applicants.
The UK Home Office implemented an automated triage system in April 2019 to detect potential sham marriages using machine learning algorithms. The system processes marriage applications where one or both partners are from outside the European Economic Area, lack settled status, or have insufficient visa documentation. The algorithm assigns couples either a 'green light' (no investigation needed) or 'red light' (requires investigation) rating based on eight undisclosed criteria. Internal government documents obtained by the Public Law Project revealed that the system flags 20-25% of marriages involving Bulgarian, Greek, Romanian and Albanian nationals for investigation. An equality impact assessment conducted by the Home Office identified potential indirect discrimination based on age differences between partners. The system was trained on historical Home Office marriage referral data from an unspecified three-year period. Red-flagged couples face invasive investigations including interviews and home visits, with potential consequences including deportation and prevention of marriage if they refuse to cooperate. Legal advocates argue the system perpetuates existing biases in Home Office enforcement data and lacks transparency in its decision-making criteria.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.