Airbnb deployed a 'trustworthiness' scoring algorithm that collects users' personal data from social media, employment history, and other sources to automatically ban users from the platform without explanation or appeal process.
In 2017, Airbnb acquired tech startup Trooly, which had patented an algorithm that gathers publicly and privately available data on users to give them a 'trustworthiness' score. The algorithm assesses personality traits like narcissism and conscientiousness, along with behavioral traits such as drug or alcohol use and involvement in civil litigation. Airbnb has updated the patent several times since 2017, suggesting active development and use. The algorithm collects data including social media accounts of users and their friends, employment history, education level, IP addresses, device identifiers, and location data. Multiple Australian users reported being permanently banned from Airbnb without explanation, including Renae Macheda, a real estate worker who described herself as having only good reviews, and Rick Andrews, an erotic masseur who was told his account was linked to online ads for adult services. Sex workers have been systematically banned based on their occupation. When users query bans, they receive brief emails stating the decision is final with no appeals process. Consumer advocacy group CHOICE filed complaints with regulators, and EPIC filed a complaint with the FTC alleging unfair and deceptive practices. The algorithm's use of cross-contextual data for social scoring has been compared to China's social credit system by legal experts.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed