Uber's Real-Time ID Check system, which uses Microsoft Cognitive Services facial recognition technology, repeatedly suspended transgender drivers' accounts when their appearance changed during gender transition, causing work disruptions and financial losses.
Uber deployed a security feature called Real-Time ID Check in September 2016 that uses Microsoft Cognitive Services facial recognition technology to verify driver identity. The system occasionally prompts drivers to take selfies that are compared to photos on file, suspending accounts if no match is detected. Transgender drivers experiencing physical changes during gender transition have been disproportionately affected by false flags from this system. Janey Webb, who began transitioning in October 2017, had her account deactivated just before July 4, 2018, despite uploading updated photos throughout her transition. She was forced to drive two hours to an Uber Greenlight Hub for manual verification, losing three days of work including a high-earning holiday. Another transgender driver, Lindsay from Michigan, reported being prompted for identity verification approximately 100 times over 18 months. Uber confirmed Webb's case occurred but stated such issues can often be resolved remotely. The company has not provided statistics on average verification prompts or implemented features to accommodate gender transitions, though it added pronoun options for internal employee directories in April.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed