Apple and its security contractor SIS used facial recognition technology that falsely identified Ousmane Bah as a serial Apple Store thief across multiple states, leading to his arrest and prosecution for crimes committed by someone else using his stolen learner's permit.
Between April 2018 and December 2018, Apple and Security Industry Specialists (SIS) used facial recognition technology to identify suspected thieves across Apple Stores in multiple states including New York, New Jersey, Massachusetts, and Connecticut. The system incorrectly linked 18-year-old Ousmane Bah to a series of thefts actually committed by Mamadou Barrie, who had been using Bah's lost temporary learner's permit as identification. The facial recognition system matched Barrie's face from surveillance footage with Bah's personal information, creating false security records and BOLO notices that were circulated to Apple Stores and law enforcement. Despite physical discrepancies between the two individuals (Barrie was 6'1' while Bah was 5'7'), Apple and SIS continued to identify Bah as the perpetrator across multiple incidents. This led to Bah being arrested at his home by NYPD in November 2018, charged in multiple jurisdictions, and forced to defend himself against false allegations. The misidentification was only discovered when police officers noted that Bah looked nothing like the person in the surveillance footage and arrest warrant photo. Bah filed a $1 billion lawsuit against Apple and SIS, claiming the false accusations caused severe stress, damaged his reputation, and affected his education. Most charges were eventually dropped, though some cases remained pending.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed