Police departments across the United States used facial recognition systems with inappropriate inputs including celebrity photos, forensic sketches, and digitally altered images, leading to arrests based on unreliable matches and potential misidentifications.
Between 2017 and 2018, multiple police departments including the NYPD used facial recognition systems with problematic probe images that violated basic biometric principles. The NYPD's Facial Identification Section used a photo of actor Woody Harrelson when a CVS theft suspect's surveillance photo was too pixelated for the system to process, ultimately leading to an arrest of someone who was not Woody Harrelson. Police departments also submitted forensic sketches to facial recognition systems despite studies showing correct match rates of only 4.1-6.7% and conclusions that 'sketch searches mostly fail.' Additionally, officers digitally altered probe photos by adding facial features from other people, including pasting mouths from Google image searches and using clone tools to fabricate missing facial data. The NYPD made 2,878 arrests based on facial recognition searches in the first 5.5 years of using the technology. Florida agencies run approximately 8,000 facial recognition searches monthly. Defense attorneys typically do not receive information about the role facial recognition played in identifying their clients, creating due process concerns. Companies like Amazon, Cognitec, and Vigilant Solutions marketed these problematic practices as legitimate uses of their facial recognition systems.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed