Meta's AI content moderation system wrongfully disabled multiple Instagram, Facebook, and WhatsApp accounts by falsely flagging users for violating community standards on child sexual exploitation, abuse and nudity.
Starting around July 2, 2023, multiple Meta platform users across the US received notifications that their Instagram, Facebook, and WhatsApp accounts had been disabled for allegedly violating community standards on child sexual exploitation, abuse and nudity. The affected users, including a pediatric ICU nurse from Azusa named Natalia and a Long Beach resident named Marty Macias, reported that the accusations were completely unfounded. When Natalia's account was disabled, her boyfriend Gavino Cervantes' professional Instagram account for his general contracting business was also disabled because their emails were connected. Cervantes reported losing approximately five potential customers who had been messaging him for quotes on the day his business account was taken down. The affected users submitted appeals but had not received responses from Meta. All Meta-owned platforms (Instagram, Facebook, WhatsApp) were disabled for these users, causing them to lose years of personal memories and business communications. Users suspected that an AI program had wrongfully flagged their accounts, and USC Communications Professor Karen North commented that this appeared to be an issue with over-reliance on early AI rollout. Meta had not responded to ABC7's request for comment, and users reported having no access to customer service support to resolve the issue.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed