Facebook's AI content moderation systems malfunctioned in November 2020, incorrectly blocking advertisements from thousands of small businesses for policy violations, causing significant financial losses during the critical holiday shopping season.
In November 2020, Facebook's artificial intelligence content moderation systems experienced widespread malfunctions that incorrectly flagged and blocked advertisements from thousands of small businesses worldwide. The incident began on November 11, 2020, when businesses like HoneyGramz (a New York honey business) and a permanent makeup clinic in Ottawa had their ad accounts suspended for alleged policy violations. Facebook had increased reliance on AI moderation because human moderators were focused on election and COVID-19 misinformation. The AI systems made numerous errors, including flagging onions as 'overtly sexual,' blocking solar companies for cryptocurrency violations, and suspending ads from pizza vending machines, water bottle companies, and coffee delivery services. Ruth Harrigan from HoneyGramz lost an estimated $5,000 in revenue before her account was restored after she contacted a Facebook executive on Twitter. The company's automated systems removed 22 million posts for hate speech in Q3 2020, three times more than the previous year, with a 40% increase in posts later restored due to errors. Facebook acknowledged the errors and apologized, stating they regularly work to improve their AI systems, but many small businesses experienced weeks or months of account suspensions during the crucial holiday shopping period.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed