Facebook's downranking system designed to suppress misinformation malfunctioned for six months, instead boosting harmful content including misinformation, nudity, violence, and Russian state media by up to 30% to potentially half of all News Feed views.
Between October 2021 and March 11, 2022, Facebook's AI-powered content downranking system experienced a software bug that caused it to malfunction in the opposite way to its intended purpose. Instead of suppressing posts from repeat misinformation offenders flagged by external fact-checkers, the system was boosting their distribution by as much as 30% globally. The bug, which originated in 2019 but didn't create noticeable impact until October 2021, affected up to half of all News Feed views and exposed users to what Facebook internally called 'integrity risks.' During this period, the system also failed to properly demote nudity, violence, and Russian state media content, including during Russia's invasion of Ukraine. Facebook engineers first noticed the issue when a sudden surge of misinformation began flowing through News Feeds but were unable to identify the root cause for months. The company designated this a level-one site event, reserved for high-priority technical crises. Meta spokesperson Joe Osborne confirmed five separate instances of 'inconsistencies in downranking' but claimed the bug 'has not had any meaningful, long-term impact on our metrics' and didn't affect content that met deletion thresholds.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed