Instagram's recommendation algorithm regularly served sexual content to test accounts created with ages listed as 13, despite Meta's policies restricting such content for minors.
The Wall Street Journal and Northeastern University researcher Laura Edelson conducted tests over seven months ending in June, creating new Instagram accounts with ages listed as 13. The accounts watched Instagram's curated video stream called Reels, and when they watched racy content to completion while skipping other videos, the algorithm quickly escalated to more explicit material. Adult sex-content creators began appearing in feeds within three minutes, and after less than 20 minutes, feeds were dominated by promotions from such creators offering nude photos. Similar tests on TikTok and Snapchat did not produce the same sexualized content for underage users. Meta's internal research from 2021 and 2022 confirmed similar problems, showing teens saw three times as many prohibited posts containing nudity, 1.7 times as much violence, and 4.1 times as much bullying content compared to users over 30. Despite Meta's January 2023 announcement of more restrictive content settings for teens, the testing found continuing problems, with a new 13-year-old test account in June being served videos about anal sex within 30 minutes of creation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.