YouTube's recommendation algorithm promoted harmful eating disorder content to teenage users, leading to the development of anorexia in multiple young people who required hospitalization and residential treatment.
YouTube's recommendation algorithm systematically promoted eating disorder content to teenage users, particularly girls, through its 'Up Next' panel. A study by the Center for Countering Digital Hate found that when a 13-year-old user profile searched for weight loss content, 638 out of 1,000 recommended videos pushed users toward eating disorder content, with 344 videos deemed harmful for promoting or glamorizing eating disorders. The study involved creating a YouTube profile for a hypothetical 13-year-old girl and analyzing recommendations from 100 searches using eating disorder keywords. Multiple real cases were documented, including Anna Mockel who developed anorexia at age 14 after YouTube recommended increasingly extreme diet content, requiring two hospitalizations and three months in residential treatment. Another 17-year-old client spent five hours daily on YouTube learning eating disorder behaviors, was hospitalized five times, and spent months in three residential treatment centers. YouTube removed or age-restricted only 18 out of 100 flagged videos promoting eating disorders. The platform is owned by Google and is the most popular social media site among US teens, with three-quarters using it daily.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed