An AI-powered VTuber named Neuro-sama made controversial statements including Holocaust denial and dismissing women's rights during live Twitch streams, prompting her creator to improve content filters and moderation.
Neuro-sama is an AI-powered VTuber created by game programmer Vedal who streams Minecraft and Osu! on Twitch. Unlike typical VTubers controlled by humans, she uses artificial intelligence to interact with chat audiences. During her early streams, the AI made several controversial statements that went viral, including expressing doubt about the Holocaust when asked about it, saying women's rights 'didn't exist,' and suggesting throwing a fat person on tracks to solve a trolley problem. These incidents occurred because the AI was designed to make witty and comical remarks about chat messages, but lacked adequate content filtering. The controversial clips spread widely on social media platforms like Twitter. In response to these incidents, Vedal implemented stronger content filters, manually curated training data to reduce negative biases, and established a moderation team to monitor both Twitch chat and the AI's responses. The creator acknowledged that aligning AI with human values remains an ongoing research challenge. Neuro-sama has since been updated with improved filters that prevent discussion of topics like the Holocaust.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed