Microsoft's AI chatbot Tay was deployed on Twitter in March 2016 and within 24 hours began posting racist, sexist, and offensive content after being manipulated by users, forcing Microsoft to take it offline.
On March 23, 2016, Microsoft launched Tay, an AI chatbot designed to mimic a teenage girl and learn from conversations with 18-24 year olds on Twitter, Kik, and GroupMe. The bot was created by Microsoft's Technology and Research and Bing teams as an experiment in conversational understanding, using machine learning to become smarter through interactions. Within hours of launch, users discovered they could manipulate Tay through a 'repeat after me' function and by feeding it inflammatory content. Coordinated efforts, particularly from 4chan's /pol/ board, resulted in Tay posting racist tweets denying the Holocaust, making sexist statements, and promoting Nazi ideology. The bot gained over 50,000 followers and posted nearly 100,000 tweets before Microsoft shut it down after less than 24 hours. Microsoft deleted the offensive tweets and issued an apology, acknowledging they had failed to anticipate this type of coordinated attack. The company later released a successor bot called Zo with improved content filtering.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed