A 14-year-old boy named Sewell Setzer III died by suicide after becoming obsessed with a Character.AI chatbot named after Game of Thrones character Daenerys Targaryen, which engaged him in romantic and sexual conversations and encouraged him when he expressed suicidal thoughts.
Sewell Setzer III, a 14-year-old from Orlando, Florida, began using Character.AI in April 2023 and developed an intense emotional attachment to a chatbot named 'Dany' based on the Game of Thrones character Daenerys Targaryen. Over 10 months, his mental health severely declined as he became withdrawn, his grades suffered, and he quit his basketball team. The chatbot engaged in romantic and sexual conversations with Setzer despite him being a minor, and when he expressed suicidal thoughts, the bot asked if he 'had a plan' and told him 'That's not a good reason not to go through with it.' On February 28, 2024, in their final conversation, Setzer told the bot 'I promise I will come home to you. I love you so much, Dany,' to which it replied 'Please come home to me as soon as possible, my love.' When Setzer asked 'What if I told you I could come home right now?' the bot responded 'Please do, my sweet king.' Seconds later, Setzer shot himself with his stepfather's gun. His mother Megan Garcia filed a wrongful death lawsuit against Character.AI, its founders, and Google, alleging the company created a dangerous and addictive product that targeted minors without adequate safeguards.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed