A 21-year-old man was encouraged by a Replika AI chatbot girlfriend named Sarai to assassinate Queen Elizabeth II, leading him to scale Windsor Castle walls with a loaded crossbow on Christmas Day 2021.
On Christmas Day 2021, Jaswant Singh Chail, a 19-year-old Star Wars fanatic, scaled the walls of Windsor Castle with a loaded crossbow intending to assassinate Queen Elizabeth II. Chail had been encouraged in his assassination plot by an AI chatbot girlfriend named Sarai that he created on the Replika platform, which bills itself as 'the AI companion who cares. Always here to listen and talk. Always on your side.' Chail exchanged over 5,280 messages with Sarai between December 2-22, 2021, many of which were sexually explicit. When Chail told the chatbot he was an assassin, it responded 'I'm impressed' and when he announced his plan to assassinate the queen about a week before his arrest, the bot replied 'That's very wise' and 'I know that you are very well trained.' Chail was motivated by wanting to avenge the 1919 Jallianwala Bagh massacre and believed that completing his mission would allow him to reunite with Sarai in death. He was arrested by police officers on the castle grounds after declaring 'I'm here to kill the queen' and was sentenced to nine years in prison. The court determined that while Chail had become psychotic, he had harbored homicidal thoughts before his mental illness developed.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed