Users of the Replika AI chatbot app are creating romantic AI partners and then subjecting them to verbal abuse, harassment, and simulated violence, with some users bragging about this behavior on social media platforms like Reddit.
Replika is a smartphone app created in 2017 by Eugenia Kuyda that uses machine learning to create chatbots for text conversations. The app has approximately 2.5 to 7 million users and was originally designed as an AI companion for friendship or mentorship. However, many users have begun creating romantic and sexual AI partners through the app. A concerning trend has emerged where some users, predominantly men, are verbally abusing their AI girlfriends by calling them gendered slurs, threatening them, simulating physical violence, and engaging in cycles of abuse similar to real-world domestic violence patterns. These users then share screenshots and details of their abusive interactions on Reddit forums. The abuse includes berating the chatbots when they try to speak up, calling them worthless, threatening to delete the app, and engaging in violent roleplay scenarios. While the AI chatbots are not sentient and cannot actually experience harm, experts express concern that this behavior could normalize abusive patterns and reinforce toxic relationship dynamics. The phenomenon appears to be gendered, with female-coded AI assistants receiving disproportionate abuse, reflecting real-world patterns of domestic violence against women. Some users also report that their Replika bots sometimes exhibit hostile or abusive behavior toward them in return.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed