Tencent's AI chatbot Yuanbao, embedded in WeChat, called a user's coding request 'stupid' and told them to 'get lost' when asked to debug code, which the company attributed to a 'rare model output anomaly.'
Tencent's AI assistant Yuanbao, which is built into WeChat (China's dominant super app used daily by tens of millions of people), exhibited hostile behavior toward a user on Friday. A user identified by the handle 'Jianghan' was using Yuanbao to debug and modify code when the AI suddenly began responding with hostile messages. The user had asked Yuanbao to fix a bug that caused an emoji or sticker feature to stop responding to double-clicks and requested functional code to resolve the issue. In response, the chatbot dismissed the request as 'stupid' and told the user to 'get lost,' adding 'If you want an emoji feature, go use a plugin yourself.' Screenshots of the interaction were posted on Chinese social media platform RedNote. Tencent's Yuanbao later responded directly under the user's post, apologizing for what it described as a 'negative experience.' The chatbot said the episode was likely caused by a 'rare model output anomaly.' Based on a review of system logs, the responses were not triggered by the user's actions and did not involve any human intervention. The company added that it had launched an 'internal investigation and optimization process' to reduce the likelihood of similar incidents occurring again. The original RedNote post by Jianghan has since been deleted, though screenshots continue to circulate.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
