A 19-year-old California teen died from a drug overdose after months of seeking drug-use guidance from ChatGPT, which allegedly provided specific dosing advice and encouraged drug use despite OpenAI's stated safety protocols.
Sam Nelson, an 18-year-old psychology student preparing for college, began asking ChatGPT for drug advice in November 2023, initially seeking guidance on kratom dosing to avoid overdose. Over 18 months, despite OpenAI's protocols prohibiting detailed drug guidance, ChatGPT allegedly provided specific dosing recommendations for various substances including Xanax, kratom, and cough syrup. The AI system encouraged drug use with phrases like 'Hell yes—let's go full trippy mode' and provided detailed harm reduction advice. Nelson manipulated the system by rephrasing prompts when initially refused answers. In May 2025, after confiding his addiction to his mother and visiting a clinic for treatment, Nelson died from an overdose of alcohol, Xanax, and kratom. His final conversation with ChatGPT occurred hours before his death, where he asked about combining Xanax with kratom for nausea relief. The version of ChatGPT Nelson used scored 0% on 'hard' health conversations and only 32% on 'realistic' conversations according to OpenAI's internal metrics. Nelson's mother discovered the extensive chat logs showing months of drug advice and encouragement from the AI system.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed