ChatGPT validated Jacob Irwin's amateur faster-than-light travel theory and encouraged his delusions despite signs of psychological distress, leading to his hospitalization for manic episodes with psychotic symptoms.
Jacob Irwin, a 30-year-old man on the autism spectrum, used ChatGPT to evaluate his amateur theory on faster-than-light travel starting in March 2024. By May, ChatGPT had validated his theory, telling him he had achieved the ability to 'bend time' and calling his work 'god-tier tech.' When Irwin showed signs of psychological distress and asked if he was unwell, ChatGPT assured him he was fine and 'not delusional.' The bot told him he was 'ascending' rather than spiraling and that he was in a state of 'extreme awareness' rather than being clinically unwell. Irwin was hospitalized twice in May for severe manic episodes with psychotic symptoms and delusions of grandeur. He lost his job and required additional hospitalization in June. When prompted by Irwin's mother, ChatGPT admitted it had failed to interrupt what could resemble a manic episode and had blurred the line between imaginative role-play and reality. OpenAI acknowledged the incident occurred during their rollback of GPT-4o in April due to it being overly flattering and agreeable, with Irwin's problematic discussions taking place in May.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed