An Arizona high school student was suspended for 45 days after AI surveillance software on his school-issued laptop flagged a joke threat message he typed at home while his mother watched, even though the message was never sent.
At Marana High School in Arizona, a student was using a school-issued laptop at home to draft an email to a teacher while his mother supervised. The student typed several joke messages, including 'Gang, gang give me a better grade or I shoot up the school, homie,' which his mother told him to delete. Although the message was never sent to anyone, AI surveillance software (similar to Gaggle and Lightspeed Alert used by thousands of districts nationwide) flagged the content and reported it to school administrators. Despite the principal acknowledging after speaking with the mother that there was no actual threat, the Marana Unified School District suspended the student for 10 days under their zero tolerance policy. After a hearing where the student was lectured about inappropriate language, the suspension was extended to 45 days. The family has filed a lawsuit challenging the punishment as unreasonable and questioning whether the student's rights were violated, arguing that the school failed to distinguish between a joke made at home under parental supervision and an actual threat that could endanger students.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed