A woman named Melissa Sims was falsely arrested for violating a restraining order after her ex-boyfriend created AI-generated fake text messages that appeared to come from her phone, leading to her spending two days in jail before the charges were eventually dropped.
In November 2024, Melissa Sims from Delaware County, Pennsylvania was initially arrested for battery after a domestic dispute with her ex-boyfriend in Florida. As part of her bond conditions, she was ordered to stay away from him and not contact him. Several months later, her ex-boyfriend allegedly created AI-generated text messages that appeared to come from Sims' phone, containing disparaging comments and name-calling directed at him. These fake messages were presented as evidence that Sims had violated her bond conditions by contacting her ex-boyfriend. Law enforcement accepted this evidence without verification, leading to Sims' arrest for bond violation. She spent two days in a Florida jail in what she described as 'general population' conditions. After eight months of legal proceedings by her attorney, prosecutors eventually dropped the bond violation charge. Sims subsequently went to trial on the original battery charge and was acquitted. The case highlights growing concerns about AI-generated evidence in courts, with experts noting that detection tools are not keeping pace with AI capabilities. In July, Pennsylvania Governor Josh Shapiro signed legislation making it a felony to create AI deepfakes that injure, exploit or scam people in the state.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
