AI-generated deepfake videos and articles featuring medical professionals were used to promote fraudulent health supplements, leading to financial losses and dangerous health decisions by consumers who stopped prescribed medications.
Starting approximately 18 months ago, scammers created AI-generated deepfake videos and articles featuring medical professionals including Dr. Norman Swan and diabetes researcher Jonathan Shaw to promote unproven health supplements. The deepfakes were sophisticated, using AI voice cloning and facial mapping technology that could be created with basic software for a few dollars. These fake advertisements appeared on Facebook and Instagram, targeting vulnerable consumers with products like Glyco Balance and Keto Flow + ACV. The scams led to direct financial harm, with victims like Janis losing over $300 and being signed up for unauthorized monthly debits. More seriously, the fake content encouraged diabetes patients to stop taking prescribed medications like metformin in favor of unproven supplements. David Bell, a diabetes patient, ceased his prescribed medication after reading a convincing fake article attributed to Dr. Swan. The scammers operated through registered businesses including Vellec Group NZ Ltd, Healthy Life Choices NZ Ltd and Apex United Pty Ltd, with offices in Christchurch and the Gold Coast. Meta removed over a billion fake accounts in Q4 2024 but acknowledged a 0.3% miss rate. The Australian Competition and Consumer Commission reported over $2 billion lost to scams in 2024, representing a 25% decline from previous years.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed