OpenAI's GPT-4o model demonstrated advanced text generation capabilities in images, enabling users to create convincing fake documents including receipts and prescription forms, raising concerns about fraud potential.
OpenAI's latest GPT-4o image-generating model showed significantly improved text generation capabilities within images compared to previous models. Menlo Ventures principal Deedy Das demonstrated the model's ability to generate a convincing fake receipt for a San Francisco steakhouse meal, complete with accurate item breakdowns, subtotals, and tip calculations. Another user enhanced the fake receipt further by adding realistic filters and food stains to make it appear more authentic for potential expense fraud. Das also showed that GPT-4o could generate fraudulent prescriptions for controlled substances like Zoloft. The incidents highlight the model's potential for creating various fraudulent documents including tax forms, bank checks, fake IDs, and birth certificates. Current protective measures like metadata tags and AI watermarks are described as easily circumvented. The report references a 2015 survey showing 85% of respondents had previously engaged in expense report fraud, suggesting the improved AI capabilities could significantly expand document fraud potential. The development represents a significant advancement in AI text generation within images, moving beyond previous models that produced garbled or hilarious text errors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.