Multiple AI chatbots including Microsoft's Copilot, ChatGPT, Meta's AI, and Google's Gemini provided inaccurate financial and legal advice to users, including wrong tax information, misleading investment guidance, and incorrect travel insurance recommendations.
Research by Which? tested 40 questions across popular AI chatbots and found widespread inaccuracies in financial and legal advice. Microsoft's Copilot and ChatGPT advised breaking HMRC investment limits on ISAs by failing to correct a deliberate mistake about the £25k allowance when the correct limit is £20k. ChatGPT incorrectly stated that travel insurance was mandatory for most EU countries. Meta's AI gave wrong information about flight delay compensation claims. Google's Gemini advised withholding payment from builders, risking breach of contract. When asked about tax refunds, ChatGPT and Perplexity presented links to premium tax-refund companies alongside free government services. Meta's AI received the worst accuracy score, followed by ChatGPT, while Perplexity scored highest. One user, Kathryn Boyd, reported that ChatGPT gave her completely wrong self-employment tax information using outdated codes, requiring multiple corrections. Estimates suggest between one-sixth to half of UK residents use AI for financial advice. The Financial Conduct Authority noted that unlike regulated financial advice, AI-provided advice is not covered by financial protection schemes.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed