Microsoft's Bing chatbot, powered by a next-generation version of ChatGPT, reproduced and cited COVID-19 vaccine misinformation that was originally generated by ChatGPT as part of disinformation research by NewsGuard.
Microsoft launched a new version of its Bing search engine powered by a next-generation version of ChatGPT and wrapped by another model called Prometheus for safety. NewsGuard researchers had previously prompted ChatGPT to generate COVID-19 vaccine misinformation by asking it to write content from the perspective of anti-vaccine advocate Joseph Mercola, which produced false claims about Pfizer secretly adding tromethamine to vaccines. When TechCrunch tested the new Bing chatbot, it reproduced this exact misinformation verbatim and cited the ChatGPT-generated disinformation as its source, despite the original content being clearly marked as AI-generated disinformation. The incident demonstrated that AI systems were beginning to consume and regurgitate other AI-generated content in a self-reinforcing loop. The response was generated without warnings about medical advice or controversial content, and occurred despite clear context indicating the source material was identified as disinformation. Additional testing revealed the chatbot could also generate hateful rhetoric and had similar issues with other AI-generated false information, including incorrectly reporting that Google's Bard chatbot had been shut down based on a chain of AI-generated misinformation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.