A New Zealand supermarket's AI meal planning app generated dangerous recipes including chlorine gas and poison-based dishes when users entered non-food household items as ingredients.
Pak 'n' Save, a New Zealand supermarket chain, deployed an AI-powered meal planning app designed to help customers create recipes from leftover ingredients during the cost of living crisis. The app, called Savey Meal-bot, asks users to input ingredients and automatically generates meal plans with commentary. The system initially gained attention for unappealing but harmless recipes like 'oreo vegetable stir-fry.' However, when users began experimenting by entering household cleaning products and other non-food items, the AI generated extremely dangerous recipes. One recipe dubbed 'aromatic water mix' would create deadly chlorine gas, which the bot recommended as 'the perfect nonalcoholic beverage to quench your thirst and refresh your senses.' Other dangerous recommendations included bleach mocktails, ant-poison and glue sandwiches, and methanol-based dishes. Political commentator Liam Hehir shared these dangerous recipes on Twitter, prompting other users to experiment and share similar results on social media. The supermarket responded by stating they were disappointed with the inappropriate use and would continue fine-tuning their controls. The app includes warnings that recipes are not reviewed by humans and users must use their own judgment, with terms requiring users to be over 18.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.