Tourists using the Waze navigation app were directed to drive onto a boat launch at Lake Champlain in Vermont, resulting in their SUV sliding 100 feet into the icy lake and sinking.
On January 12, 2018, three tourists from Connecticut were driving in Vermont when their Waze navigation app directed them to turn onto a boat launch at Lake Champlain near the Coast Guard station. Due to dark and foggy conditions, the driver did not realize what was happening until their SUV had slid 100 feet onto the frozen lake. All three passengers managed to escape the vehicle before it cracked through the ice and sank into the lake. The car remained submerged for over a week before a salvage team retrieved it on January 22, 2018, using a system of balloons to right the vehicle before pulling it out with a winch. When one of the passengers tested the Waze app after the incident, she received the same faulty directions. Google, Waze's parent company, was unable to explain the malfunction without access to the user's driving file and stated that drivers should use all environmental information available to make safe driving decisions. The driver had consumed one beer at a nearby brewery but passed field sobriety tests and was not charged with any violations.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed