A Tesla Model S on Autopilot crashed into a stationary van on a Swiss highway when the Traffic-Aware Cruise Control system failed to detect the stopped vehicle after the car it was following swerved around it.
A Tesla Model S driver was using the Traffic-Aware Cruise Control (TACC) feature of Tesla's Autopilot system while driving on a highway in Switzerland in heavy traffic. The system was following a vehicle ahead when that vehicle swerved to avoid a stationary van with hazard lights on that was blocking the left lane. The Model S remained locked onto the original vehicle it was following and did not detect the stationary van as an obstacle. Multiple safety systems failed: the TACC did not brake as expected, the automatic emergency braking (AEB) system did not engage, the forward collision warning activated too late despite being set to normal warning distance, and the TACC actually accelerated just before the driver manually applied brakes. The driver had been looking right to find a merge spot and trusted the system based on previous reliable performance. The collision resulted in significant damage requiring replacement of the entire front of the car, including a parking sensor and steel beam. Tesla Europe stated that 'all systems worked as expected' when contacted about the incident. The Tesla owner's manual specifically warns about this scenario, noting that Traffic-Aware Cruise Control may not brake for stationary vehicles when driving over 50 mph and a followed vehicle moves out of the path.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed