A Tesla Model 3 operating on Autopilot crashed into a semi-trailer crossing a highway in Florida, killing the 50-year-old driver when the AI system failed to detect the truck and the driver was not actively monitoring the road.
On March 1, 2019, at 6:17 a.m. in Delray Beach, Florida, a 2018 Tesla Model 3 equipped with Autopilot technology crashed into a semi-trailer that was crossing State Road 7. The Tesla driver, 50-year-old Jeremy Banner, had activated Autopilot approximately 10 seconds before the collision and was traveling at 68.4 mph when his vehicle struck the truck, which was crossing at about 11 mph. The Tesla's roof was sheared off as it under-rode the semi-trailer, and the vehicle continued for approximately 1,680 feet before stopping in the median. Banner was killed instantly, while the truck driver was uninjured. According to the NTSB investigation, Banner had not applied his hands to the steering wheel for 7.7 seconds before impact, indicating driver disengagement and over-reliance on the automation system. The Autopilot system failed to detect the crossing truck and did not provide any visual or audible warnings to alert the driver to regain control. The system's collision avoidance features also failed to activate due to design limitations. The NTSB determined that the truck driver failed to yield right of way, but also cited Banner's inattention due to over-reliance on automation and Tesla's failure to limit Autopilot use to appropriate conditions as contributing factors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed