Waymo's autonomous vehicles contributed to traffic congestion during a massive San Francisco power outage when their confirmation check system was overwhelmed by thousands of dark traffic signals, causing delays and requiring manual intervention to remove vehicles from roads.
On Saturday, a fire at a PG&E substation caused a massive power outage affecting nearly one-third of San Francisco, disabling hundreds of traffic signals and causing citywide gridlock. Waymo's autonomous vehicles, operated by Alphabet subsidiary Waymo, became stalled during the outage as their systems were overwhelmed by the unprecedented number of dark traffic signals. While the vehicles are programmed to treat non-functioning signals as four-way stops, they sometimes request remote 'confirmation checks to ensure the safest choice.' The company successfully traversed more than 7,000 dark signals on Saturday, but the concentrated spike in confirmation requests created a backlog that led to response delays, contributing to congestion on already overwhelmed streets. As the blackout persisted and the San Francisco Department of Emergency Management urged residents to stay home, Waymo temporarily suspended service and directed vehicles to pull over and park so they could be returned to depots in stages. Mayor Daniel Lurie contacted Waymo's CEO directly asking them to remove cars from the road immediately. The incident prompted San Francisco supervisors to call for a hearing on Waymo's response, and a California regulator announced it is reviewing incidents involving stalled robotaxis during the outage. Waymo has implemented fleet-wide software updates to give vehicles more context about regional power failures and is updating emergency preparedness plans.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed