A Serve Robotics delivery robot crossed police crime scene tape in Los Angeles when a human operator remotely controlling the robot decided to proceed after bystanders waved it through, despite company policy prohibiting such actions.
Serve Robotics, an Uber spinout that has been piloting delivery robots with Uber Eats since May, experienced an incident where one of its Level 4 autonomous delivery robots crossed yellow caution tape at a crime scene near Hollywood High School around 10 a.m. The video of the incident went viral on Twitter with over 650,000 views. The robot initially stopped at the crime scene tape as designed, but a human operator who was remotely monitoring the robot per company policy made the decision to proceed when bystanders, including a TV cameraman, lifted the tape and waved the robot through. According to CEO Ali Kashani, the robot's AI system would never have crossed on its own and requires human approval at intersections and blockades. The company has standard operating procedures requiring human operators to remotely monitor and assist robots at every intersection and take control when obstacles are encountered. Following the incident, Serve Robotics pulled data from the event and is developing new protocols to prevent similar occurrences, including better training for operators and potentially using software to identify law enforcement presence and provide better context for human decision-makers.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.