An Australian telecommunications company deployed an AI bot to manage 100% of network incidents, but the system incorrectly sent excessive expensive technician visits due to its inability to understand complex scenarios that required human judgment, resulting in approximately $11 million in operational costs.
In early 2018, an Australian telecommunications company deployed an AI program for incident management, expecting to save over 25% of operational costs. The bot was designed to intercept all network incidents and take one of three actions: remotely resolve the issue, dispatch a technician, or escalate to a human operator. Within weeks, the bot was sending an excessive number of costly technician visits to customer premises. The team discovered that certain incident scenarios required human understanding and judgment that the bot could not provide. The automation team was unable to turn off the bot because it was implemented in an all-or-nothing fashion, sitting between the user and operator interface. The company had already released staff for cost savings and was not ready to handle the workload manually. The telco spent several million dollars continuing service with the flawed bot and running a fix-up project that lasted over a year. Eventually, the company allegedly spent approximately $11 million in operational costs due to this incident. The implementation lacked proper testing, a kill switch mechanism, and adequate governance for managing the AI system.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.