Portland Water Bureau's AI pilot program randomly selected wealthy CEO Tim Boyle for a 40% water discount despite his high consumption and ability to pay, exposing flaws in the machine learning system designed to identify customers needing financial assistance.
In February, Portland's City Council approved a $350,000 contract with SERVUS to use machine learning to better distribute water bill discounts to needy customers. The Water Bureau's customer assistance program, which has $10 million available, has consistently fallen short of helping its target of 10,000 customers annually since 1995, never exceeding 8,500 customers per year despite $28 million in outstanding late payments from customers. SERVUS is running two 90-day pilot programs using randomization software to test whether its algorithm can accurately identify customers' ability to pay. In October, Columbia Sportswear CEO Tim Boyle, who earned over $1.6 million in compensation last year and whose shares are worth over $1.7 billion, received a letter offering him a 40% discount through the 'Smart Discount Program.' Boyle was previously identified as Portland's ninth-largest residential water consumer in 2021, using over 770,000 gallons annually. The Water Bureau's customer service director Quisha Light explained that the randomized control trial aims to reach a broader, more diverse group of customers, including those who might not typically apply for assistance. Boyle declined the discount, stating that someone who needs it should receive it instead.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed