A Dutch municipality used a black-box algorithm to assess property values, but when challenged in court, could not explain how the algorithm reached its decision, violating transparency requirements under Dutch law and GDPR.
In 2016, the municipality of Castricum in the Netherlands used an algorithmic system to assess the WOZ (property tax) value of a claimant's home at 320,000 euros. The claimant challenged this assessment in court, arguing the property was damaged by a 1997 earthquake and worth less. When the claimant demanded to know how the valuation was calculated, the municipality was unable to explain the algorithm's decision-making process because it was a black-box system. The Amsterdam court of appeal acknowledged this violated Dutch case law and GDPR requirements for explainable automated decisions by public bodies, ordering the municipality to pay court costs. However, the court still upheld the 320,000 euro valuation by comparing it to nearby property sales, effectively accepting the black-box algorithm's output despite the transparency violation. According to an official from the valuation chamber, most Dutch municipalities use statistical tools from five companies for WOZ assessments, and the chamber instructs municipalities to ensure their models are explainable and prohibits black-box models. Legal experts warned this ruling could set a dangerous precedent allowing black-box algorithms in public administration as long as results seem reasonable.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed