A coding error in Ireland's Leaving Certificate grading algorithm caused over 6,000 students to receive lower grades than they should have and nearly 8,000 students to receive higher grades than deserved.
In 2020, Ireland implemented an AI-powered algorithm to calculate Leaving Certificate grades for students affected by COVID-19 disruptions. The algorithm was custom-built by contracted firm Polymetrika to implement a blended formula based on students' past performance. A single line of code out of 50,000 contained two critical errors: it substituted students' worst two subjects for their best two subjects, and it incorrectly included Junior Cycle Civic, Social and Political Education results in the calculation. These coding errors resulted in over 6,000 students receiving grades lower than they should have and nearly 8,000 students receiving grades higher than the algorithm should have assigned. The errors were discovered through an internal audit by Polymetrika and reported to Department of Education officials. The government directed third-level institutions to make additional spaces available for students who received incorrectly low grades. The Department stated there wasn't sufficient time to test the code thoroughly due to urgency and resource constraints. A US-based specialist firm, Educational Testing Service, was contracted to review the coding for additional errors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed