AI-powered online proctoring systems used by Canadian universities during COVID-19 were found to violate privacy laws and discriminate against students based on socioeconomic factors, with facial recognition and audio monitoring flagging innocent behaviors as potential cheating.
During the COVID-19 pandemic, many Canadian universities deployed AI-powered online proctoring systems including Respondus, Proctorio, Monitor, ProctorU, and Examity to monitor students during remote exams. These systems use data mining, facial recognition, biometric keystroke analysis, eye tracking, and audio monitoring to detect suspicious behavior that could constitute cheating. A University of Ottawa academic report supported by the Office of the Privacy Commissioner of Canada found multiple legal issues with these systems. The AI tools failed to ensure free, clear and individual consent from students whose biometric data was being collected. The systems exhibited significant potential for discrimination based on socioeconomic factors - flagging innocent behaviors like pet noises, children in the background, multigenerational households, rural locations, and lack of proper equipment as potential cheating incidents. The report found particular risks for intersectional students who could face 'extreme risk of discrimination.' Additionally, many of these US-based technology companies required data transfer to the United States, creating privacy concerns under Canadian law. The report concluded that the potential for harm was 'not insignificant' and recommended collective reflection on whether certain AI uses should be prohibited.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed