Out-of-domain data
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
"Without proper validation and management on the input data, it is highly probable that the trained AI/ML model will make erroneous predictions with high confidence for many instances of model inputs. The unconstrained inputs together with the lack of definition of the problem domain might cause unintended outcomes and consequences, especially in risk-sensitive contexts....For example, with respect to the example shown in Fig. 5, if an image with the English letter A" is fed to an AI/ML model that is trained to classify digits (e.g., 0, 1, …, 9), no matter how accurate the AI/ML model is, it will fail as the input data is beyond the domain that the AI/ML model is trained with. U"(p. 5)
Other risks from Zhang et al. (2022) (6)
Data bias
1.1 Unfair discrimination and misrepresentationDataset shift
7.3 Lack of capability or robustnessAdversarial attack
2.2 AI system security vulnerabilities and attacksModel bias
1.1 Unfair discrimination and misrepresentationModel misspecification
7.3 Lack of capability or robustnessModel prediction uncertainty
7.3 Lack of capability or robustness