BackModel bias

Model bias

Towards risk-aware artificial intelligence and machine learning systems: An overview

Zhang et al. (2022)

Sub-category

Risk Domain

1.1Unfair discrimination and misrepresentation

Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.

"While data bias is a major contributor of model bias, model bias actually manifests itself in different forms and shapes, such as presentation bias, model evaluation bias, and popularity bias. In addition, model bias arises from various sources [62], such as AI/ML model selection (e.g., support vector machine, decision trees), regularization methods, algorithm configurations, and optimization techniques."(p. 5)

Entity— Who or what caused the harm

Human

Due to a decision or action made by humans

AI system

Due to a decision or action made by an AI system

Other

Due to some other reason or is ambiguous

Intent— Whether the harm was intentional or accidental

Intentional

Due to an expected outcome from pursuing a goal

Unintentional

Due to an unexpected outcome from pursuing a goal

Other

Without clearly specifying the intentionality

Timing— Whether the risk is pre- or post-deployment

Pre-deployment

Occurring before the AI is deployed

Post-deployment

Occurring after the AI model has been trained and deployed

Other

Without a clearly specified time of occurrence

Supporting Evidence (3)

Model form error:"When all explanatory variables are available, but the model fails to characterize the relationship between the explanatory variables X and the quantity of interest Y. The specified functional form is inadequate to characterize the true relationship, leading to underfitting of the training data."(p. 6)

Model overfitting: "When a very complex model is fit, it may show excellent performance on the training data but poor performance on data beyond the training set. The model's performance is unstable when making predictions, and it might not generalize well on the testing data."(p. 6)

Variable inclusion error "There are two types of variable inclusion error: (1) Significant variables that should be included in the model are omitted, resulting in the model's inability to characterize the underlying data-generation process and leading to omitted-variable bias. (2) Irrelevant variables are included in the model, which may lead to model overfitting."(p. 6)