Exclusionary norms

"Exclusionary norms can manifest in “subtle patterns like referring to women doctors as if doctor itself entails not-woman” [15], emphasis added."(p. 216)

"Where a LM omits, excludes, or subsumes those deviating from a norm into ill-fitting categories, affected individuals may also encounter allocational or representational harm [100, 159]. Exclusionary norms can place a disproportionate burden or “psychological tax” on those who do not comply with these norms or who are trying to change them."(p. 217)

"A LM trained on language data at a particular moment in time risks excluding some groups and creating a “frozen moment” whereby temporary societal arrangements are enshrined in a model without the capacity to update the technology as society develops [70]. The risk, in this case, is that LMs come to represent language from a particular community and point in time, so that the norms, values, categories from that moment get “locked in” [15, 59]."(p. 217)

"Rare entities can become marginalised due to a ‘com- mon token bias’, whereby the LM frequently provides common but false terms in response to a question rather than providing the less common, correct response. For example, GPT-3 was found to ‘often predict common entities such as “America” when the ground- truth answer is instead a rare entity in the training data’, such as Keetmansoop, Namibia [206].1"(p. 217)