Market concentration and single points of failure

Sub-category

Risk Domain

6Socioeconomic & Environmental

6.1Power centralization and unfair distribution of benefits

AI-driven concentration of power and resources within certain entities or groups, especially those with access to or ownership of powerful AI systems, leading to inequitable distribution of benefits and increased societal inequality.

"Market shares for general- purpose AI tend to be highly concentrated among a few players, which can create vulnerability to systemic failures. The high degree of market concentration can invest a small number of large technology companies with a lot of power over the development and deployment of AI, raising questions about their governance. The widespread use of a few general- purpose AI models can also make the financial, healthcare, and other critical sectors vulnerable to systemic failures if there are issues with one such model."(p. 123)

Entity— Who or what caused the harm

Human

Due to a decision or action made by humans

AI system

Due to a decision or action made by an AI system

Other

Due to some other reason or is ambiguous

Intent— Whether the harm was intentional or accidental

Intentional

Due to an expected outcome from pursuing a goal

Unintentional

Due to an unexpected outcome from pursuing a goal

Other

Without clearly specifying the intentionality

Timing— Whether the risk is pre- or post-deployment

Pre-deployment

Occurring before the AI is deployed

Post-deployment

Occurring after the AI model has been trained and deployed

Other

Without a clearly specified time of occurrence

Supporting Evidence (2)

"The market is so concentrated because of high barriers to entry. Developing state- of- the- art, general- purpose AI models requires substantial up- front investment. For example, the overall costs for developing a state- of- the- art model can currently reach hundreds of millions of US dollars. Key cost factors are computing power, highly skilled labour and vast datasets. ● In addition, market leaders benefit from self- reinforcing dynamics that reward winners. Economies of scale allow bigger AI companies to spread one- off development costs over an ever- larger customer base, creating a cost advantage over smaller companies. Network effects further allow larger companies to train future models with user data generated through older models."(p. 123)

"Market concentration: The degree to which a small number of companies control an industry, leading to reduced competition and increased control over pricing and innovation. ● Single point of failure: A part in a larger system whose failure disrupts the entire system. For example, if a single AI system plays a central role in the economy or critical infrastructure, its malfunctioning could cause widespread disruptions across society."(p. 123)