The Giggle app used AI facial recognition technology to exclude transgender women from a female-only social platform, causing discrimination and harm to trans women and women of color who were systematically denied access.
Giggle, a social app launched in early 2020 by Australian founder Sall Grover, marketed itself as a female-only networking platform using AI facial recognition for gender verification. The app required users to submit selfies that were analyzed by Kairos facial recognition technology, which needed to be 95% certain the person was female before allowing account creation. The AI system faced criticism for failing to properly identify women of color due to algorithmic bias, with research showing Kairos misgendered darker-skinned females 22.5% of the time in 2019. Initially, some transgender women were able to access the platform, but Grover later began manually removing trans women after they criticized the app online. Jenny, a 23-year-old trans woman, and Victoria Morris, a 27-year-old trans woman, were among those who gained access but were subsequently banned by Grover personally. The app was temporarily removed from Google Play in December 2021 following negative reviews but was restored in January 2022. The incident highlighted issues with AI bias affecting both transgender women and women of color who did not conform to Eurocentric facial features.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed