Facebook's 'On This Day' feature automatically surfaced painful memories including posts about deceased friends and family members, causing emotional distress to users who were unexpectedly confronted with reminders of loss.
Facebook deployed its 'On This Day' feature in March 2015, which automatically highlights past posts on users' private pages and sometimes inserts them into News Feeds. The feature began rolling out to limited groups and was designed to capitalize on nostalgia by showing users their historical posts. However, the algorithmic system failed to filter out emotionally painful content, resulting in users being confronted with posts about deceased friends, family members, and traumatic events. Sean Forbes was shown a memorial post he had made three years earlier about a Navy friend who had committed suicide, with the original post containing 'R.I.P.' and expressing his shock at the friend's passing. Another user reported seeing the last post her friend made before taking his own life. This incident followed a similar problem with Facebook's Year in Review feature in 2014, which had shown users painful images including deceased children, dead pets, and other tragic events under celebratory banners. Facebook stated it had implemented some precautions for On This Day, including filters for de-listed relationship statuses and memorialized accounts, but these measures proved insufficient to prevent emotional harm. The company acknowledged that users could adjust the frequency of On This Day posts but provided no option to opt out completely.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed