How Pinterest Is Using A.I. to Battle Self-Harm Posts on Its Site

When people think of Pinterest, crafts and fashion posts are often what comes to mind. But like many parts of the internet, underneath the surface of the social media site, there’s a darker side. And the company is working on removing it.

According to an announcement made by the company on Thursday, Pinterest says it has used machine learning to see a 88% reduction in reports of self-harm posts, a feat that other social media sites like Facebook, Twitter, and Instagram have struggled with.

Self-harm posts are an issue that Pinterest has prioritized in recent months. Earlier this year, the social network updated its self-harm policy, with guidance informed by suicide prevention charity Samaritans, that broadened the definition of what types of posts are not allowed, as well as gave the site more control over the types of posts it would either remove or hide from other users. Pinterest posts that display harmful text or images can be triggering for people who are struggling with self-injurious behavior.

To combat the issue, Pinterest used machine learning strategies that can automatically detect and hide content that involves self harm and mutilation.

The company also took additional measures to make its platform better for users struggling with mental illness or harmful thoughts. For instance, it’s made the Suicide Prevention Hotline readily available on the app and website through the Pinterest Help Center, removed self injury-related recommendations, and made it easier to find healthy activities and information on the platform. Browsing the term “#pinterestwellbeing” will soon come to U.S. users, and will bring up recommendations for emotionally healthy activities. The suggestions come from collaborative work with Brainstorm: the Stanford Lab for Mental Health Innovation, Vibrant Emotional Health, and the National Suicide Prevention Lifeline. Information from the World Health Organization and the American Academy of Pediatrics are also available on Pinterest.

In an announcement of similar measures released earlier this year, Pinterest project Manager Annie Ta noted that user activity regarding such features are private, so people don’t have to worry about their potential struggles becoming public to other users, or being used in targeted ads.

Pinterest’s success comes after other social media sites have been criticized for not removing harmful content quickly enough. Other social networks like Instagram and Facebook have still yet to see even similar measures of success, though they operate on a larger scale. Both have tried systematically removing self-harm posts, but have not reported the kinds of success metrics that Pinterest has revealed.