This is facebook using suicide as a wedge issue. I find facebook difficult enough. We spent an evening last week blocking people, switching off apps that had accumulated like dust bunnies, and disconnecting personalization.
AI has been tested in suicidal ideation. Let us be charitable and say that this is 95% specific and 99% sensitive. (It is not). If the suicide rate is 10 to 15 per 100 000 facebook users — those are the figures for most western countries, then the software will flag something like 1 in 1000 people.
To put it another way, 99.9% will not be suicidal. Assange is correct. This is monitoring for crimethink, and the crisis services are swamped already, as spring ends and the stress of advent (and exam results) hits my academic town.
Facebook also will use AI to prioritize particularly risky or urgent user reports so they’re more quickly addressed by moderators, and tools to instantly surface local language resources and first-responder contact info. It’s also dedicating more moderators to suicide prevention, training them to deal with the cases 24/7, and now has 80 local partners like Save.org, National Suicide Prevention Lifeline and Forefront from which to provide resources to at-risk users and their networks.
“This is about shaving off minutes at every single step of the process, especially in Facebook Live,” says VP of product management Guy Rosen. Over the past month of testing, Facebook has initiated more than 100 “wellness checks” with first-responders visiting affected users. “There have been cases where the first-responder has arrived and the person is still broadcasting.”
The idea of Facebook proactively scanning the content of people’s posts could trigger some dystopian fears about how else the technology could be applied. Facebook didn’t have answers about how it would avoid scanning for political dissent or petty crime, with Rosen merely saying “we have an opportunity to help here so we’re going to invest in that.” There are certainly massive beneficial aspects about the technology, but it’s another space where we have little choice but to hope Facebook doesn’t go too far.
Treat all social media with suspicion: use facebook for local things, until something better comes around. In my small village, that is the low tech monthly newsletter. In the meantime, Adblock exists to starve the borg. Use it.