WSJ has an story on the use of artificial intelligence to predict potential suicides. The article cites a study indicating that “the traditional approach of predicting suicide,” which includes doctors’ assessments, was only slightly better than random guessing.
In contrast, when 16,000 patients in Tennessee were analyzed by an AI system, the system was able to predict with 80-90% accuracy whether someone would attempt suicide in the next two years, using indicators such as history of using antidepressants and injuries with firearms. (It would have been nice to see numbers cited for false negatives vs false positives…also, I wonder how nearly the same results could have been achieved by use of traditional statistical methods, without appeal to AI technology.)
Another approach, which has been tested in Cincinnati schools and clinics, uses an app which records sessions between therapists and patients, and then analyzes linguistic and vocal factors to predict suicide risk.
Both Apple (Siri) and Facebook have been working to identify potential suicides. The latter company says that in a single month in fall 2017, its AI system alerted first responders in 100 cases of potential self-harm. (It doesn’t sound like the system contacts the first responders directly; rather, it prioritizes suspected cases for the human moderators, who then perform a second-level review.) In one case, a woman in northern Argentina posted “This can’t go on. From here I say goodbye.” Within 3 hours, a medical team notified by Facebook reached the woman, and, according to the WSJ article, “saved her life.”
It seems very likely to me that, in the current climate, attempts will be made to extend this approach into the prediction of self-harm into the prediction of harm to others, in particular, mass killings in schools. I’m not sure how successful this will be…the sample size of mass killings is, fortunately. pretty small…but if it is successful, then what might the consequences be, and what principles of privacy, personal autonomy, and institutional responsibility should guide the deployment (if any) of such systems?
For example, what if an extension of the linguistic and vocal factors analysis mentioned above allowed discussions between students and school guidance counselors to be analyzed to predict the likelihood of major in-school violence? What should the school, what should law enforcement be allowed to do with this data?
Does the answer depend on the accuracy of the prediction? What if people identified by the system will in 80% of cases commit an act of substantial violence (as defined by some criterion), but at the same time there are 10% false positives (people the system says are likely perpetrators, but who are actually not)? What if the numbers are 90% accuracy and only 3% false positives?