08 May 2020

News Credibility Warnings

Welcome back. Friends or acquaintances sometimes email articles to me, implicitly or explicitly saying Look at this! Occasionally, what I’m directed to look at doesn’t ring true. If the subject is of any consequence and if I’ve time--I’m retired and have no idea why I seldom have time--I’ll review the article and its source. If I find it’s nonsense or the like, I’ll reply to the sender with my fact check, wondering why the sender didn’t do their own checking. 

I’ve blogged about Fake News Detection, but here's a new idea. A recent study explored whether people would share an article with friends on social media if the article’s headline carried a warning regarding its credibility. 
Example of obscure False headline with Fact Checkers credibility indicator used in study (from acm.org/doi/abs/10.1145/3313831.3376213).
Spoiler alert: Credibility indicators reduced sharing, though the affect varied with the stated source of the warning and the senders’ political affiliation, gender and age.

Testing Credibility Indicators
Researchers with New York University, New York University Abu Dhabi, the University of Sydney and Indiana University Bloomington enlisted just over 1,500 online participants (mean age 38, 51% male) to review and decide about sharing 12 headlines, some with credibility indicators.

The 12 headlines were from articles on a variety of topics. They included 2 well-known and 2 obscure articles chosen randomly from 24 in each of 3 categories--True, False and Satire--half of which were well-known and half obscure.

To test credibility indicators, the researchers divided the participants into 5 groups--a control group and 1 group for each of 4 credibility indicators: 

1. Fact Checkers: Multiple fact-checking journalists dispute the credibility of this news.
2. News Media: Major news outlets dispute the credibility of this news.
3. Public: A majority of Americans disputes the credibility of this news.
4. Artificial Intelligence (AI): Computer algorithms using Artificial Intelligence techniques dispute the credibility of this news.

Participants in each credibility indicator group were shown the indicator in red below False and Satire headlines and nothing for True headlines; participants in the control group saw no indicators. All participants had to decide if they would share the articles (unseen) with friends on social media based on the headlines with or without credibility indicators.

Analysis of Credibility Indicators
The analysis of over 18,100 responses on sharing intent addressed the four credibility indicators and, for broader understanding, such factors as political affiliation, gender and age. I’ll focus on key findings.

All four credibility indicators were somewhat effective in persuading participants to avoid sharing non-true (combined False and Satire) headlines. The Fact Checkers indicator stood out, reducing non-true headline sharing by 43% compared to the control, which was nearly double that of the other three indicators (22% to 25%).

The participants’ political affiliations were categorized as Democrat, 39%, Republican, 20%, and Independent, 41%. Democrats were most responsive to the credibility indicators in reduced sharing of non-true headlines; Republicans were least responsive. The Fact Checkers indicator was the most effective regardless of political affiliation.

Changes in percentages of non-true sharing across credibility-indicator groups separated by political affiliation (from acm.org/doi/abs/10.1145/3313831.3376213).
Females were generally more responsive than males to credibility indicators, the largest differences being associated with Public and News Media indicators. Males were slightly more responsive to the Fact Checkers indicator.
Changes in percentages of non-true sharing across credibility-indicator groups separated by gender (from acm.org/doi/abs/10.1145/3313831.3376213).
Participants older than 35 were inclined to avoid sharing non-true headlines more than those younger than 35. Both age groups were most influenced by the Fact Checkers indicator.

Wrap Up
Socializing was the dominant reason for sharing a headline. Participants chose not to share in about 30% of the cases because they found the headline uninteresting or irrelevant, apart from its veracity. Nevertheless, a notable proportion of participants chose not to share non-true headlines because of their false nature.

Overall, the study demonstrated that credibility indicators could help reduce the spread of misinformation. Although the Fact Checker indicator was by far the most effective, fact checking is labor intensive. We look forward to continued improvements in artificial intelligence.

Thanks for stopping by.

Study of social media news credibility indicators in Proceedings of 2020 CHI Conference on Human Factors in Computing Systems: acm.org/doi/abs/10.1145/3313831.3376213
Article on study on EurekAlert! website: www.eurekalert.org/pub_releases/2020-04/ntso-rmc042820.php

1 comment:

  1. I am disappointed in Republicans, though not terribly surprised.