Welcome back. I’ve got something new and interesting in fake news studies.
Cambridge University researchers judged that where and how best to fight online fabricated news requires a unified way of measuring how vulnerable a person is to being duped. Collaborating with researchers at the University of British Columbia, University of Virginia and Austria’s Karl Landsteiner University of Health Sciences, they developed the first validated Misinformation Susceptibility Test (MIST).
 |
| Online fake news (graphic from guides.lib.berkeley.edu/fake-news). |
The effort included three studies to show how to develop, validate and apply MIST.
Study 1 (N=409 participants) used a neural network language model to generate items and three methods--factor analysis, item response theory and exploratory graph analysis--to create MIST-20 (20 items; test completion time <2 minutes), MIST-16 (16 items; <2 minutes) and MIST-8 (8 items; <1 minute).
Study 2 (N=7674) confirmed the internal and predictive validity of MIST in 5 national samples in the UK and US over 2 years from 3 sampling platforms.
Study 3 (N=421) demonstrated how MIST can provide novel insights on existing psychological interventions, thereby advancing theory development.
Test Real and Fake News Headlines
The Cambridge team developed assessment tools that enabled them to arrive at the right level and mix of fake and genuine headlines to produce the most reliable results.
They took real news from outlets such as the Pew Research Center and Reuters. To create false but confusingly credible headlines in an unbiased way, they tapped artificial intelligence, ChatGPT version 2.
As noted, variations of the survey were tested extensively in experiments involving thousands of UK and US participants.
Example Results
For an April 2023 test, YouGov presented MIST-20 to 1,516 adult US citizens, adding questions regarding the respondents’ demographics, politics and online behavior. Overall, the respondents correctly classified a headline as real or fake about two-thirds of the time, 13 of 20, with similar accuracy for real and fake headlines.
 |
| Categorizing 10 fake news headlines as real or fake on YouGov survey (from today.yougov.com/topics/politics/articles-reports/2023/06/29/americans-distinguish-real-fake-news-headline-poll). |
Surprise! Younger adults did worse than older adults at identifying false headlines. Only 11% of 18-29 year olds got high scores (more than 16 of 20 correct); 36% scored low (10 or fewer correct). In contrast, 36% of those 65 or older scored high while 9% scored low.
The more time someone spent online recreationally, the worse they were at distinguishing real from fake news. Some 30% of those who spent 0 to 2 recreation hours online each day scored high, compared to 15% of those who spent 9 or more hours online.
Comparison of older vs younger adult scores on YouGov survey (from today.yougov.com/topics/politics/articles-reports/2023/06/29/americans-distinguish-real-fake-news-headline-poll).
Although Democrats performed better than Republicans (33% vs 14% scoring high), almost a quarter of both were low scorers.
Wrap Up
The researchers offer the opportunity to test your misinformation susceptibility and participate in this academic study at https://yourmist.streamlit.app/
You will be asked to categorize 20 news headlines as real or fake and answer a few optional questions about your background. It should take 2-3 minutes to complete. All data will be kept anonymous, and you can use the app without sharing your data. You must be 18 years or older to participate.
Go for it with my thanks for stopping by.
P.S.
Study of the development of the Misinformation Susceptibility Test in Behavior Research Methods journal: link.springer.com/article/10.3758/s13428-023-02124-2
YouGov survey applying MIST in fake news testing: today.yougov.com/topics/politics/articles-reports/2023/06/29/americans-distinguish-real-fake-news-headline-poll
Article on Misinformation Susceptibility Test on EurekAlert! website: www.eurekalert.org/news-releases/993707
No comments:
Post a Comment