What do we study when we study misinformation? A scoping review of experimental research (2016-2022)
Harvard Misinformation Review
INTRODUCTION
We reviewed 555 papers published from 2016–2022 that presented misinformation to participants. We identified several trends in the literature—increasing frequency of misinformation studies over time, a wide variety of topics covered, and a significant focus on COVID-19 misinformation since 2020. We also identified several important shortcomings, including overrepresentation of samples from the United States and Europe and excessive emphasis on short-term consequences of brief, text-based misinformation. Most studies examined belief in misinformation as the primary outcome. While many researchers identified behavioural consequences of misinformation exposure as a pressing concern, we observed a lack of research directly investigating behaviour change.
RESEARCH QUESTION
What populations, materials, topics, methods, and outcomes are common in published misinformation research from 2016–2022?
ESSAY SUMMARY
- The goal of this review was to identify the scope of methods and measures used in assessing the impact of real-world misinformation.
- We screened 8,469 papers published between 2016 and 2022, finding 555 papers with 759 studies where participants were presented with misinformation.
- The vast majority of studies included samples from the United States or Europe, used brief text-based misinformation (1–2 sentences), measured belief in the misinformation as a primary outcome, and had no delay between misinformation exposure and measurement of the outcome.
- The findings highlight certain elements of misinformation research that are currently underrepresented in the literature. In particular, we note the need for more diverse samples, measurement of behaviour change in response to misinformation, and assessment of the longer-term consequences of misinformation exposure.
- Very few papers directly examined effects of misinformation on behaviour (1%) or behavioural intentions (10%), instead measuring proxy outcomes such as belief or attitudes. Nevertheless, many papers draw conclusions regarding the consequences of misinformation for real-world behaviour.
- We advise caution in inferring behavioural consequences unless behaviours (or behavioural intentions) are explicitly measured.
- We recommend that policymakers reflect on the specific outcomes they hope to influence and consider whether extant evidence indicates that their efforts are likely to be successful.
IMPLICATIONS
In this article, we report a scoping review of misinformation research from 2016-2022. A scoping review is a useful evidence synthesis approach that is particularly appropriate when the purpose of the review is to identify knowledge gaps or investigate research conduct across a body of literature (Munn et al., 2018). Our review investigates the methods used in misinformation research since interest in so-called “fake news” spiked in the wake of the 2016 U.S. presidential election and the Brexit referendum vote. While previous publications have reflected critically on the current focus and future pathways for the field (Camargo & Simon, 2022), here we address a simple question: what do we study when we study misinformation? We are interested in the methods, outcomes, and samples that are commonly used in misinformation research and what that might tell us about our focus and blind spots.
Our review covers studies published from January 2016 to July 2022 and includes any studies where misinformation was presented to participants by researchers. The misinformation had to be related to real-world information (i.e., not simple eyewitness misinformation effects), and the researchers had to measure participants’ response to the misinformation as a primary outcome. As expected, we found an increase in misinformation research over time, from just three studies matching our criteria in 2016 to 312 published in 2021. As the number of studies has grown, so too has the range of topics covered. The three studies published in 2016 all assessed political misinformation, but by 2021, just 35% of studies addressed this issue, while the remainder examined other topics, including climate change, vaccines, nutrition, immigration, and more. COVID-19 became a huge focus for misinformation researchers in late 2020, and our review includes over 200 studies that used COVID-related materials. Below we discuss some implications and recommendations for the field based on our findings. MORE