Evidence-Based Techniques for Countering Mis-/Dis-/Mal-Information: A Primer

Executive Summary

US allies and adversaries have used tools of persuasion and influence throughout the 20th century, and adversary attempts to use persuasion and influence to harm the US and its allies have been considered a national security priority for decades. In this context, the US has long worried about foreign efforts to use persuasion and influence against US servicemembers. Though concern about this issue waned with the end of the Cold War, the need to harness research to actively protect US servicemembers from malign persuasion and influence—from mis-/dis-/mal-information (MDM)—has once again become pressing. According to a May 2020 survey directed by senior Army leadership, almost 90 percent of US Army soldiers and civilian employees had not received any information from their units regarding adversarial propaganda about COVID-19 despite the fact that both Russia and China had been circulating virus-related MDM since March 2020. This lack of awareness—and lack of counter-MDM training—left servicemembers vulnerable to external influence. It also effectively ceded the battlespace to US adversaries, allowing Russia and China to act with uncontested impunity to potentially influence servicemembers in the information sphere.

In promising news, an increasingly robust body of research addresses how to effectively counter MDM. Much of this research builds on 20th-century work by social scientists, with adjustments to compensate for the reality of social media, which has seriously exacerbated the scope of this threat. In this paper, we offer a plain-language explanation of the evidence-based research on four types of counter-MDM interventions: inoculation, debunking, fact-checking, and media literacy. More specifically, we discuss the origins and logic of each intervention, summarize overall research findings, identify issues of ongoing analysis, and discuss how long the effects of each intervention typically last.

Interventions

It is important to acknowledge, at the outset, that a comprehensive human-centric approach to this challenge (versus a technology-centric approach such as using AI to identify MDM) must take into consideration the fact that this is both a psychological and social issue. As such, it is critical that we identify, design, and implement counter-MDM solutions that address both the psychological vulnerabilities that make us receptive to MDM (e.g., our tendency to accept at face value content that looks official), and the social structures that make organizations vulnerable to the spread of MDM (e.g., our tendency to believe content from authoritative figures in hierarchical organizations). This work focuses on the former, but work on the latter is equally important.

Note that the interventions we describe are not designed to change people’s strongly held beliefs—or even people’s lightly held opinions. This work simply aims to protect people from being manipulated by the systems and actors trying to circumvent their ability to engage in reasoned and critical thinking.

We also note that protecting oneself from MDM is more complicated than it might seem. Being savvy about the media landscape is not enough, nor is knowing that you are being exposed to MDM. This content works by exploiting normal psychological mechanisms that people use in their day-to-day lives. As an analogy, keeping your front door locked at night is a great first step in protecting your home, but it won’t stop a burglar who breaks in through your dryer vent (i.e., something you didn’t think of as a vulnerability). In the same way, being intelligent, thoughtful, and critical—and even recognizing MDM in your newsfeed—is not adequate protection against MDM because this type of content circumvents normal defenses. The training interventions outlined in this literature are designed to bolster defenses, including those at the metaphorical front and back doors (which may be strong but not strong enough) and those at the dryer vent and heat exhaust (which may not yet exist).

Inoculation

Inoculation is the practice of exposing individuals to persuasive messages containing weakened arguments that threaten an attitude or belief in order to “inoculate” them against stronger persuasive messages and attacks on this attitude or belief in the future. Inoculation builds resilience to manipulation (Table 1).

At its core, the theory of inoculation holds that a “vaccine” consists of two parts: a threat or forewarning of MDM and a refutational preemption (sometimes referred to as a “prebunk”). Through inoculation, the individual is forewarned of or threatened by a counter-attitudinal “attack” (a term used to describe a counterargument) that motivates resistance, and then they receive the skills or information to refute the counterargument. Studies and experiments have found that inoculation theory effectively neutralizes and builds resistance to MDM. According to one article, “Over the last 50 years, a large body of evidence across domains—from health to political campaigning—has revealed that inoculation messages can be effective at conferring resistance to persuasion.” This general consensus is borne out by the majority of recently published literature.

Debunking

Debunking is the use of a concise correction to MDM that demonstrates that the prior message or messaging campaign was inaccurate (Table 2).

The logic of debunking is relatively straightforward: it involves the targeted provision of correct information in response to incorrect information. In this respect, debunking is primarily a therapeutic intervention that responds directly to MDM after it has been circulated, but it can also be a quasi-prophylactic intervention when the correction alerts recipients to specific bad actors or sources who are likely to spread MDM. A common finding of debunking research is that corrections succeed in reducing belief in the accuracy of MDM, or as one debunking expert we consulted put it: “Corrections are wildly effective.” Overall, research indicates that MDM debunking, when well executed, is an effective tool for countering MDM. In fact, the limited cross-comparative studies comparing inoculation (a.k.a., “prebunking”) and debunking found debunking to be even more effective, although both reduced MDM reliance.

Fact-checking

Fact-checking is a journalistic practice designed to reject clearly false claims with empirical evidence from neutral or unimpeachable sources (Table 3).

Debunking and fact-checking are quite similar, but fact-checking is primarily employed by journalists and newsrooms, is typically impartial (to the extent that it adheres to journalistic standards of accuracy), and aims to correct all falsehoods within a given context (e.g., a political debate). The general logic of fact-checking is relatively straightforward: (1) an individual is presented with false information, (2) the individual is presented with subsequent information that corrects the initial information, and (3) the individual updates their belief about the accuracy of the original information to more closely align with the factual information. Factchecking is, as a result, a therapeutic intervention that directly responds to a specific piece of false information. At present, the findings on fact-checking are slightly mixed, and a recent meta-analysis determined that the overall effectiveness of fact-checking seems to be contingent upon a range of issues, including the political sophistication of the individual reading the fact-check, the nature of the message, and the preexisting beliefs of the individual.

Media literacy

Media literacy is an individual’s ability to critically assess a piece of content. It includes the skills required to evaluate a piece of content, as well as an understanding of the structures that produced that content (Table 4).

Media literacy can be thought of as a process or set of skills based on critical thinking. Media literate individuals have the tools and abilities necessary to critically evaluate a piece of “media,” whether that be a tweet, an article, a TV show, or other content. Media literacy training teaches a range of skills including, but not limited to, asking questions, analyzing sources, assessing bias, and valuing the role of an independent media. In this sense, media literacy is content neutral: it does not advance or counter specific ideas but rather teaches wholly nonpartisan skills. Media literacy, like inoculation theory, is a preventative or prophylactic intervention, and its original goal was to help protect young people from negative media exposure effects. As described by some scholars, media literacy trainings and interventions are examples of “logic-based inoculations.” In these framings, media literacy education and critical-thinking interventions become types of inoculation. Not all scholars call their media literacy programs inoculations, but many cite well-known scholars and specific tenets from the inoculation theory literature. For example, Vraga and Tully (2015) note that a media literacy public service announcement may serve a similar role as a booster in an inoculation campaign by reminding people of things they learned in earlier media literacy education. Because media literacy interventions were found to be increasingly effective outside of classrooms, against both inaccurate headlines and biased media, they emerged as a promising preemptive intervention that might (similar to, but distinct from, inoculation efforts) combat MDM.

Potential concerns

In reviewing the literature on MDM interventions, we identified three concerns that scholars repeatedly raised: the backfire effect, the continued influence effect, and news cynicism. In broad terms, the backfire effect is a worry that counter-MDM interventions and trainings will result in unforeseen consequences. However, research shows that worries about the backfire effect are likely overwrought. Although earlier studies detected backfire effects, recent research has found little evidence of these effects, and studies have been unable to show that they occur under only certain conditions. The continued influence effect is a worry that MDM cannot be truly eliminated but will continue to exert an influence even after an intervention or training. Research suggests that this concern is legitimate; however, given the nature of the continued influence effect (i.e., a failure to fully eliminate the influence of MDM), this issue is not a reason to avoid counter-MDM trainings or interventions. The third concern, news cynicism, is slightly more complicated. As the research highlights, counter-MDM interventions and training might increase cynicism related to real news. And yet, numerous experts point out that this outcome may not necessarily be bad. Certainly, people doubting the veracity of all information would be a negative outcome, but people approaching all headlines (from both partisan and nonpartisan sites) with a critical eye may be socially healthy.

Conclusion

We completed this review in support of a broader project whose goal is to recommend a single intervention (or suite of interventions) that the US government might adopt to protect servicemembers from malign foreign influence. Although it stands alone as a useful primer for those hoping to understand the state of research on these issues, more specific guidance—in the form of an assessment of applicability to military populations, a list of best practices, and recommendations for near-term implementation—can be found in the companion report: Protecting Servicemembers from Foreign Influence: A Counter-MDM Toolkit.