Study Finds Fact-checking Videos Better Than Text at Correcting Misinformation

At a time when fact-checkers are proliferating around the globe and experimenting in format and tone, a new study looks at whether text or video is more effective in correcting misinformation — and whether humor helps.

The result? Both funny and non-humorous fact-checking videos were considered “more interesting and understandable” than a comparable print-based fact-checking story, according to the study published in Journalism & Mass Communication Quarterly. “Video formats are a really effective way to deliver fact-checking information because they can increase attention and reduce confusion, when compared with a regular text-based fact-check,” said lead author and Annenberg alumna Dannagal G. Young (Ph.D. '07), an associate professor of communication at the University of Delaware and a distinguished research fellow at the Annenberg Public Policy Center.

Young said humor often has a powerful impact on messaging, but the tone of the video did not change the result in this case. “In our experiment, the humor was not the special sauce that had any effect — it was the audiovisual nature of the information itself that made the difference,” she said.

Young conducted the study with Kathleen Hall Jamieson, director of the Annenberg Public Policy Center (APPC) and co-founder of its project FactCheck.org. The researchers used a 2012 FactCheck.org article and two short videos (one humorous, one non-humorous) produced for its political-literacy companion website, FlackCheck.org. Fact-checking has been shown to be least effective in the most controversial areas, so the researchers chose a less polarizing topic, the Keystone XL Pipeline, to avoid triggering “strongly rooted well-worn biases.”

The FactCheck.org article looked at, among other things, a claim in a flyer from Republican Rep. Lynn Westmoreland of Georgia that the Keystone XL Pipeline was “expected to create tens of thousands of jobs.” FactCheck.org found that figure to be inflated and said the State Department estimated that 5,000 to 6,000 jobs would be created.

The experiment, conducted in January 2016, involved an online panel of 525 U.S. adults. The panel was randomly divided into five groups. Everyone was shown the flyer. Four groups were each shown one of the following: the FactCheck.org article; a 59-second humorous video about the jobs claim developed by comedian Marty Johnson for FlackCheck.org; a 48-second non-humorous video on the jobs claim; an unrelated 62-second funny video of a baby spouting gibberish in a bathtub. The fifth group, the control group, saw only the flyer. The groups were questioned before and after exposure to the information.

Correcting Facts and Changing Beliefs

All three of the fact-checks — the two videos and one text — corrected misinformation to a greater extent than the irrelevant baby video and the control group. But the text was found to be more confusing than the videos, and the videos did a better job at correcting not just the number of jobs that would be created but an inference drawn from that about the broader economic impact of the project.

“We were able to change people’s perceptions of the number of jobs that would be created,” Young said. “Getting people to change their broader belief system is, unsurprisingly, more challenging. But the videos were successful at doing that.” She added, “Correcting the facts is good, but it’s not enough. We want to help people correct the interpretations that are drawn from these facts.”

Surprising Results

After seeing the fact-checks, Republicans showed greater movement toward the correct jobs number than Democrats, the study found. Republicans had been more influenced by the flyer’s deceptive jobs estimate to begin with, perhaps because they were told that it came from a fellow Republican, Young said. So the post-correction result may have reflected a greater adjustment from the initial deception, she said. She suggested that as a “quantitative piece of information,” the jobs number might not have triggered the “backfire” or “boomerang” effect that occurs when people’s partisan belief systems are threatened.

Young was surprised that the funny fact-checking video was not more effective than the non-humorous one. “Humor usually reduces audience resistance and argument scrutiny,” she said. It’s possible, she said, that the humorous video in this case might not have been “funny enough” to set off the psychological processes that typically result from humor. “Perhaps the biggest lesson learned is that it is very difficult to produce genuinely humorous stimuli that also deliver a pointed argument,” Young said. “Funny? OK. Argument-based? OK. Both? Very tricky.”

The study concludes: “For democracy to function, citizens need accurate information that is accessible and understandable. In an information-saturated and politically polarized world, fact-checkers must find ways to stimulate interest and promote accurate inference making through pithy, clear information that corrects one false claim at a time. This study suggests that short, simplified video-based fact-checks (whether humorous or not) offer a successful way to accomplish these goals.”

To download the complete news release, click here.

Media Contact

Michael Rozansky
(215) 746-0202
michael.rozansky@appc.upenn.edu