Does fact checking work?
It seems like a simple question about a simple operation: a politician makes a claim, reporters investigate it, a decision on its veracity is made. In one way, it’s an essential motion of journalism.
But as Brendan Nyhan, an assistant professor of political science at Darmouth University, is discovering, fact-checking effectively—which is to say, not only catching and correcting a falsehood but doing so in a way that dislodges it from the public mind—is a complex and frustrating endeavor, the nuances of which we’re only beginning to understand.
Nyhan has released two studies in the past two weeks to determine whether fact checking works and how it can be improved. The first found that the mere presence of fact checkers, and an awareness of them by candidates, kept politicians honest. Candidates who received letters informing them that fact-checkers were aggressively monitoring them were 55% less likely to receive negative ratings from the watchdog groups, suggesting that fact-checkers’ primary function can be preemptive. The second and more detailed study, released Monday, examined how fact-checkers could make their corrections more penetrating—a question that involves psychology and cognitive behavior research as much as it does political science.
Nyhan got his start in the early 2000s by working for Spinsanity, a precursor to PolitiFact and similar fact-checking groups. “I experienced firsthand how difficult it was to change people minds about misperceptions,” Nyhan told Mediaite, “even the sorts of people who would go to the website like that in the first place.”
Observing not only how politicians presented false information but how people perceived and assimilated it into their beliefs gave Nyhan insight into methods to best correct falsehoods—starting with acknowledging that reversing misinformation is an incremental and incomplete process. “The media needs to be aggressive about trying to set the record straight, but I also think they need to be realistic about that approach,” Nyhan said. “Our research suggests it is difficult to change people’s minds. And there are better ways to present corrective information that will leave your readers with a better understanding of the facts.”
In the most fascinating of Nyhan’s experiments, he and coauthor Jason Reifler created a fictional Alaskan congressman and reported to four different groups that he’d resigned. The control group received the information without context; a second group was told the congressman had resigned over a bribery scandal; a third that the bribery scandal was false; and a fourth that the bribery scandal was false and that the congressman was resigning for the benign reason of running a university. The authors then polled participants as to the congressman’s favorability and trustworthiness.
They found that being told the scandal was false had some effect on participants’ belief in the story, but not nearly as much as being given an alternative explanation. “The innuendo significantly increased the perceived likelihood that Swensen was corrupt,” the authors wrote. “Disturbingly, this effect persisted in the denial condition; respondents who were told of Swensen’ s denial were still significantly more likely than controls to believe in the corruption story. However, the causal correction undid this effect.” Nyhan and Reifler recommended that alternative explanations be presented with corrections whenever possible.
The experiment shows what fact-checkers are up against. When false information is let loose, it is very difficult to refute, especially when it corresponds to a preexisting narrative about a public figure.
A recent case helpfully illustrated this dilemma. The Associated Press’ Bob Lewis, a venerable Virginia political reporter, recently published a story claiming that Virginia gubernatorial candidate Terry McAuliffe had been named in a court document as having lied during a federal investigation. The story turned out to be untrue—the “TM” named in the report was not Terry McAuliffe, though he had been questioning in the investigation—but not before it had been blared across the internet by political and media figures already soured on McAuliffe’s integrity.
“That’s a great example of how humans are very good pattern recognizers,” Nyhan said about how a set of initials became a scandal. “We can be especially sensitive to seeing patterns that confirm what we expect to see, even if it’s not necessarily the case. That’s something you see in misinformation quite often, when people take a piece of evidence confirming a belief they have about a public figure. If you’re operating within the narrative of Terry McAuliffe’s questionable business dealings, and you see initials that match his, that could lead reporters astray.”
“Our research suggest it’s very difficult to unwind these stories once they’re out there,” Nyhan continued. “Especially when the false information that you’re trying to contradict coincides with some narrative about a public figure. The fact that this particular case doesn’t support the narrative about McAuliffe’s business dealings doesn’t mean that people will change their mind about McAuliffe.”
Nyhan added that in many cases similar to the AP/McAuliffe story, even when candidates are cleared of the initial charge they are still loathe to trumpet this fact, as any reiteration of a false charge, even a refutation of it, is likely to reinforce its negative impression. “Sometimes you have cases in which essentially both sides go quiet, and that gives the media incentive to not give coverage to a correction,” Nyhan said.
It is here that fact-checkers are most useful. Nyhan praised the emergence of dedicated fact-checking organizations since the early days of Spinsanity for encouraging the media to operate outside the boundaries of simply covering competing claims. “I think they’ve helped to raise awareness about the perils of what I would characterize as braindead he-said/she said reporting,” he said. “Reporters are much more aware of the problems that fact-checking was designed to address. The journalistic culture around these issues is changing.”
But he also stressed that many fact-checking resources were misapplied to national campaigns, where the volume of coverage enervated the effect of fact-checking. Focusing on local politicians, on the other hand, where even the smallest coverage can be influential on the outcome of a race, could have a trickle-up effect upon more prominent campaigns. “If fact-checkers start fact-checking state legislators, state elected officials, governors and House members more aggressively, it will not only put those people on notice, but their colleagues as well,” he said.
And while Nyhan was generally supportive of dedicated fact-checkers like PolitiFact and Washington Post’s Glenn Kessler, he cautioned that “ghettoizing” fact-checkers could create the impression that fact-checking was a specialty rather than an industry-wide practice.
“When facts are issued in a story, reporters should be trying to check them as best they can,” Nyhan said. “I sometimes worry about too much division of labor, where reporters say, ‘That’s what the fact-checkers do, I do something different.’ Fact-checking to me is a core function of journalism.”
[Image via HAC]
Have a tip we should know? email@example.com