Just How Far Off Was Trump’s FDA Chief on Benefits of Convalescent Plasma for Covid? Like, A LOT

Trump FDA chief Dr. Stephen Hahn apologized for remarks he made about convalescent plasma, an unproven coronavirus treatment, but his explanation didn’t really illustrate just how far off the mark he had been.

Trump’s Sunday announcement of an emergency authorization for convalescent plasma — made during a press conference that also featured Hahn and HHS Secretary Alex Azar — included a claim from Hahn that drew intense criticism, and for which Dr. Hahn apologized Monday night.

“I have been criticized for remarks I made Sunday night about the benefits of convalescent plasma. The criticism is entirely justified. What I should have said better is that the data show a relative risk reduction not an absolute risk reduction,” he wrote on Twitter.

But how bad a mistake was it? To the untrained ear, this could sound like something in the range between “tomato/tomahto”-level bad to “Do you say gif or jif?” bad.

First of all, here’s what Hahn said that drew so much criticism (emphasis added):

In the optimal treatment — the optimal patients, as described by Secretary Azar, treated with convalescent plasma at the highest titers, there was a 35 percent improvement in survival, which is a significant clinical benefit.

Now, we’re waiting for more data. We’re going to continue to gather data. But this clearly meets the criteria that we’ve established for emergency use authorization, and we’re very pleased with these results.

So, let me just put this in perspective. Many of you know I was a cancer doctor before I became FDA commissioner, and a 35 percent improvement in survival is a pretty substantial clinical benefit. What that means is — and if the data continue to pan out — 100 people who are sick with COVID-19, 35 would have been saved because of the administration of plasma.

It turns out nobody has any idea where the administration got that 35 percent figure from, let alone that it describes the number of people who would be saved by the therapy.

The study, which was not randomized or controlled, didn’t actually compare people who were treated with people who were not. It compared “Patients who received transfusions of convalescent plasma within three days of diagnosis” with “Patients who received plasma after four or more days,” and found a 2.9 percent improvement in the mortality rate for those who received the treatment earlier.

Here’s the egghead explainer on how wrong Hahn was, and what the study actually said:

During the press conference, Azar claimed the use of convalescent plasma offered patients a 35% survival benefit in patients “who benefited the most.”

Or, as STAT writers Matthew Herper and analyst Adam Feuerstein discussed yesterday, “…misspoke, claiming that giving plasma would help 35 out of 100 people treated.”

To which Feuerstein said, “Misspoke is being kind. Hahn grossly mischaracterized the benefit of convalescent plasma on Sunday night. I’ll just quote him here: ‘A 35% improvement in survival is a pretty substantial clinical benefit. What that means is—and if the data continue to pan out—100 people who are sick with COVID-19, 35 would have been saved because of the administration of plasma.” The data don’t show that!”

The actual figures in the data suggest at best the treatment would benefit five out of 100 people, and some analysis suggests only three out of 100 people. And even that is questionable because it is based on subgroups of observational data. Herper noted, “For the main subset in the study, which was led by the Mayo Clinic, mortality at seven days was 11% for those who got lots of antibodies, versus 14% for those who got few. That’s three out of 100—again, with a grain of salt.”

Now, if you’re not a math expert, this means that Dr. Hahn overstated the potential benefit of convalescent plasma by about a thousand percent. Missed it by that much.

Watch the clip above via The White House.