Shot in the dark: Early COVID-19 vaccine efficacy explained
By John Miller
ZURICH (Reuters) - Moderna Inc <MRNA.O> delivered the latest good news from COVID-19 vaccine developers, saying its experimental vaccine is 94.5% effective in preventing the disease based on interim data from a late-stage trial.
Pfizer Inc. <PFE.N> and BioNTech SE <22UAy.DE> on Nov. 9 trumpeted early data indicating their mRNA candidate is more than 90% effective.
A Russian project then touted 92% efficacy for the Sputnik V candidate, named after the Soviet-era satellite, based on a smaller data set.
HOW DO MANUFACTURERS ARRIVE AT EFFICACY NUMBERS?
In Pfizer's case, it waited until 94 volunteers in its late-stage clinical trial of more than 43,500 people - half got the vaccine, the other half got a placebo - tested positive after developing symptoms.
For 90%-plus efficacy, no more than eight people among those who tested positive had received the vaccine, with the rest having received the placebo.
"Roughly speaking, it's probably around eight to 86 cases in the treated and placebo groups," David Spiegelhalter, a Cambridge professor of risk and an expert in statistics, told Reuters. "You don't need a lot of fancy statistical analysis to show that this is deeply impressive. It just hits you between the eyes."
Moderna's interim analysis was based on 95 infections among trial participants who received either a placebo or the vaccine. Of those, only five infections occurred in participants who received the vaccine, which is administered in two shots 28 days apart.
In Russia, Sputnik V-developer Gamaleya Institute reached its preliminary 92% efficacy figure based on 20 illnesses in 16,000 volunteers as its late-stage trial progresses. It aims to reach 40,000 people.
Of the 16,000 people, about quarter got the placebo.
"It suggests that there is some effect, but it's insufficient to estimate the magnitude of it," Spiegelhalter said.
WATCH BELOW: WHY GETTING A FLU SHOT THIS YEAR IS MORE IMPORTANT THAN EVER
HOW MANY PEOPLE MUST GET SICK IN BIG VACCINE TRIALS?
Some experts say that, ideally, 150 to 160 people in a trial of tens of thousands of participants must get sick before making a reliable assessment of a vaccine's efficacy. That's a bit of a rule of thumb, though, open to interpretation.
"There is no such regulatory standard requiring X number of events for making a reliable decision," the government-funded Swiss Clinical Trial Organisation said. "The amount of (infections) has to be seen in relation to the disease and its risk profile. It's rather a case-by-case evaluation."
Go HERE for our complete coverage of the COVID-19 pandemic
Typically, regulators strive to have at least 95% certainty that the trial read-out is not the result of random variations with nothing to do with the tested compound.
For trial sponsors there is safety in numbers as a large enough trial can ensure that 95% reliability hurdle is cleared.But the larger the underlying clinical benefit, the fewer trial participants needed to create that clarity.
In Pfizer and BioNTech's trial, they planned a final analysis when 164 people had become sick, with multiple, pre-planned interim analyses along the way. They skipped an analysis at 32 patients, and once they were ready to release a look at the 62-person mark, 94 had come down sick.
Moderna said it expects to have registered 151 infections when it seeks regulatory approval, meaning the trial will have hit its final endpoint.
Details from the Russian trial are unclear, without access to its protocol.
WATCH BELOW: PFIZER CEO HAS VERY OPTIMISTIC WORDS ABOUT NEW TRIAL VACCINE
HOW DO THESE RESULTS STACK UP TO OTHER DRUGS, OR VACCINES FOR OTHER ILLNESSES?
In normal drug trials, for diseases like terminal cancer, benefits of new medicines may be less apparent, with survival benefits of just a few months sometimes revolutionary for patients at death's door.
For vaccines, however, marginal protection is inadequate, and the World Health Organization ideally wants to see at least 70% efficacy in trials, while the U.S. Food and Drug Administration wants at least 50%.
The efficacy rate reported in the three trials beats those, and appears to exceed that of typical flu vaccines, which the U.S. Centers for Disease Control and Prevention (CDC) estimate reduce the risk of sickness by 40%-60%.
For other shots, the CDC estimates the efficacy of a two-shot measles vaccine at 97%, and a two-dose chicken pox vaccine at 90%. Two doses of polio vaccine are 90% effective, rising to nearly 100% with a third.
CAN WE EXPECT EFFICACY RATES TO HOLD UP AS TRIALS ADVANCE?
Pfizer acknowledged that its final vaccine efficacy percentage may vary. Still, Spiegelhalter said the study's design seems likely to generally hold up, based on the 94 sick participants.
"In this case, the effect is so huge, even if there is a little bit of fallback - if the effects become slightly smaller over time - that is very unlikely to be significant."
WHAT ABOUT REAL-WORLD EFFICACY, SHOULD THE VACCINES BE APPROVED?
The interim data is promising, since it appears to demonstrate that a vaccine can be effective in preventing COVID-19.
The jump to mass vaccinations, however, presents new hurdles, in particular for an mRNA vaccine like Pfizer and BioNTech's that must be stored and shipped at minus 70 degrees Celsius (-94°F).
The three vaccines so far require two doses. In the Pfizer-BioNTech case, it's ideally 21 days apart and Moderna 28 days. If people do not stick to the timetable, it may affect the vaccine's efficacy.
Protection against the mumps, for instance, drops from nearly 90% to 78%, if people don't get a follow-up shot.
Swiss epidemiologist Marcel Tanner, president of Switzerland's Academies of Arts and Sciences and one of the government's top COVID-19 science advisers, expects possible variations in efficacy among older people, whose immune systems wane with time, or those with immune disorders.
"Efficacy says, 'Does it work?' Effectiveness says, 'Can it be applied? Can you carry the efficacy to the people?'" Tanner said. "But no question: 90% efficacy, at that stage, is a pretty good result."
(Reporting by John Miller in Zurich, Kate Kelland in London, Ankur Banerjee in Bengaluru, Julie Steenhuysen in Chicago and Polina Ivanova in Moscow; Editing by Josephine Mason, Nick Macfie and Timothy Heritage. Thumbnail image courtesy of Getty Images/Stock)
