Here’s a case study on how to deliberately misunderstand statistics. And science. And probably lots of other things.

A few months ago, The Stream published, “Billboards Near CDC Campus Ask: ‘How Many More Kids Have to Die’?” The Stream is my favorite source of right-wing insanity.

The billboard ads were paid for by the Vaccine Safety Research Foundation. Who’s going to argue with a group that wants vaccine safety? Hopefully all of you, after we finish exploring the story behind the arresting demand, “How many more kids have to die before the CDC stops the Covid shot?”

The first red flag is that covid vaccination “science” is being done here by billboard and discredited researchers, not scientific conferences and journals. But set that aside, and let’s look at the argument defending that question. This is from The Stream’s article:

VSRF recently released a compelling video titled, Are the Kids OK? citing data that approximately 4,650 children in the U.S. have died following COVID-19 vaccination.

That’s U.S. children who died from covid vaccinations. (That’s the claim, anyway. Hold your judgment until we review everything.) Compare that with deaths from covid itself. The CDC reports 1,658 children aged 0–18 years have died from covid. (The CDC is the Centers for Disease Control and Prevention, the US’s national public health agency.) That means almost three times more kids were killed by covid vaccinations than covid disease.

At least, that’s the claim. Here’s how they justify that:

This death calculation is derived from the 150 child deaths following COVID shots that have been reported as of Sept. 23, 2022, on the CDC’s own vaccine injury tracking system, the Vaccine Adverse Event Reporting System (VAERS), multiplied by the VAERS underreporting factor of 31 established in a peer-reviewed study by scientist Jessica Rose, Ph.D.

We need to take that apart. There’s a lot of shallow thinking there.

What is VAERS?

The CDC established the Vaccine Adverse Event Reporting System (VAERS), an online database for reports of adverse events due to vaccines. These events can be as mundane as a sore arm or as severe as death. It’s an early-warning system, and the CDC monitors it to look for patterns that might identify, say, a bad batch of vaccine or a subset of the population that is showing more side effects than expected.

Healthcare professionals who administer the vaccinations are obliged to use VAERS to report severe events they observe. But it’s for more than that: “Anyone can submit a report to VAERS, including parents and patients.… Please report clinically important adverse events that occur after vaccination of adults and children, even if you are not sure whether the vaccine caused the adverse event.”

This means that the data isn’t necessarily relevant. Perhaps a 90-year-old died of cancer six months after getting vaxxed. That’s certainly a severe adverse event (SAE) that occurred after a vaccination, but there’s no reason to think that the vaccination caused the event. And, though it’s illegal, during these politically charged times there must be some report that are fraudulent.

VAERS cautions users [bolding in original], “While very important in monitoring vaccine safety, VAERS reports alone cannot be used to determine if a vaccine caused or contributed to an adverse event or illness.… VAERS reports may contain information that is incomplete, inaccurate, coincidental, or unverifiable. Reports to VAERS can also be biased. As a result, there are limitations on how the data can be used scientifically.”

Only when they can say, with evidence, “Covid vaccine caused these deaths” will they add to the conversation.

Child deaths due to vaccine

Now on to “150 child deaths following COVID shots.” This data comes from the VAERS database. I checked the VAERS data for 2021 and found 76 children under 18 years old who died, so I can believe that 150 number. But remember, this is just reports of 150 children who got the covid shot and then, days or months later, died. This gives us no good reason to think that any children died because of the vaccination.

The VAERS database is only meant to record adverse events after a vaccination. The Stream article flirts with the post hoc ergo propter hoc fallacy: just because a death happened after a covid shot doesn’t mean it was a death because of the shot.

Consider the claim made in the Stream article, “approximately 4,650 children in the U.S. have died following COVID-19 vaccination.” Technically, that’s correct, but death followed lots of things in these children’s lives. Did a child who died get a vaccine beforehand? Maybe they rode a skateboard beforehand as well—could that have been the cause? But of course, if the article made clear that they weren’t saying that the vaccine was a cause, there would be no purpose to the article.

And the billboard was unambiguous: “How many more kids have to die before the CDC stops the Covid shot?” They are plainly declaring that the covid vaccination caused deaths.

VAERS underreporting factor of 31

Next, the article wants to increase the number of children who died: 150 children x 31 = 4,650 children total.

What is this underreporting factor? The Stream article refers to “the VAERS underreporting factor of 31 established in a peer-reviewed study by scientist Jessica Rose, Ph.D.” That study is “Critical Appraisal of VAERS Pharmacovigilance: Is the U.S. Vaccine Adverse Events Reporting System (VAERS) a Functioning Pharmacovigilance System?” (I’ll call this “the Rose article”). I question whether this journal is objective, but let’s set that aside and give the article the benefit of the doubt. How does it justify this underreporting factor?

The Rose article ignores the question of whether the 150 deaths of children in the VAERS database were due to the vaccine and wonders how many adverse events happened that didn’t get reported. It sensibly notes,

Under-reporting of mild AEs [Adverse Events] such as rashes or low-grade fever would most likely be far greater than for SAEs [Severe Adverse Events], such as death.

But it immediately ignores this bit of common sense and calculates a single under-reporting factor for all events, from mild fevers to death.

Computing the expected number of SAEs

The under-reporting factor is the ratio of the expected number of SAEs (Severe Adverse Events) to the observed number.

The expected number of SAEs comes from the data from the vaccine trials conducted by the manufacturers. Only the results of the Pfizer trial were considered. A summary of their results is Vaccines and Related Biological Products Advisory Committee Meeting. In a section titled “Overview of Adverse Events” is this table.

The important category is near the bottom, “From Dose 1 through cutoff date (safety population).” The safety population tests the safety of the vaccine and is every participant in the trial who received a real vaccine rather than a placebo. Note “SAE … 124/18801 (0.7).” This says that out of 18,801 vaccine recipients, 124 (or 0.7%) had an SAE, a severe adverse event.

Unlike the adverse events recorded in the VAERS database, every one of these SAEs was recorded by a medical professional. We can have confidence that the number is complete and reliable, but, as with VAERS, we still don’t know cause and effect.

Skip to the bottom of Table 14 to the “Deaths” line. Out of the entire population of vaccine recipients were two deaths. That might initially sound shocking until you consider that a group of 21,621 people over the course of a months-long trial might be expected to have a couple of deaths. Skip further to the right and see “4/21631,” which is the deaths for the placebo group. In other words, the group that got an inert injection had twice the number of deaths. The Pfizer document makes no special mention of these deaths, so they apparently were unremarkable to the Pfizer and FDA statisticians.

Who’s going to argue with a group that wants vaccine safety? Hopefully all of you, after we finish exploring the story behind the arresting demand, “How many more kids have to die?”

The study was double blinded (neither the participants nor the administrators knew who got a vaccine and who got a placebo), and the placebo group was as large as the vaccine group. They each had similar numbers of SAEs, 101 for the placebo group and 124 for the vaccine group. As with the deaths, when you monitor two groups of almost 20,000 people for months, you will notice occasional severe medical problems.

This shows one important benefit from the placebo group. Without it, the SAEs from the vaccine group would be ungrounded. Is 124 SAEs a lot? Is it a little? But the placebo group’s SAEs help identify a baseline.

We now have our scaling factor: 124/18801 = 0.007 = 0.7%. This is the fraction (or percentage) of vaccine recipients who should be expected to have an SAE. As with the deaths, 0.7% of recipients getting a “severe adverse event” sounds pretty bad, but similar numbers in the placebo group show that it’s expected.

Here’s how the Rose study uses the scaling factor.

As of August 10th, 2021, 197,399,471 million [sic] Pfizer/ BioNTech COVID-19 product doses had been administered in the U.S. and therefore the number of expected SAE occurrences in the U.S. volunteer recipients of the Pfizer/BioNTech products should be ~1.4 million SAEs.

That is, with 197 million doses administered, we should expect 197 million x 0.7% = roughly 1.4 million SAEs.

But that’s not enough to satisfy what looks like an agenda behind this article. SAEs are bad, but they’re vague. Child deaths are much more captivating. And the Pfizer document doesn’t itemize SAEs besides the deaths, and there were no children in the vaccine trials. The value to them of VAERS is that it has children’s deaths.

Computing the observed number of SAEs

We’re finally ready to compute that underreporting factor.

The observed number of SAEs is the SAEs in the VAERS database. This was given without explanation as 43,948 (so I have no confirmation). The underreporting factor is the expected number of SAEs (1.4 million, which uses the fraction from the Pfizer study document) divided by the observed number of SAEs (43,948, apparently the number of SAEs in VAERS). Find the ratio, and there’s your underreporting factor: 1.4 million / 43,948 = 31. Therefore, any VAERS statistic needs to be scaled up by 31.

Now we can finally understand that first line quoted from the Stream article:

VSRF recently released a compelling video titled, Are the Kids OK? citing data that approximately 4,650 children in the U.S. have died following COVID-19 vaccination.

The VAERS database claims 150 deaths of children. Multiply by the VAERS underreporting factor, and you get 150 x 31 = 4,650.

Critique

After that long and cumbersome analysis, we can finally see where the claims came from. The argument has lots of problems.

• Causal link. While the Stream article tries to avoid stating plainly that covid vaccinations are killing people, that is obviously their point. There would be no article otherwise. Still, they can point to nothing causative. Only when they can say, with evidence, “Covid vaccine caused these deaths” will they add to the conversation.
• VAERS database can’t show cause. The CDC’s VAERS database is key to the Rose argument, and, as the CDC makes clear, “VAERS reports alone cannot be used to determine if a vaccine caused or contributed to an adverse event or illness.” Therefore, the argument can’t make any claims of covid vaccination causing death.
• Scientific consensus. The CDC is convinced that covid vaccines provide much, much more benefit than any side effects create harm. This is the scientific consensus.
• Vaccine is a net good. The Rose article multiplies deaths found in VAERS by the under-reporting factor to conclude there are 205,000 deaths due to vaccination. Even if we ignore that VAERS can’t prove a single death, there is no admission that vaccines have saved 3.2 million American lives. Even with their unsupported assumption, vaccines would save 16 times more lives than they caused.
• One-size-fits-all scaling factor. The Rose article correctly observes that VAERS will under-report mild adverse events much more than serious adverse events. Nevertheless, it then uses VAERS data to compute a single under-reporting factor that it uses for serious adverse events like death.

The Stream article and the supporting Rose document may not be deliberately playing games with the facts to support a political agenda, but that’s certainly how it looks. Whether willful fraud or clumsy errors, its conclusions don’t hold up.

Priests of the different religious sects
… dread the advance of science
as witches do the approach of daylight,
and scowl on the fatal harbinger
announcing the subdivision of the duperies
on which they live.
— Thomas Jefferson,
letter to Correa de Serra, April 11, 1820