Experts Reveal ‘Major Shortcomings’ With FDA Analysis of Safety Outcomes in COVID-19 Vaccinated Recipients

JHVEPhoto/Shutterstock

The U.S. government’s safety surveillance system monitoring COVID-19 vaccine adverse events is “woefully inadequate” and may be missing safety signals, according to researchers who say the U.S. Food and Drug Administration (FDA) made multiple decisions to ensure its first published analysis only identified known safety signals.

In a peer-reviewed letter published June 16 in the journal Vaccine, a team of experts revealed “major shortcomings” with the FDA’s near real-time surveillance study assessing outcomes in U.S. COVID-19 vaccine recipients.

Dr. Joseph Fraiman, an emergency room physician associated with the Baromedical Research Institute in New Orleans, and his co-authors raise serious concerns about whether the surveillance system is fit for its purpose and how the FDA performed its analysis.

“The FDA has repeatedly stated that it is conducting intensive, historically unprecedented monitoring of COVID-19 vaccine safety and that the only serious harms associated with mRNA COVID-19 vaccines are anaphylaxis, myocarditis, and pericarditis,” the researchers said in an email to The Epoch Times. “However, in our letter, we detail why the U.S. government’s safety surveillance system is woefully inadequate and, as a result, potentially missing safety signals.”

In its first-ever surveillance analysis published Oct. 26, 2022, in Vaccine, the FDA assessed 17 adverse outcomes following COVID-19 vaccination with Pfizer, Moderna, and Johnson & Johnson’s vaccines and concluded 15 outcomes did not meet the threshold for a statistical signal.

The FDA based its analysis on medical and pharmacy claims data of 16 million vaccinated individuals aged 12 to 64 from Optum, HealthCore, and CVS Health using their Biologics Effectiveness and Safety (BEST) System—an active post-marketing surveillance program to ensure the safety and effectiveness of biologic products, including vaccines.

The FDA concluded myocarditis and pericarditis met the requirements to trigger an early detection safety signal for Pfizer’s COVID-19 vaccine in two of three large commercial insurance databases assessed, while anaphylaxis met the statistical threshold for Pfizer and Moderna vaccines in all three databases.

The agency did not detect any other adverse outcomes, including those previously acknowledged. Their results, the FDA said, were “consistent with published literature.”

FDA Analysis ‘Not Sensitive Enough’ to Detect Safety Signals

In the letter to the editor, researchers said the FDA only identified COVID-19 vaccine safety signals for already established adverse events, and the analysis was not sensitive enough to detect safety signals for some known adverse events, such as myocarditis.

Myocarditis is inflammation of the heart muscle that can lead to cardiac arrhythmia and death. The heart condition is a recognized side effect of the mRNA COVID-19 vaccines, according to previous research and medical examiners. Yet the FDA did not detect myocarditis for Moderna’s COVID-19 vaccine in any data source and only detected it with Pfizer’s vaccine in two of three sources.

“This raises serious concerns about whether the surveillance system is fit for its purpose,” the researchers wrote. “Another major concern is the FDA’s approach towards false positives.”

The study also “failed to identify a single new positive despite running several hundred different analyses,” suggesting the system is “too strongly weighted toward avoiding false positives and will too easily miss true positives,” they added. “The FDA made multiple decisions to ensure their surveillance did not identify false positives at the expense of sacrificing the ability to identify true positives.”

According to the letter, safety surveillance systems should be optimized for high sensitivity—erring on the side of caution—to ensure real safety problems are not missed. A highly sensitive approach will result in some false positives, but upon further study, can quickly be identified as a true positive or false positive.

“In contrast, because fewer associations are identified at the surveillance stage, fewer associations will result in further study, and more true associations will be missed,” the researchers wrote.

FDA Used Test Margin to Minimize Risks and Reduce Harms

Experts also raised concerns the FDA used a test margin for its analysis for each adverse event of special interest based on “expert guidance to avoid minimal risk increases that were ‘unlikely to be clinically relevant.’” Yet the agency did not provide details concerning how or which experts determined whether a risk was “minimal” or “unlikely” to be clinically relevant.

“Given a vaccine administered to billions, we are concerned that even minimal risk increases would imply harm to thousands, or perhaps millions, of younger people, many of whom may be at low risk of serious complications from coronavirus infection,” they wrote.

In an email to The Epoch Times, the researchers said allowing anonymous experts to “determine something they called testing margins that they felt represented the level of increased risk considered ‘unlikely to be clinically relevant,’ translates to setting thresholds for the system that will not identify whether a true increase in harm below this threshold is occurring.”

“For example, for 10 harms, including heart attacks, blood clots, bleeding disorders, strokes, brain bleeds, and others, they set the testing margin at 1.25, which to our best interpretation would mean that if the vaccine actually increases these harms by less than 25 percent—the system would not signal the harms,” they said.

“The FDA offered no rationale for how these experts determined these testing margins, which is concerning because, for many of these harms, an increase in risk of 10-20 percent would typically be considered highly clinically relevant.”

The FDA “tightened the statistical significance margin such that fewer associations would reach the threshold of detection,” resulting in fewer associations identified at the surveillance stage and increasing the chance true positives are missed and become false negatives, they added.

“The FDA gave no power calculation, and the rationale provided for its choice was to avoid false positives. It appears this decision was made without weighing that concern against concerns about false negatives.”

FDA Fails to Pool Data and Excludes 43 States From Analysis Without Rationale

Another shortcoming of the FDA’s analysis was the agency’s decision not to pool data from its three data sources—increasing the chance of missing true associations—and its exclusion of Immunization Information Systems data from 43 states.

“Pooling data would increase power and we see no valid rationale not to do so,” the researchers wrote. “Many of the same authors previously pooled these same three data sources to investigate myocarditis risk from the vaccines.”

Although the FDA acknowledged it only provided data for seven states, the agency provided no rationale for excluding critical data from 43 states. According to the researchers, including data from other states would “increase the power of the analysis, improving the ability to identify harms.”

The FDA’s results demonstrate serious problems with the agency’s surveillance system, researchers said. “True pharmacovigilance requires that safety signals be detected and reported without delay.” Yet, the FDA did not publish its results until 21 months after the COVID-19 rollout.

“As no preprint or other public announcement of the results was made prior to publication, this cannot be realistically termed a ‘near real-time’ surveillance system,” they added.

The letter was authored by Fraiman, Dr. Juan Erviti, a hospital pharmacist and pharmacoepidemiology researcher, Dr. Mark Jones, a biostatistician and associate professor at the Institute for Evidence-Based Healthcare, Dr. Sander Greenland, emeritus professor of epidemiology and statistics at the University of California–Los Angeles (UCLA), Dr. Patrick Whelan, associate clinical professor of Pediatrics in the Division of Rheumatology at UCLA, Dr. Robert Kaplan with the Stanford School of Medicine Clinical Excellence Research Center, and Dr. Peter Doshi associate professor at the University of Maryland School of Pharmacy and senior editor at The British Medical Journal.