A perspective on observed negative vaccine effectiveness

A perspective, published in the International Journal of Infectious Diseases by authors of this CITF-funded study, reported that various biases in how data are collected and analyzed could lead to false negative conclusions regarding vaccine effectiveness (VE). Such potential biases should be addressed when communicating real-world immunity research. This paper is a collaboration between Drs. Sharmistha Mishra and Jeffrey Kwong (University of Toronto).

The authors suggest that this study provides important perspective in response to studies that attributed negative VE to vaccines during the Omicron era. These findings caused a media flurry and raised concerns about whether vaccination should be the primary measure for controlling the COVID-19 pandemic.

Key findings:

• Biases related to negative VE measurements influenced the estimates of:
• the true levels of infection, including asymptomatic infections, by vaccination status (e.g., infection after a certain number of vaccine doses); or
• the observed levels of infection, including asymptomatic infections, by vaccination status.
• Mechanisms of bias can produce false negative VE when they are based on data showing vaccinated individuals becoming infected at higher rates than their unvaccinated counterparts for a variety of reasons. These biases are related to uncontrolled and often unknown differences in contact, exposure, susceptibility, and immunity.
• Results suggesting negative VE may be influenced by:
• Top-down testing policies (e.g., clinical or employment-based criteria for who has access to testing), which can vary across jurisdictions and institutions (e.g., different hospitals may have different testing policies).
• Individual testing behaviours, which can be shaped by experiences such as living in households with individuals at greater risk of severity (e.g., older adults and/or persons with compromised immune systems).
• VE can be underestimated (and negative VE observed) when vaccinated individuals have either more access to testing or are more likely to seek testing.
• Future communication could benefit from interpreting the findings of one or more negative VE measurements. Other methods that could be used to address negative VE are –
• systematic SARS-CoV-2 testing, infection and symptom data collection to reduce bias related to both differential testing and unknown prior infections.
• Quantitative bias analysis (QBA), which involves a process of systematically examining and testing for the potential impact of systematic errors. For example – contact and testing surveys could help elucidate whether contact, exposure or testing vary by vaccination status and, if so, provide an estimate of the strength of the relationships.

Overall addressing false negative VE reporting will help improve our ability to interpret existing and future VE studies and create opportunities to develop new frameworks and methods that can generally advance how real-world immunity research is conducted.