This is a summary, written by members of the CITF Secretariat, of:

Ma X, Li Z, Whelan MG, Kim D, Cao C, Yanes-Lane M, Yan T, Jaenisch T, Chu M, Clifton DA, Subissi L, Bobrovitz N, Arora RK. Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance. Vaccines (Basel). 2022 Nov 24;10(12):2000. doi: https://doi.org/10.3390/vaccines10122000.

The results and/or conclusions contained in the research do not necessarily reflect the views of all CITF members.

A systematic review carried out by the CITF-funded SeroTracker team, published in Vaccines, concluded that third-party or independent evaluations of serology test performance found that manufacturers report slightly higher sensitivityAbility of a test to correctly identify patients with SARS-CoV-2 antibodies and specificityAbility of tests to correctly identify patients without SARS-CoV-2 antibodies of their serological assays targeting SARS-CoV-2. This may have an impact on the validity of estimates and impose bias by under- or over-estimating seroprevalence by up to 9.5%. This study was led by Dr. Rahul Arora (University of Calgary).

Key points:

  • Among 1807 serosurveys, 192 unique commercial assays and 380 self-developed assays were identified.
  • According to manufacturer data, 28.6% of all commercial assays met the World Health Organization criteria for emergency use, which requires 90% sensitivity and 97% specificity.
  • Third parties indicated that manufacturers overstated the sensitivity and specificity of the serology assays by 1.0% and 0.9%, respectively. Independent group evaluations indicated that manufacturers overstated the sensitivity and specificity of the serology assays by 3.3% and 0.2%, respectively.
  • These results imply that there may be more false positives and false negatives than expected by manufacturer-verified test evaluations, with inaccurate sensitivity and specificity statements potentially biasing seroprevalence estimates.

The large number of diverse serological assays used worldwide can affect both the interpretation and the reliability of seroprevalence estimates. The authors caution against accepting the sensitivity and specificities reported by manufacturers and recommend employing third-party or independently evaluated assays which indicate assay properties in a particular context. Further, using multiple testing strategies that provide combined sensitivity and specificity can minimize the risk of bias in seroprevalence studies.

The team used search dates of January 1, 2020, to November 19, 2021, for articles found using Medline, EMBASE, and Web of Science preprints on Europe PMC. All SARS-CoV-2 seroprevalence studies in humans which reported a sample size, sampling date and locale, and prevalence estimate were included. In total, 72,799 titles and abstracts and 4,876 full texts were screened and 1,807 serosurveys were used for the final analysis.