There is a simple solution to skewed pandemic estimates

There’s something odd about Minnesota’s COVID-19 vaccination data. As of summer 2021, official vaccination rates for older Black and Asian American residents are 100%. This, of course, is wrong. Every demographer recognizes the problem: a

There’s something odd about Minnesota’s COVID-19 vaccination data. As of summer 2021, official vaccination rates for older Black and Asian American residents are 100%. This, of course, is wrong.

Every demographer recognizes the problem: a mismatch between who is counted in the numerator (people vaccinated) and the denominator (population counts). In a changing population, denominators can quickly become obsolete. New data available suggests that the number of black seniors in Minnesota increased by more than 20% between 2018 and 2021, but around the time the state began tracking vaccinations, the most recent available denominators were outdated for years. Pairing 2021 vaccination figures with population estimates from just a few years ago severely overestimates vaccination rates.

An erroneous denominator also sparked alarm in England when it incorrectly suggested that COVID-19 infections in residents aged 40 to 79 were higher in vaccinated people than in unvaccinated people. The problem? An overestimated denominator led to infections being linked to an unrealistic pool of unvaccinated people, which lowered the apparent infection rate.

Excess mortality is particularly sensitive to bad denominators, a working tool that captures how many more people have died than would normally be expected; too small a denominator inflates estimates of excess mortality rates in two ways. If the estimates assume that a population has fewer people than it actually has, we expect too few deaths – and the resulting overestimation of the number of “excess” deaths is attributed to a population that is too small.

Denominators err by being outdated (as in Minnesota) or by drawing numerators and denominators from sources capturing different groups of people (as in England). Even slightly outdated denominators quickly become inaccurate when a drastic event disrupts births, deaths, or migrations. Major social crises often disrupt all three.

To solve the problem of the denominator, demographers, specialized in measuring populations, must unite. It’s time for them to provide detailed guidance on constructing denominators when official data is outdated. We need collaboration that can unify efforts to address outdated denominators and, most importantly, make them accessible to epidemiologists, health officials, and others.

Minnesota’s vaccination data illustrates the impasse in which researchers and policymakers find themselves. It’s because the state has done such a glowing job of offering helpful numerators, reporting vaccinations by race and age together, that its denominator problem is so acute. As subpopulations become more defined — as in these older black populations — the data becomes more useful for understanding inequalities and planning health campaigns, but the relevant denominators are the least up-to-date.

As an individual researcher, I can adjust the denominators in any way that best suits my research question, making reasonable guesses about the evolution of a population. But public health officials working in highly polarized environments are loath to do so for fear of being accused of cooking the numbers and playing politics. It’s understandable that they default to unadjusted – but outdated – data.

And the freedom of individual researchers to adjust the denominator comes at a cost: when each article constructs the denominators differently, the results cannot be easily compared between studies. Public health officials tell me they are frustrated that each study makes different adjustments. And even experienced researchers might not realize how much their estimates depend on how they update the population (or not) — some excess mortality analyzes don’t even say what denominator data is used. I myself was amazed at how important this can be in my own work.

Instead of each analyst’s situation for themselves, we need a collective solution – one that would be authoritative, clear, and enforceable enough for demographers to rely on and civil servants to follow. adopt.

What would that entail? We would need specific, open-source software routines to update older data with recent statistics (births, deaths, migrations) and to combine newer data with projected trends for specific populations. A collective solution would also require guidance, based on empirical and what-if analyzes that answer questions such as the magnitude of a demographic shock (such as a spike in deaths from an infectious disease) before it is a bad idea of ​​projecting older trends (which might no longer apply) into the present? Such questions rarely matter much in normal times. In a crisis, they can be crucial.

Demographers are well placed for this effort. We work in various academic disciplines, so we can draw on a wealth of expertise, for example, measuring mortality and migration, estimating populations in small areas and developing forecasting models.

Consensus is crucial, as public officials can only adopt techniques that are accepted as legitimate and impartial. A major international initiative could produce something really useful in a year and a more complete set of answers in a few years. Reliable estimates are needed for this and subsequent pandemics. They will be needed as climate and other disasters hit, allowing us to detect deaths even if reporting practices lag.

The best approach to future crises is to start now.

Competing interests

The author declares no competing interests.