At a press conference last week, Gov. Ned Lamont took a victory lap for choosing to defy federal guidelines and roll out the COVID-19 vaccine almost exclusively by age.
“Our strategy to prioritize those most at risk based on age has contributed to a more rapid decline in people dying from COVID compared to our neighboring states,” his office tweeted.
In addition to allowing us to open up access to everyone sooner, our strategy to prioritize those most at risk based on age has contributed to a more rapid decline in people dying from COVID compared to our neighboring states. pic.twitter.com/0jVFvG1gxQ
— Governor Ned Lamont (@GovNedLamont) March 25, 2021
The number of Connecticut’s COVID-related deaths did decrease more quickly over the time period than neighboring states. But do the data and analysis presented prove the claim that the age-based rollout gets the credit? No, say Saad Omer and Ted Cohen, epidemiologists at the Yale School of Public Health.
Does this mean that Lamont’s claim is necessarily wrong? Also no.
The unsatisfying answer is that the data are limited in what they can show, particularly now, since deaths lag from infection by at least two weeks. Even when the data have a chance to catch up, correlation doesn’t necessarily imply causation, and isolating the relative merits of Connecticut’s largely age-based rollout policy requires a more refined analysis that takes into account a number of confounding factors.
In the coming months, policymakers and scientists will pore over reams of data to assess which approaches to fighting the COVID pandemic were the most successful, but finding clear answers in the data will be a challenge, even after the pandemic ends.
Here’s why.
Death data lags, and states have differed in vaccine allocation strategies at different points over the past few months.
It wasn’t until March 1, when Connecticut opened vaccine eligibility to people 55 and over, that Connecticut officially bucked CDC guidance. Deaths lag infections by at least two weeks, so any change in the number of deaths attributable to rollout of the vaccine to the 55-and-older group wouldn’t have moved the needle much last week, when the governor made the claim.
But age-based rollout is not a new policy to Connecticut, said the state’s chief operating officer Josh Geballe. The state has been prioritizing populations based on age since January and has been a consistent national leader in the administration of first doses. The state carved out a few exceptions to its age-based policy: It also prioritized some congregate settings such prisons and began vaccinating educators and school staff when it expanded eligibility to those above 55.
Connecticut’s neighbors have also prioritized age in their rollouts after prioritizing health care workers and nursing homes, though they have also made teachers, essential workers and people with underlying conditions eligible in tandem at different points in the pandemic. New York announced that teachers and some essential workers were eligible in January along with individuals 75+; Massachusetts included people with co-morbidities in its 65+ rollout in late February. Rhode Island deviated from an age-based strategy around mid-March when it opened up eligibility to teachers, and later to individuals with co-morbidities.
Prioritizing based on age is not a novel approach, said Omer, arguing that what’s really under debate is the decision to continue to do so for people younger than 65. Omer has previously criticized the state’s age-based rollout for de-emphasizing underlying conditions, though he has since described the state’s plan to prioritize individuals with co-morbidities after opening eligibility to all individuals 16+ as a “reasonable middle ground, depending on how it’s implemented.”
The vaccine rollout isn’t the only factor influencing COVID deaths.
Over the time period considered, do Connecticut’s deaths decrease faster than its peers’ do? Yes. Does the state’s vaccine allocation and administration plan necessarily contribute to that? No.
On the other hand, might it have? Yes.
Correlation isn’t causation — meaning, just because things happened together doesn’t mean one caused the other — and the burden of proof for causation is higher. Connecticut also had a number of other initiatives in place to bend the death curve that weren’t reliant on vaccines, such as its own reopening policies. Isolating the effect of vaccines would require epidemiologists to plot out the expected decline in deaths as a product of the other policies in place within each state — and then compare it to the observed decline in deaths. That difference could be attributable to vaccines, Cohen said.
Connecticut chose the state’s nearest neighbors for comparison in part to control for some confounding factors, since the peaks and troughs of cases and infections have largely been in step across states in the Northeast, Geballe said. “Many of the big variables that I would think cause divergence in those curves are pretty well controlled for when you just look at our neighboring states. In terms of seasonality, prior experience, mask wearing rates, interventions on business closures.”
But the demographic profile of those who have died varies across states as well. Even if Connecticut and its neighbors had rolled out vaccines exactly identically, Connecticut may still have been able to reduce deaths faster because rollouts universally began with nursing homes. A study by the CDC and Yale School of Public Health found that vaccines significantly reduced COVID spread in Connecticut nursing homes after just one dose.
Data from the COVID Tracking Project through March 7 suggest that 57% of Connecticut’s recorded COVID deaths have been in nursing homes, while for New Jersey and New York, those reported death numbers drop to around 34-37%. (It’s worth noting that New York’s numbers are an undercount, as Gov. Andrew Cuomo’s reporting of COVID deaths has come under fire.) Figures from Rhode Island and Massachusetts are more similar to Connecticut’s, at 63% and 53% respectively.
While those numbers span both waves of COVID deaths, they illustrate how death profiles can influence the analysis. Using that metric to compare states’ vaccine allocation and administration strategies — without adjusting for this difference in death profiles — can be misleading, Omer said.
States report death data differently
Any comparison between states needs to account for their differing populations. A state that has more people will have more deaths resulting from COVID-19.
While the chart presented last week didn’t adjust for population, the state also looked at the data on a per capita basis and found that the trends were about the same, Geballe said. A CT Mirror analysis confirmed this.
But any comparison between states also needs to account for how different states report their deaths. Rhode Island’s death line looms above the other states in the Northeast on the graph, but any conclusions about the state’s management of the pandemic – or the efficacy of its rollout – must consider that the Ocean State had a data dump of 80 deaths in February from prior months, as the New York Times reported. That renders Rhode Island’s daily reported deaths incomparable to other states over that timeframe.
Will we ever know for sure if Connecticut’s approach is saving more lives than other approaches would have?
In an ideal world, there would exist an alternative Connecticut that differed in just one key respect — that over the past few months, it took an entirely different approach to vaccine prioritization. That is what statisticians and logicians call a “counterfactual.”
Such a state does not exist, and any analysis will be, by definition, imperfect. But with time, epidemiologists can adjust for differing death demographics, reopening policies and reporting anomalies to evaluate the effect of vaccine allocation and administration specifically. That task is only made more complicated as all states converge on opening up eligibility to all adults ahead of President Joe Biden’s May 1 goal. Connecticut made all adults 16+ eligible for a vaccine yesterday, and its neighbors will soon follow suit in the coming weeks.
At the very least, “it’s OK to be open to the possibility that we made a good decision,” Geballe said.