Only one day after releasing downgraded projections about the COVID-19 pandemic in the United States, an influential model often cited by health experts is still projecting hospitalization numbers that are much higher than the actual numbers in a large number of states.
The Institute for Health Metrics and Evaluation at the University of Washington, a preferred model of the White House coronavirus task force, has been, to its credit, on the lower — and consequently more accurate — end of the spectrum as far as models go.
In late March, IHME projected that more than 90,000 Americans would die as a result of the virus. On Wednesday, that projection was revised to just over 60,000. That number is significantly lower than the 100,000-240,000 predicted by the White House task force, and massively lower than the 2.2 million U.S. deaths predicted by the Imperial College in London model — though that model’s author said that social distancing wasn’t taken into account for the number.
But even after IHME researchers revised their models downward on Wednesday, they still overshot the number of hospitalizations that would occur for even the next day.
To state the obvious, it is easier to predict the near future than the far future, and one would expect more uncertainty regarding predictions that are for events that are in the distant future. The fact that the IHME models did such a dismal job of predicting the very next day ought to be a source of concern for the viability of the rest of the model’s predictions.
What are the details?
Using data from The COVID Tracking Project, which collects reporting from individual states on COVID-19 testing, hospitalizations, and deaths, TheBlaze discovered that IHME’s hospitalization numbers were in most cases significantly overestimated, often doubling or tripling the actual hospitalization numbers.
Below is a chart showing the discrepancies between the projected number of hospitalization and the actual number of hospitalizations in states that reported totals on that day.
Note: the projected totals were taken from the April 8 revised IHME model and not subsequent updates.
Arkansas: Projected 188 / Actual 73
California: IHME projected that 4,386 hospital beds would be needed for COVID-19 patients on April 8 — the day the revised projections were released — just 2,825 were hospitalized at the time.
Connecticut: IHME projected that 3,686 hospital beds would be needed on April 9, but data shows that only 1,464 were hospitalized on that date. To make matters worse, IHME researchers concluded that the state only had 1,738 hospital beds available in the state. Their projections far exceeded the number of total beds available, but the actual number has yet to exceed capacity.
*Florida: IHME projected that 4,449 hospital beds would be needed on April 9, but to-date the cumulative number of hospitalizations in Florida has only been 2,422.
Iowa: Projected 329 / Actual 115
*Massachusetts: IHME projected that on April 9 there would be 4,705 hospitalizations, but to-date there have only been 1,747 cumulative hospitalizations in the state.
Michigan: The April 9 number was projected to be 5,781. The actual number on that date was 3,826.
Minnesota: Projected 430 / Actual 145
Montana: Projected 70 / Actual 13
Nevada: Projected 777 / Actual 282
New Jersey: IHME projected there would be 12,801 hospitalizations on April 9, but on that date there were actually only 7,363. Like Connecticut, New Jersey’s hospital bed capacity, 7,815, has been exceeded only by the projections.
New Mexico: Projected 365 / Actual 73
New York: In the state hardest hit by the virus, IHME’s numbers were closer, but still too high in the days following updated projections. The projections say 21,943, while the actual number is 18,279.
North Carolina: Projected 1,026 / Actual 398
North Dakota: Projected 392 / Actual 14
Oklahoma: Projected 647 / Actual 186
Rhode Island: Projected 624 / Actual 160
*Tennessee: Projected 957 / 505 cumulative
Texas: Projected 2,030 / 1,439 Actual
Vermont: Projected 75 / Actual 33
Washington: Projected 994 / Actual 639
It should be noted that IHME arrives at their projected number within an estimated range. These ranges give IHME huge leeway in their estimates, which brings into question their usefulness, especially in larger states.
Moreover, the actual totals are consistently on the very low end of their ranges and far below the most likely result. And in at least four states, they are outside their margin of error already after only one day.
There were, of course, outliers to the trend. In 5 states that had current hospitalization reporting — Delaware, Louisiana, Missouri, Pennsylvania, and Virginia — IHME’s projections either matched the actual number of hospitalizations or underestimated the number. But in the majority of states where current hospitalization statistics were available, the IHME projections updated just days before were way off.
How is that even possible? Are current statistics from the states not being used as a baseline for the models?
Maxing out or exceeding hospital capacity was held out by health experts as the real danger in the early days of the coronavirus pandemic. The need to shut down the country in order to “flatten the curve” was justified, in large part, by predictions that the U.S. health care system would be overwhelmed if we didn’t.
As such, it is appropriate to at least question the narrative when influential models predicting the extent of the coronavirus outbreak can’t seem to chart out the hospitalization figures with much precision for even one day.
TheBlaze reached out to IHME for comment, but the institute did not respond in time for publication.