COVID-19 Response AAR
What we did right and what we did wrong.
After Action Review
It has now been over two years since America realized that COVID-19 was going to be a big deal1 and started taking action. Given where we are with the pandemic, it’s past time to take a look at what we did in light of what we now know. The point of this kind of AAR is not just to make judgements on what happened or how we reacted. The purpose here is also to reevaluate the decisions that were made based on our current understanding so that going forward we don’t repeat the mistakes we made the first time around. Because the way things look now, we’re on the path to do exactly that when the next wave or pandemic hits.2
I’m going to break this down a number of different ways. First I’ll look at what we got right, what had mixed results, and what we did wrong. Where it matters, within each category I’ll divide the analysis into the different phases of the pandemic. Those phases are:
Initial Outbreak
Vaccine deployment
Delta wave
Omicron wave
What marks these phases is that they represent either a change in our relationship to the virus or a change in the virus itself. These changes should have lead to changes in how we responded. In some cases they did, but not always. I should also note that I’m focusing almost entirely on the reaction in the United States. Finally, if you really want the best info on how the virus spread across the different regions of the country and what the data really means, Matt Shapiro’s Marginally Compelling newsletter is mandatory reading. He did the best work I’ve seen over the past two years of charting the data and providing rock solid analysis.
First, a general observation. One assumption that is constant among all government and bureaucratic types is the unvarying belief that whatever action was taken made the situation better. There doesn’t have to be any evidence that the measures worked. Challenges to action X are inevitably met with “But think how much worse it would have been if we hadn’t done action X!” This would be like me watching the Broncos lose a close game when I’m wearing my Broncos jersey, and then saying that because they almost won I need to wear my Broncos jersey and my Broncos socks and my Broncos hat for the next game and then they'll win. As if what I wear has anything to do with how well the team plays. This refusal to admit that an action was pointless (or even worse, actually harmful) is a huge obstacle to improvement. It’s why policies that should have been discontinued after weeks have lasted months or even years. Virginia Hume gets into this in much more detail in her excellent essay Our Failed COVID Response over at The Dispatch. It covers much of the same ground as I do here in the “What we got wrong” section, but does it much more thoroughly and clearly. It should be read by every policy maker and public health official daily until the lessons sink in.
What we got perfect
crickets chirping
Okay, perfect is a pretty high standard. Not one I’d expect bureaucracies and governments agencies not named NASA3 to meet very often. (If ever.) If anyone can think of something that belongs here, let me know in the comments. Moving on.
What we got right
Unquestionably our biggest success has been the development and deployment of the vaccines. It wasn’t perfect, but the benefits have been huge and the hiccups were relatively minor. Another success has been the development of effective treatments for the virus from the monoclonal antibodies to Paxlovid (nirmatrelvir/ritonavir).
First things first. I have many, many criticisms of how the Trump administration (and President Trump specifically) handled the pandemic, but those are almost entirely outweighed by Operation Warp Speed. If you had told anyone in 2019 that we could have multiple vaccines developed and approved for a novel virus in less than a year, they would have called you crazy. And they would have been right! Especially if you added in that some of the vaccines would use mRNA, something never before used in an approved vaccine. Many experts stated confidently that it was impossible. But we did it. Moreover, the vaccines work amazingly well at preventing serious illness and death. They have been the game changer. More on that in a bit.
Now, about those hiccups. For as fast as the FDA approved the vaccines, they could have been approved faster. There are very strong arguments that when hundreds of people are dying each day, waiting on efficacy testing for approval is not just wrong but immoral. I realize that the emergency authorizations moved at light speed for the FDA, but that’s because the process is so painfully slow to begin with. Bureaucracies will always be risk averse, but that’s where the administration should have done a better job. Preliminary approvals for high-risk populations should have come once the safety testing was completed and showed a positive benefit. Knowing exactly how beneficial could wait.
The FDA also erred in pausing the J&J vaccine. It’s very clear in the data that the pause damaged confidence in the vaccines and slowed the deployment. It provided ammunition to the anti-vaxx crowd and was unnecessary.
The Republican Party and conservative movement in general erred in letting the anti-vaxx movement take hold in their ranks. It wasn’t just a tactical or political error either. Lots of preventable deaths happened because people didn’t trust the vaccine. Prior to the vaccines, conservative leaning states didn’t stand out in the number of deaths per capita. Indeed, they often did better than progressive states. But as the vaccines rolled out, it those states start to climb the charts. Once the Omicron wave hits, almost all the states at the top are dominated by conservatives. Vaccines (and the lack thereof) aren’t the only reason for this, but they’re almost certainly a major factor. Letting your own voters die is generally a bad idea for a political party (outside of Chicago that is.)
Finally we get to vaccine performance itself. As I stated, across all phases of the pandemic and all variants they have drastically reduced the rates of hospitalization and death. That was by far the most important function of the vaccines. Second in importance was preventing the spread of the virus. We wanted them to be sterilizing vaccines, and they seemed to be doing well with the original virus. Breakthrough infections were initially rare. Unfortunately they weren’t as successful in that regard as COVID-19 mutated over time. They were less successful preventing infection with the Delta variant and do even worse at stopping infection by the highly contagious Omicron variants. Without that ability to prevent the initial infection, we were never able to come close enough to true herd immunity with vaccines alone. (Omicron might have gotten us there. Until the next mutation anyway.)
The speed of the vaccine development and the success of it preventing serious illness and death put the vaccines firmly in the good category. Delays in the approvals, missteps in the deployment, and bad luck with mutations keep it out of the perfect.
Where we had mixed results
Lockdowns, shutdowns, restrictions, social distancing… we tried all kinds of different strategies to “flatten the curve” and “slow the spread” of the virus. While the general consensus seems to be that these all failed, the actual record is mixed. In theory of course, lockdowns are the perfect solution. If the virus can’t spread, it will die out. In the real world however, there’s no way4 to do lockdowns that are drastic enough to make that theory work. The initial restrictions in March of 2020 were too much, too soon for most of the country. We locked down in places where there was no threat, damaging those economies and building resistance to future restrictions. Some of the restrictions were just plain stupid, even based on what we knew then. Arresting people for surfing alone, closing down specific aisles in stores to prevent selling of some goods but not others, chaining up playgrounds, banning elective surgeries… overzealous restrictions like these were useless and counterproductive.
But, where the community spread was extremely high in 2020 (pre-vaccine), targeted restrictions could make a difference. Their true value was in communicating to people the need to exercise caution. Once the vaccine became available, however, the calculus changed. Shutting everything down (or even just some things down) made no sense. But it was the people who chose to remain unvaccinated that paid the price.
What we got wrong
Where do I begin? Masks. Masks is where to begin when it comes to bad policy. Especially masks on kids5, which ties into the second way we got things wrong, school closures. Our overly technocratic rules and metrics also compound our failure to respond to the actual nature of the pandemic. The illogical obsession with zero-COVID did considerable damage as well.
Like the flu and other similar diseases, COVID-19 is most dangerous for the elderly. Unlike those it’s not significantly dangerous for young children (in the absence of other conditions). We’re used to dealing with viruses that are just as bad for kids as they are for senior citizens. COVID-19 is different, but we ignored that in all too many cases. About a thousand adults died for every kid under age 18 who died with COVID-19. It’s roughly three thousand to one if you look at kids under age 4. And the vast majority of those adult deaths occurred in people over the age of 50. We acted like children were in just as much danger as adults, when the exact opposite was true. We should have been hyper-focused on protecting the elderly, but instead we closed playgrounds and kept kids out of school. Worst of all we let kids think that they were in danger when they really weren’t. We taught them that they were the carriers of the disease instead of the ones who had the least to worry about.
That leads to school closures. Like the lockdowns discussed above, it wasn’t unreasonable for school districts to be cautious during the first phase of the pandemic. Closing schools when community spread was high was not an unreasonable decision then. It took time to be clear that kids weren’t at the same risk of getting and spreading the virus. But once that was clear, and especially once vaccines were available, schools should have reopened right away. Going forward, closing schools should be off the table. Not only were the benefits minimal (at best), the costs were extraordinarily high. Some students were able to maintain their academic progress during remote learning, but those who couldn’t fell rapidly behind. The students who were already struggling suffered the worst. Some of those kids will never catch up. Our schools don’t really do remedial education any more. If you don’t learn what you’re supposed to in first grade, too bad. It’s time to move on to second. And if what they missed was foundational (as it almost always is in the lower grades), everything going forward will suffer as well.
Back to masks. Amazingly enough, we got this right when the pandemic first started. The experts looked at the data and said cloth masks won’t help, and we need to keep the N-95 and disposable paper masks for medical professionals and first responders. And then they changed their minds. All of a sudden cloth masks were encouraged, then quickly they became mandatory. Now cloth masks aren’t useless; they’re just the next best thing. But even a marginal something is better than nothing was the theory, and so masks became the symbol of compliance with the various COVID-19 regulations. Confident that the cloth mask they were wearing (often improperly) would protect them, some people went out and ended up getting sick. I don’t know that there’s any way to disentangle those deaths from the ones that would have happened anyway, or that we could really calculate how many lives were saved because of masking. But proponents of masking assume that the former is low and the latter is high, despite the lack of evidence to support their position.
During the initial outbreak phase and while the vaccine deployment was getting started, you can possibly make the argument that even the miniscule amount of protection offered by cloth masks was worth it. Once the vaccines were widely available however, that rationale evaporated. Mask mandates only distracted us from focusing on vaccinations.
As I said, masks became a symbol for those who wanted to signal that they were taking the virus seriously. And what becomes a symbol for one side will be a symbol for the other side as well. It became impossible to discuss masking rationally. Still, we should try. As mentioned, any masking rule that allows cloth masks is pointless. There’s some decent evidence that properly worn N-95 masks and surgical grade masks reduced spread. The key words there are “properly worn.” Considering the challenge of supplying everyone with the appropriate masks and getting them to wear the masks correctly, broad mask mandates don’t make sense. Targeted mask mandates, like in nursing homes and assisted living facilities, should be used when appropriate. Pointless mandates, like the transportation one that the federal government refuses to end, should be dropped immediately.
Another big, big failure was that the people making the rules didn’t follow the rules. Whether it was that they didn’t think the rules were effective or that they just felt like they were above the rules, there are countless examples of politicians and bureaucrats flaunting the restrictions that they had imposed. A similar problem is when the restrictions were waived or declared unimportant for political reasons. We can say now that outdoor protests are probably not very risky, but that’s not why the rules weren’t applied to protests in the summer of 2020. It was explicitly stated in some cases that the specific issue being protested outweighed the need for restrictions. That was a mistake. The rules have to be consistently applied.
Finally, lots has already been written about the CDC and their missteps at the beginning of the pandemic. Their insistence on controlling all the testing directly (and then screwing up the test kits they sent out) set us back weeks when every day mattered. Bureaucratic inertia and risk aversion are a problem during normal times, but the culture of the CDC was not prepared for dealing with a pandemic. It took them way too long to realize they were the problem with the testing. They still don't see that their zero-risk mentality is hindering our success. The virus was always going to be endemic. Policies and messaging with zero-COVID in mind are doomed to fail, but we’re still getting that from the public health bureaucrats.
What we got really wrong
In one word: China. We still do not know how COVID-19 started, and apparently have no real interest at high levels at finding out. A lot of experts seem very convinced that the SARS-CoV-2 virus which causes COVID-19 evolved naturally and spread from bats to humans, with possibly another animal host in between. From there it either made its way into the wet markets in Wuhan or somewhere in that vicinity and begin to spread. Other experts are equally convinced that the virus came from one of the only facilities in the world studying those types of viruses, the Wuhan Institute of Virology. Both sides have some good arguments, but those on the natural origin side have some big incentives for motivated reasoning. If you’re an expert in these types of viruses, do you really want to be the one to say that studying the viruses caused the problem?
I should make one thing perfectly clear: I’m not in any way talking about the fantasy of SARS-COV-2 being designed in a lab as some nefarious biological weapon by the Chinese government. Not only is there no evidence to support the idea, but it’s just unbelievably stupid to begin with. For starters, that’s just not how bio weapons work. The real lab-leak theory is that scientists at the WIV were studying the SARS-CoV-2 virus for legitimate and beneficial reasons but made some mistake and allowed it to start spreading. I’m not going to argue for or against the lab-leak theory here, but I do think it’s important we find out as much as we can.
Why does it matter where SARS-CoV-2 originated? I can give you six million reasons why it matters. That’s at least how many people died worldwide (so far), and the actual number may be double that. If a mistake at a Chinese government lab caused millions of deaths, shouldn’t there be some consequences for China? I think that’s a question all too many people are happy to avoid, and if we never know how the virus came to be then we won’t have to worry about it.
Even if the natural origin theory is correct (because sometimes unlikely coincidences do happen), that shouldn’t let China off the hook. The Chinese government did its best to deny the virus existed, and when they couldn’t do that anymore, to downplay the severity of it until it was too late. Their lies, deceptions that caused the deaths of hundreds of thousands of people, have been completely ignored because it’s too difficult to deal with. Rather than take on the challenge of imposing consequences on the Chinese government, we decided that any acknowledgement of the Chinese origins of the virus was racist and memory-holed the entire debate. China (and other autocratic governments) now know they can freely lie to the world and we won’t say boo. That’s the biggest failure of the pandemic.
Conclusions
It could have been a lot worse. If the SARS-COV-2 virus had followed the usual pattern of being dangerous to old and young alike, we would have easily broken a million deaths here in America. Society would have been torn apart. Without the vaccines we would have been devastated by a variant like Omicron. But, aside from the vaccines, we could have done much, much better in our response. Here are the biggest takeaways from the last two years:
Focus efforts on the most vulnerable populations.
Targeted restrictions using recommendations, not mandates, at the local level.
Clear metrics for the imposition and ending of the restrictions with common sense oversight.
Less theater and virtue signaling, more honesty and humility.
Do I think that we’ll learn these lessons? I’m not optimistic. Our public health “experts” have shown no sign that they’re ready to learn from their mistakes. They can’t even admit mistakes were made. But we do have a way to make it more likely that the next pandemic isn’t bungled: elect politicians who don’t worship at the alter of bureaucracy.
It’s not real until Tom Hanks gets sick and the NBA stops playing.
Right on schedule, the city of Philadelphia has reinstated their mask mandates due to poorly defined criteria. https://polimath.substack.com/p/2-mask-2-furious/ [Those mandates have now ended, just days after being reimplemented.]
While NASA gets it amazingly close to perfect on critical occasions like the Mars Rover landings and the recent James Webb telescope launch, there’s a lot of room for improvement too. See https://www.commentary.org/articles/james-meigs/good-nasa-vs-bad-nasa/
China would like us to believe that it’s a different story and that their lockdowns worked for the original virus, though it’s hard to say given that we can’t trust a single word out of their government. What we're seeing now in Shanghai shows that no lockdown can be tight enough to stop Omicron.
Read Mary Katherine Hamm’s article on kids and resilience in The Atlantic: https://www.theatlantic.com/ideas/archive/2022/01/protect-kids-faulty-risk-assessments-covid-resilience/621353/