MMWR appears to no longer be in the science business
Enter a new study in MMWR— the CDC’s pet journal; It is getting widely tweeted and cited, and that is unfortunate. The paper is entirely, irredeemably flawed. Its flaws are so evident that it should not have been published nor promoted. When an issue is deeply polarizing, publishing bad science helps no one. It cannot convince skeptics, proponents don’t need convincing, and it deepens mistrust in institutions. Let’s consider the paper.
The paper is a case control, test negative study. Basically, anyone in California btw Feb and Dec 2021 who got a COVID19 test could be enrolled. People were told if they tested + or -. Two days later, someone tried to call COVID-positive people. 13.4% of people who tested positive answered the phone. For each positive person, a negative person matched by age and sex who was called was identified. Only 8.9% of people who tested negative answered the phone.
We could stop right now. Very few people answered the phone. Moreover, there may be a big difference in who answered the phone. A person sick enough to be tested for COVID, who was +, might not be doing so well 2 days later. Which ones answered? How about those who tested negative? Are thee comparable people? Sadly, researchers continued….
Among people who answered the phone, people who said they had not spent time in indoor public settings were discarded. People who had a known COVID19 contact were discarded. The analysis just compares those who tested + vs - if they said they spent time in indoor public settings.
It turns out that people who tested positive were different than those who tested negative.
77% of people who tested positive got tested because of symptoms
17% who tested negative got tested because of symptoms
More people who tested negative, got tested “just cuz”
[clip]
What is wrong with this paper?
1. The people who test + and those who test - were seeking testing for totally different reasons. You are comparing sick people to people who wanted to get tested ‘just cuz’ or whose employer was swindled by these testing companies. Some people (more who test -) were getting tested before a medical procedure— these people may be extremely fastidious pre-procedure. I know I absolutely would not want to get sick before an elective surgery, and may even further alter my behavior. (though I am pretty chill)
In case you don’t see how this is a bias, the explanation is that the people getting tested who test neg. are far more affluent/ neurotic/ or precautious and will do anything they are told more than the avg. Joe. They definitely wear masks more. Even if masks did nothing, there would be an association. If you went on twitter for months and told them carrying a Fauci statue in their pocket would protect them from COVID, you could prove Fauci statues work with this design. Put more technically, the root virtue of test negative strategy (comparability) is violated and unmeasured confounding is injected. Paper cannot be salvaged.
2. The response rate is poor. I thought that awful day care study had a low response rate, but this one is lower. When you get a response rate this low you wonder if you are inserting biases you cannot even imagine. Who are the sick COVID people who answer the phone. Are they the least sick ones? The sick ones don’t take calls. Who are the people who answer the phone who test negative? The most anxious ones? The most gullible? ( I never answer unknown callers). Perhaps the ones most likely to carry Fauci statues in their pocket? (aka cloth masks
3. It’s self reported. Mask usage is self reported. A person who was just told they have covid19 might be more likely to assume or believe they must have slipped in terms of mask usage. Self reporting after results are known is a huge bias. A true believer will convince themselves they didn’t wear masks as well as they ought to.
4. The effect size is implausibly large. In the randomized trial in Bangladesh, the arm that reached significance had a 11% relative risk reduction. In this study the same mask had 66% lower odds. That is implausibly large, and should raise red flags.
5. This is protection to the individual. If masks worked this well, they work this well for the wearer. So you don’t need me to mask. Your surgical mask is so massively protective you that mandating I wear a mask is not needed. This runs counter to the rhetoric in this space.