As we seek to get our economies up and running again and loosen shelter-in-place and stay-at-home orders, many have argued how important it is that we have reliable testing for both active COVID-19 infections as well as for antibodies that indicate whether an individual has had a past exposure and has developed some immunity.
Calls for increased testing of course make sense, but there are a few reasons to be cautious about reaching the conclusion that it’s all about testing. I’ve found the perspectives from CIDRAP and its director Dr. Mike Osterholm to be particularly informative in this area. Listen to a podcast that talks about upcoming reagent shortages that may limit testing and read more about antibody testing at their website.
Let’s take a quick peek at serological tests for antibodies. Johns Hopkins has a comprehensive overview of the available tests, both those that have been approved and others. I’m not an expert in these tests, but I did want to speak a moment about their usefulness using some simple ideas from applied probability. To do so, we need to understand test sensitivity and specificity. Sensitivity is sometimes called the true positive rate: given a population of individuals that have received a test, the sensitivity measures the probability (likelihood) that a test returns a positive (correct) result for an individual who has a condition (like a disease or the presence of antibodies). Specificity focuses on the negative results: again, given the tested population, the specificity measures the probability that a negative test results from an individual who does not have a condition.
To express these ideas with probability, let \(T\) be the outcome event from a test, equal to 1 if the test is positive and 0 if negative. Let \(D\) be the event that someone has the condition, again 1 if true (positive) and 0 if not true (negative). Then,
\[Sen \equiv P(T=1 \;|\; D=1) \text{ and } Spe \equiv P(T=0 \;|\; D=0)\]Then, the so-called false positive rate is \(1-Spe\) and this rate is critically important in the case of antibody tests. If someone receives a positive antibody test, then there may be an assumption of immunity. False positives therefore can place individuals at serious risk, and these risks are likely highest for medical professionals and others who are most likely to come into contact with infected individuals.
False positives are a particular concern when the prevalence of the condition is (initially) low in the test population; using the notation, the prevalence is \(P(D=1)\) for a tested individual if the tested population represents a true random sample of the overall population. The Johns Hopkins website notes that one approved serological antibody test has the following estimated sensitivity and specificity: \(Sen = 0.938\) and \(Spe = 0.956\). While these numbers seem reasonably high, let’s focus in on why a test of this quality may lead to tough choices when the prevalence of the condition (acquired antibodies) is low.
Suppose that this test were used on 1M individuals, and suppose prevalence were relatively low such that \(P(D=1)=0.10\); in this situation, we imagine that initially 10 percent of the tested population has been exposed and has developed antibodies (and likely immunity). In this case, the likelihood that someone tests positive for antibodies is given by:
\[P(T=1) = P(T=1 \;|\; D=1) P(D=1) + (1 - P(T=0 \;|\; D=0)) P(D=0)\]which, in this example, is \(P(T=1)=(0.938)(0.1)+(0.044)(0.9)=0.1334\). In a population of 1M tests, that is 133,000 testing positive for antibodies which sounds great! However, since the prevalence is relatively low (0.1) and the specificity is strong but not amazing (0.956), the fraction of positives that are false positives (and thus, false reassurances) is high. We can compute it with conditional probability as follows:
\[P(FP=1) = P(D=0 \;|\; T=1) = \frac{(1 - P(T=0 \;|\; D=0)) P(D=0)}{P(T=1)}\]which, in this example, is \(P(FP=1)=\frac{(0.044)(0.9)}{0.1334}=0.297\). Make sure your interpretation here is clear: you have tested positive, but there is a 30% chance that you do not have antibodies and are not immune. And perhaps more to the point, almost 40,000 people receiving a positive test for antibodies are actually not immune! This false positive risk seems a bit too high to make informed judgements about which individuals are no longer at risk for infection, in my view. If you think about the expressions, what is needed are tests with higher specificity when prevalence is low and with higher sensitivity when prevalence is high.