Wendover’s COVID-19 Testing Explainer Gets the Math Wrong—and Oversells the Solution
Wendover Productions set out to explain a genuine problem in July 2020: America could collect COVID-19 samples faster than its laboratories could process them. Its account of Quest Diagnostics' overnight aircraft network, supply shortages and testing delays provides a useful picture of the crisis.
But the episode becomes considerably less reliable when it starts calculating solutions. Its most important pooled-testing example contains a mathematical error that reverses its conclusion. Its $3–$5 testing estimate comes from a theoretical research model, not an operating testing system. And its claim that wastewater surveillance detects outbreaks a week ahead of clinical testing needs an important qualification.
These are not reasons to dismiss pooled testing or wastewater surveillance. They are reasons to stop presenting promising research as established operational fact.
The 20% example contains a fundamental probability error
Wendover explains pooled testing using groups of four people. One laboratory test screens the entire group; if the result is positive, all four samples are tested individually.
The video correctly explains that this saves resources when relatively few people are infected. But its calculation at a 20% positivity rate is wrong.
The standard probability formula for a pool of four is:
\[ P(\text{positive pool})=1-(1-p)^4 \]
Here, \(p\) represents the probability that an individual sample is positive. This formula accounts for the possibility that several positive samples end up in the same pool. It is the established calculation used in the mathematical literature on Dorfman pooled testing, including an April 2020 research paper from MIT. MIT's pooled-testing research.
idss.mit.edu
Consider the video's two examples.
At 8% individual positivity:
- 100 people produce 25 pools.
- The actual probability of a positive pool is approximately 28.36%, not 32%.
- Approximately 7.09 pools require individual retesting.
- Expected processing requirements: approximately 53.36 tests, rather than the video's 57.
The difference is modest, but the next example is much more consequential.
At 20% individual positivity:
- 100 people produce 25 pools.
- The probability of a positive pool is 59.04%.
- Approximately 14.76 pools require individual retesting.
- Expected processing requirements: approximately 84 tests, not 105.
Wendover assumes that 20 of the 25 pools will require retesting. That effectively treats the 20 positive individuals as though every one occupied a different pool.
They do not.
When multiple positive samples occupy the same pool, they require only one initial pooled test before individual retesting.
The result is significant: under the standard independent-sample model, four-way pooling still saves approximately 16% of processing runs at 20% individual positivity.
The theoretical break-even point for four-sample pooling is approximately 29.3% positivity, assuming perfect detection and no additional operational costs.
That does not mean pooling is automatically appropriate for every hotspot. But it does mean the video's calculation incorrectly declares the technique less efficient than individual testing at 20%.
The arithmetic actually makes pooling look less useful than it is. This is precisely the sort of mistake that changes a reader's understanding of a proposed public-health intervention.
The $3–$5 estimate is not the price of a COVID test
The video introduces research from three UC Berkeley researchers: Ned Augenblick, Jonathan Kolstad and Ziad Obermeyer, together with economics doctoral student Ao Wang.
Their July 2020 working paper, Group Testing in a Pandemic: The Role of Frequent Testing, Correlated Risk, and Machine Learning, proposes combining frequent testing with risk-based grouping.
The idea is interesting. People who interact frequently could be grouped together, while different pool sizes could be assigned to populations with different infection risks.
But Wendover compresses several important qualifications.
The video states:
"Hypothetically, a pooled testing system designed by machine learning could lower the cost of a test to just $3 to $5."
The actual paper estimates that daily group testing could cost $3–$5 per person per day, assuming a marginal testing cost of $50. It explicitly states that a formal cost-effectiveness analysis is beyond the paper's scope.
faculty.haas.berkeley.edu
That distinction matters.
A modeled daily screening programme costing $3–$5 per person per day is not the same thing as a laboratory offering an individual PCR test for $3–$5.
The estimate is conditional on the proposed system achieving its modeled efficiencies. It is not a measured commercial price or a demonstrated reduction in actual laboratory expenditure.
The machine-learning system had not actually been built
There is a second, more important limitation.
The Berkeley researchers did not develop and validate a functioning machine-learning system for COVID-19 testing.
Their paper explicitly states that they stop short of developing a specific algorithm because prediction methods would need to be tailored to individual testing environments.
Their simulations assume that the system can accurately estimate infection risk and relationships between people.
One example divides a hypothetical population into:
- Low-risk individuals with a 0.1% infection probability.
- High-risk individuals with a 5% infection probability.
The model assigns groups of 40 to low-risk individuals and groups of 10 to high-risk individuals. Under its assumed conditions, the researchers calculate an 82% reduction in expected testing requirements.
But when the risk information is wrong, efficiency deteriorates substantially. In one scenario, expected tests rise from 0.18 times the population to 0.58 times the population.
faculty.haas.berkeley.edu
This is not a minor technical detail. The proposed system depends on knowing who is actually at risk and which individuals are likely to share infection patterns.
The paper itself recognizes that risk estimates can be uncertain and constantly changing.
The difference between the concrete Quest system and the Berkeley proposal is also substantial. The FDA's July 18, 2020 authorization permitted Quest to pool up to four samples. The Berkeley model explored substantially larger groups, including 40-person pools. These are different operational capabilities, not interchangeable versions of the same approved procedure.
FDA+1
There is another important assumption: the Berkeley model uses tests with perfect sensitivity and specificity.
Real PCR pooling involves dilution. The FDA acknowledged this concern when authorizing Quest's method, although Quest's own validation successfully identified all positive samples in its validation pools.
Subsequent independent research at the University of Washington found that a different four-sample pooling assay detected 30 of 32 positive specimens. The two missed specimens had particularly low viral loads. This does not establish that Quest's assay failed, but it demonstrates why perfect sensitivity cannot simply be assumed across testing systems.
FDA+2
The video also claims that daily testing would require only twice the resources of monthly testing.
That conclusion needs qualification. The Berkeley paper's basic model, without the additional benefit of interrupting transmission within groups, estimates that daily testing—approximately 30 rounds per month—requires around 5.5 times as many tests as monthly testing.
The more favourable results arise when the researchers introduce modeled infection transmission within groups and assume that frequent testing removes infected individuals before they spread the virus. Their numerical examples demonstrate that effect under particular transmission assumptions; they do not establish a universal two-times cost ratio for daily testing.
faculty.haas.berkeley.edu+1
The distinction is straightforward: Berkeley proposed a promising mathematical framework. It did not demonstrate that the United States could immediately operate an inexpensive, machine-learning-directed daily PCR testing programme.
The wastewater advantage was largely a reporting advantage
Wendover presents wastewater testing as a way to identify COVID-19 transmission approximately one week before conventional clinical testing.
This claim had a genuine research basis.
A May 2020 preprint by Yale researchers, led by Jordan Peccia, reported that SARS-CoV-2 RNA concentrations in sewage sludge were a seven-day leading indicator ahead of compiled COVID-19 testing data.
However, the study was still a preprint and had not undergone peer review when the video appeared.
medRxiv
The subsequent peer-reviewed publication in Nature Biotechnology, published September 18, 2020, clarified something important.
The New Haven study found that wastewater measurements led:
- Positive clinical test results by specimen collection date by 0–2 days.
- Hospital admissions by 1–4 days.
- Positive test results by public reporting date by 6–8 days.
In other words, the approximately one-week advantage was primarily measured against the date clinical results were publicly reported, not the date people actually provided their specimens.
PMC
This distinction matters.
Wastewater surveillance could bypass the delays associated with laboratory processing and public reporting. That is genuinely useful, particularly when testing backlogs are severe.
But it is not equivalent to detecting an outbreak a week before conventional clinical testing begins.
The video merges two different timelines into one attractive claim. A reader deserves to know which one is being measured.
The final verdict: promising technology is not proof of a working system
Wendover's account of COVID-19 testing logistics contains valuable details. The Quest aircraft network, centralized laboratories and shortages of essential supplies help explain why simply collecting more samples could not solve America's testing problems.
But the episode's technical conclusions need more discipline.
Its 20% pooled-testing example is mathematically wrong. Its machine-learning proposal is a theoretical model whose success depends on accurate risk predictions, suitable laboratory methods and unproven operational assumptions. Its $3–$5 figure is a modeled daily programme cost, not an established PCR test price. And its wastewater advantage needs to distinguish delayed public reporting from actual clinical specimen collection.
I am not arguing that these technologies were worthless. Quite the opposite: correcting the pooling calculation makes the case for pooled testing stronger.
The problem is presenting conditional research as though the difficult questions have already been answered.
A laboratory cannot operate on an attractive probability model alone. It needs validated tests, reliable supplies, accurate risk estimates, trained people, workable logistics and money.
Wendover explains the machinery of COVID-19 testing, but it does not consistently distinguish between what the machinery could theoretically achieve and what the evidence had actually demonstrated. In a public-health emergency, that distinction is too important to get wrong.
References
1. The Logistics of Covid-19 Testing — Wendover Productions, July 28, 2020. Original video and primary source for the claims examined.
2. Pooled Sample Testing for SARS-CoV-2 — Paolo Bertolotti et al., MIT Institute for Data, Systems, and Society, April 2020. Establishes the standard probability formula for expected pooled-testing requirements.
3. FDA Issues First Emergency Authorization for Sample Pooling — US Food and Drug Administration, July 18, 2020. Confirms Quest's four-sample authorization and explains the sensitivity concern associated with dilution.
4. Group Testing in a Pandemic: The Role of Frequent Testing, Correlated Risk, and Machine Learning — Ned Augenblick, Jonathan Kolstad, Ziad Obermeyer and Ao Wang, UC Berkeley, July 17, 2020. The original research behind the machine-learning proposal, including its assumptions, simulations and cost estimate.
5. Pooling of SARS-CoV-2 Samples to Increase Molecular Testing Throughput — Garrett A. Perchetti et al., Journal of Clinical Virology, August 2, 2020. Reports the sensitivity limitations observed in an independent four-sample pooling experiment.
6. SARS-CoV-2 RNA Concentrations in Primary Municipal Sewage Sludge as a Leading Indicator of COVID-19 Outbreak Dynamics — Jordan Peccia et al., medRxiv, May 22, 2020. The preliminary research available when the video was produced; it had not yet undergone peer review.
7. Measurement of SARS-CoV-2 RNA in Wastewater Tracks Community Infection Dynamics — Jordan Peccia et al., Nature Biotechnology, September 18, 2020. The peer-reviewed study clarifying the difference between wastewater lead time over specimen collection and public reporting.