The August 7, 2008 issue of Nature has an article by a biostatistician and an accompanying editorial criticizing task the logical and scientific assumptions behind anti-doping tests commonly used in sports.
Donald Berry’s article, The Science of Doping, is interesting because it looks at the methods used to take away the Tour de France championship of Floyd Landis, who continues to maintain that he did not use performance enhancing drugs. According to Berry,
Landis seemed to have an unusual test result. Because he was among the leaders he provided 8 pairs of urine samples (of the total of approximately 126 sample-pairs in the 2006 Tour de France). So there were 8 opportunities for a true positive — and 8 opportunities for a false positive. If he never doped and assuming a specificity of 95%, the probability of all 8 samples being labelled ‘negative’ is the eighth power of 0.95, or 0.66. Therefore, Landis’s false-positive rate for the race as a whole would be about 34%. Even a very high specificity of 99% would mean a false-positive rate of about 8%. The single-test specificity would have to be increased to much greater than 99% to have an acceptable false-positive rate. But we don’t know the single-test specificity because the appropriate studies have not been performed or published.
More important than the number of samples from one individual is the total number of samples tested. With 126 samples, assuming 99% specificity, the false-positive rate is 72%. So, an apparently unusual test result may not be unusual at all when viewed from the perspective of multiple tests. This is well understood by statisticians, who routinely adjust for multiple testing. I believe that test results much more unusual than the 99th percentile among non-dopers should be required before they can be labelled ‘positive’.
Berry also claims that the proper vetting of the tests used on cyclists have never been done (emphasis added),
The method used to establish the criterion for discriminating one group from another has not been published, and tests have not been performed to establish sensitivity and specificity. Without further validation in independent experiments, testing is subject to extreme biases. The LNDD lab disagrees with my interpretation. But if conventional doping testing were to be submitted to a regulatory agency such as the US Food and Drug Administration5 to qualify as a diagnostic test for a disease, it would be rejected.
The Nature editorial really drives this point home. It notes that tests for banned substances are developed essentially by having a small number of control subjects ingest the substance and then using analyses of blood and urine from those individuals to set acceptable limits of substances and their byproducts.
But not only are these small groups, but the threshold levels are never made public,
Nature believes that accepting ‘legal limits’ of specific metabolites without such rigorous verification goes against the foundational standards of modern science, and results in an arbitrary test for which the rate of false positives and false negatives can never be known. By leaving these rates unknown, and by not publishing and opening to broader scientific scrutiny the methods by which testing labs engage in study, it is Nature‘s view that the anti-doping authorities have fostered a sporting culture of suspicion, secrecy and fear.
Detecting cheats is meant to promote fairness, but drug testing should not be exempt from the scientific principles and standards that apply to other biomedical sciences, such as disease diagnostics. The alternative could see the innocent being punished while the guilty escape on the grounds of reasonable doubt.
It’s the sports world’s version of security through obscurity.
On the other hand, the idea that using performance enhancing drugs is “cheating” is itself a major part of the problem. The Economist had a reasonable article about performance enhancing drugs a couple weeks ago which made the case for bannings substances based strictly on safety not supposed performance enhancing properties. If a drug, when used properly, is completely safe there is no more reason to ban it than there would be to ban athletes from engaging in weight training, special diets, training at different altitudes, or any number of other performance enhancing techniques that athletes routinely engage in.
End the witch hunts — allow the drugs.