Friday, March 24, 2006

Bayesian Beef

An American meatpacking company, Creekstone Farms Premium Beef, “wants to test all of its slaughter cattle for mad cow disease,” according to USA Today (link via Jacob Sullum). But the USDA is resisting; they say testing should be handled by the government.

Sullum’s right that if a private company wants to test its cattle to provide an extra measure of security to its customers, it should be able to do so. But I wonder if Creekstone Farms might be shooting itself in the foot by failing to consider the implications of Bayes’ Rule for disease testing. If the prevalence of the disease in the population is low (and the article indicates that it’s no more than 3 in 660,000, or less than one one-thousandth of a percent), it’s highly likely that false positives will outweigh true positives. Or in other words, Creekstone’s testing plan stands a good chance of making the company’s beef supply look tainted even though it’s not.

News reports on the test for BSE (bovine spongiform encephalopathy, a fun-to-pronounce disease if there ever was one) tout its high accuracy. But there are two forms of accuracy: sensitivity, the ability to correctly identify infected organisms as having the disease; and specificity, the ability to correctly identify uninfected organisms as not having the disease. The BSE test’s advertised accuracy is of the first variety, and given the trade-off between the two, there could be plenty of false positives. (Here’s a short discussion of sensitivity and specificity in the context of BSE tests.)

Let’s suppose the BSE test has perfect sensitivity (no false negatives). And let’s assume its specificity is very high: only 1 in 100,000 uninfected cattle is falsely identified as positive. Even so, the incredibly low prevalence of BSE means that false positives will outnumber true positives by more than two to one. Application of Bayes’ Rule (with the given assumptions) shows that a cow with a positive test result has only a 31% chance of actually having BSE.

If Creekstone goes through with the plan, whose purpose is clearly to make consumers feel better about their product, hopefully they’ll have the sense to retest (with a different test variety) any cattle that get positive results before releasing the information to the public. Otherwise, their publicity plan could backfire.


Steven Bass said...

IT seems to me that their plan isn't to prove that their beef is safe on the way in, but instead to prove it's safety on the way out. Thus, they can say that all of the cows that they sell have been tested and are free from mad cow disease. For this purpose, it seems like slaughtering one false positive out of every 200,000 tested shouldn't be that high of a price to pay.

Ben said...

For those interested in Bayesian reasoning and its consequences to epistemology logic science etc. read this tutorial, this one and of course the seminal book by E.T. Jaynes that changed everything or this possibly illegal copy. Truly nerdalicious!

Glen Whitman said...

Steven -- good point. I suppose it's a question of their policy for revealing information. They could just announce, "We test all our slaughter cattle and destroy any that test positive," and then say nothing more about whether any cattle have ever tested positive. However, I wouldn't be surprised if there were USDA disclosure requirements, so that any positive results had to be reported.