Friday, March 24, 2006

Bayesian Beef

An American meatpacking company, Creekstone Farms Premium Beef, “wants to test all of its slaughter cattle for mad cow disease,” according to USA Today (link via Jacob Sullum). But the USDA is resisting; they say testing should be handled by the government.

Sullum’s right that if a private company wants to test its cattle to provide an extra measure of security to its customers, it should be able to do so. But I wonder if Creekstone Farms might be shooting itself in the foot by failing to consider the implications of Bayes’ Rule for disease testing. If the prevalence of the disease in the population is low (and the article indicates that it’s no more than 3 in 660,000, or less than one one-thousandth of a percent), it’s highly likely that false positives will outweigh true positives. Or in other words, Creekstone’s testing plan stands a good chance of making the company’s beef supply look tainted even though it’s not.

News reports on the test for BSE (bovine spongiform encephalopathy, a fun-to-pronounce disease if there ever was one) tout its high accuracy. But there are two forms of accuracy: sensitivity, the ability to correctly identify infected organisms as having the disease; and specificity, the ability to correctly identify uninfected organisms as not having the disease. The BSE test’s advertised accuracy is of the first variety, and given the trade-off between the two, there could be plenty of false positives. (Here’s a short discussion of sensitivity and specificity in the context of BSE tests.)

Let’s suppose the BSE test has perfect sensitivity (no false negatives). And let’s assume its specificity is very high: only 1 in 100,000 uninfected cattle is falsely identified as positive. Even so, the incredibly low prevalence of BSE means that false positives will outnumber true positives by more than two to one. Application of Bayes’ Rule (with the given assumptions) shows that a cow with a positive test result has only a 31% chance of actually having BSE.

If Creekstone goes through with the plan, whose purpose is clearly to make consumers feel better about their product, hopefully they’ll have the sense to retest (with a different test variety) any cattle that get positive results before releasing the information to the public. Otherwise, their publicity plan could backfire.

1 comment:

Glen Whitman said...

Steven -- good point. I suppose it's a question of their policy for revealing information. They could just announce, "We test all our slaughter cattle and destroy any that test positive," and then say nothing more about whether any cattle have ever tested positive. However, I wouldn't be surprised if there were USDA disclosure requirements, so that any positive results had to be reported.