Holy Deceptive Numbers, Statman!

Mark Kleiman recently penned an excellent article about how “counting the winners and ignoring the losers” allowed researchers to fudge the effectiveness of the InnerChange Freedom Initiative in rehabilitating criminals. This kind of statistical bias is nothing new, of course. But because of Mark’s article, the topic happened to be on the tip of my cerebral cortex when I opened a piece of junk mail from 21st Century Insurance. The letter states that “On average, new customers report saving $300.00 each year.” Sounds impressive, until you think about it for more than a couple of seconds. The problem is that only the people who called 21st and discovered they could actually save money switched over and became new customers. Those who called 21st and discovered they would lose money by doing so are thus excluded from the sample.

At least the company is not being duplicitous – they do say “new customers.” If they had said, “On average, you can save $300 by switching to 21st Century Insurance,” that would have been truly misleading. Still, I wonder how many people who receive the ad get the false impression that 21st Century must be offering systematically lower rates than its competitors.