Thursday, January 04, 2007

More Than 50 Percent Can Be Above Average

In this article on why policymakers are irrationally biased toward hawkish foreign policy, Daniel Kahneman and Jonathan Renshon offer a familiar example of alleged cognitive bias:
About 80 percent of us believe that our driving skills are better than average.
I used to think this was a clear example of cognitive bias. Now I’m not so sure. The idea is that only 50 percent of us can really be better-than-average drivers, so at least 30 percent of people are wrong. The buried assumption here is that driving skill is symmetrically distributed, so that the mean and median are the same. That’s not necessarily true; the distribution might be skewed by the existence of small numbers of very bad drivers. To take a very simple case, suppose that 80% of drivers have no accidents at all, while the other 20% have 5 accidents per year. Then the average (mean) is 1 accident per year, and fully 80% of drivers are better than average! Obviously this example is unrealistic because it includes only two driver types, but it’s illustrative of what I think might be going on.

I couldn’t find any figures online stating the distribution of drivers by accident frequency, so I really don’t know if the distribution is sufficiently skewed to justify 80% of us thinking we’re better than average drivers. (If anyone knows of such statistics, please point me to them.) But it’s certainly not unknown for distributions to be that skewed. To take one example I’m fond of presenting to my students, the average (mean) time to conception for women trying to get pregnant is about 7 months. But 50% of such women will be pregnant within 4 months, and 75% will be pregnant within 6 months. That means at least 75% of women are doing “better than average” in the conception race. The result is driven, of course, by the fact that some couples have fertility problems that delay conception for many months or even years. These couples are analogous to the bad drivers who might be skewing overall accident rates.

6 comments:

RiskyBiznatch said...

This is a VERY good point. I am a corporate risk manager so I deal a lot with statistics and risk perceptions. What I have found is that most people have intuition about how to understand a mean, some can grasp issues related to dispersion or second moment (i.e. standard deviation or CV) but few people intuitively grasp issues related to the third or fourth moment (skewness or kurtosis). I recently read a good discussion of this phenomenon and I think it was in Fooled By Randomness by Taleb. It might have been in one of John Allen Paulos' books.

When I run across this in business I always called it the Lake Wobegon Paradox and I thought I was being cleaver. Then I happened to type Lake Wobegon into Wikipedia and realized that I was not the least bit novel. Lake Wobegon Effect

I also found the list of Cognitive Biases interesting.

Gil said...

On the other hand, if someone as smart as Glen doesn't know enough about the distribution of bad drivers to know whether the worst drivers skew the distribution much, I think it's unlikely that most people are correctly using such knowledge.

Nice try, though.

Glen Whitman said...

Gil -- They wouldn't have to know the distribution in order to reach the right conclusion. They would just need a sense of what the mean is, and whether they are above or below it. In my simplified example, they would only need to know that the average number of accidents is about 1 per year. (Actually, given the simplicity of my example, they would only need to know the mean is greater than zero.)

But I don't mean to claim that people are *not* biased, just that the given fact (80% think they're better than average) doesn't show it. We would have to know how survey respondents interpreted the word "average." They might have taken "better than average" to mean better than half the drivers out there -- or in other words, better than the median. And in that case, we could indeed conclude that a lot of people are deluding themselves.

Tim Lee said...

Glen,

This is a good point, but I wonder if it makes any sense in this case. Driving skill is an ordinal, not a cardinal, value. It makes sense to say that person X is a better driver than person Y, but I don't know how much sense it makes to say that person X is twice as good as person Y.

Which means that when we say someone is a better than average driver, we almost certainly mean that he's better than the median driver, not that he's better than the mean value of some arbitrarily constructed driving-skill index. And obviously, it's not possible for more than 50% to be above the median in any distribution.

Glen Whitman said...

Is driving skill really ordinal? I don't know about that. Seems to me that it can be measured cardinally in various ways, such as accidents per year. Of course, it may be difficult to measure skill directly, but that's a different matter. Maybe it seems ordinal to you because it actually consists of many different skills that aren't necessarily comparable to each other -- e.g., how much reduction in your accident rate will compensate for your inability to parallel park?

Gil said...

I think there's a better explanation.

I think that when different people claim that they're better than the average driver, they're not necessarily saying the same thing.

They have different definitions of what it means to be a better driver. Good driving is a bundle of skills and practices and many people weight each of the factors differently.

So, most people can truthfully say that they are a better than average driver because when they say it, they're using their own formula to measure it, and they probably focus on their own more heavily-weighted factors more than the average driver; so they are better at them.

This reminds me of Will Wilkinson's arguments about happiness from status races being positive-sum, because different people have different dimensions where they seek status.

We can all be happy with our relative position if we have different areas of life where status matters to us.