Thursday, February 13, 2003

What I Don’t Get About the Lott Controversy

Here’s what’s been nagging me about the John Lott controversy (if you need the summary, click here). There’s no doubt that Lott’s reputation, academic and otherwise, will suffer from recent events. At a minimum, we can say (a) that he shouldn’t have relied on results from a survey that he knew he couldn’t substantiate, and (b) that the sample size for the survey, if it was indeed conducted, was too small to justify the result he stated with any degree of confidence. In addition, it’s pretty creepy to pretend to be a fictional former student who defends your integrity while writing glowing reviews of your book.

But here’s what bothers me. Some of Lott’s critics are using these recent events to discredit or dismiss Lott’s famous result – publicized in _More Guns, Less Crime_ – showing that concealed-carry laws reduce the crime rate in states that enact them. Mark Kleiman, for instance, says we should regard Lott’s celebrated conclusion as “probably false” because Lott can no longer be trusted. Keep in mind that the concealed-carry study was quite distinct from the alleged survey (which concerned with the frequency of defensive gun uses in which shots were never fired). Still, if we had to blindly trust Lott in order to accept his concealed-carry study, as would be true if the results could not be independently verified, then the critics would have a point.

However, it is my understanding (and I’m sure someone will correct me if I’m wrong) that Lott provided all of his data and primary sources to anyone who asked for them. Analysts who used these data with Lott’s econometric specifications reached the same conclusions. Analysts who used these data with different econometric specifications (for example, this paper by Rubin and Dezhbakhsh, later published in AER, which used the 1992 portion of Lott’s data) sometimes reached different conclusions. Now, there’s a legitimate debate to be had on the appropriate specifications and statistical methods, but it’s a debate that was already going on prior to the survey controversy, and it’s not the kind of debate in which academic reputations are on the line. The point is that if Lott provided his data and sources to others, there’s no issue of fabrication here. If people want to refute Lott’s claims about concealed-carry laws, they’ll have to continue attacking his work on the merits. The survey fracas is not, or at least should not be, a silver bullet.

No comments: