SSRN v. USN&WR v. Truth

Chapman University School of Law did very well in the most recent Social Sciences Research Network (SSRN) rankings. Chapman topped all other law schools in the number of new papers posted in the last 12 months, and ranked 71 out of the 353 schools in the total number of downloads over that same period. So much, so good. Chapman apparently ruffled some feathers, however, by publicly acknowledging its success.

Joseph A. Hodnicki, Associate Director for Library Operations at the University of Cincinnati Law Library, characterized Chapman's announcement as "an amazing display of hubris . . ." Prof. Dan Markel, of FSU Law School, agreed, praising Hodnicki's "super post puncturing Chapman School of Law's inflated claim to outperform Yale, Harvard, etc. . . ." And, yet, Chapman plainly did outperform its counterparts in terms of what the SSRN rankings measure. What's the problem, then?

The SSRN's "# of New Papers" measure does not discriminate between old and new publications. Hodnicki complains that Chapman aced that measure because some of its faculty members recently posted on SSRN papers they published years ago. (Hodnicki seems more comfortable with Chapman's top-100 debut in the SSRN's "total downloads" measure.) I agree with Hodnicki insofar as he thinks the SSRN should do a better job of gauging law schools' scholarly performance. Indeed, I'll go farther: I'd bet that the SSRN defines "# of New Papers" as it does in order to encourage the posting of old papers, which helps to increase the size and value of the SSRN database.

The SSRN's "# of New Papers" measure thus merits criticism—of SSRN. Chapman perhaps committed a gaucherie, granted, in trumpeting its success. And perhaps, as a member of Chapman's faculty, I lack a sufficiently objective view of the matter. It seems to me, though, that Chapman caught flack for truthfully reporting that it did a good job at sharing its scholars' work with the world. Why criticize Chapman for that?

Hodnicki tries to discredit Chapman's achievement by citing the school's low reputation score in the U.S. News & World Report rankings. As I've explained elsewhere, though, Chapman's (presently) low reputation score in the USN&WR rankings says more about law school ranking games than it does the school's actual qualities. The most USN&WR recent rankings credited Chapman with:

• a median LSAT (156) above that of USN&WR's third tier (155);
  
• an average GPA (3.31) closer to the third tier's (3.39) than the fourth's (3.22);
  
• an employment at nine months figure equal to the third tier's (91%);
  
• an acceptance rate (30.2%) better than the third tier's median (35.0%); and
  
• a Bar pass/jurisdiction figure (109.3%) better than the second tier median (103.9%).

I'm happy to join Hodnicki and anyone else in casting a skeptical eye on the SSRN rankings. Let's aim our fire at the SSRN, though, rather than on the schools it ranks. And let's not assume that USN&WR's reputation scores offer a more accurate measure of law school quality.

Update: See this more recent post for an analysis of the validity of SSRN's "# of New Papers" measure.

[Crossposted to MoneyLaw.]