In brief, I modeled the USN&WR rankings by trying to do the same things that USN&WR did to the same data that it used. That verb—"trying"—merits emphasis. With regard both to USN&WR's methodology and data, I cannot be absolutely sure that I've copied the original. With regard to the data, in particular, I've at best only come pretty close. Allow me to explain.
So far as methodology goes, I of course made reference to the explanation that USN&WR publishes on its website. That proved insufficiently detailed for my purposes, however. I gleaned further details from various other sources, most notably from Robert J. Morse and Samuel Flanigan, the Director and Deputy Director of Data Research, respectfully, at U.S. News. Those busy gentlemen proved very patient and helpful. Though there may remain some minor variations between the methodologies we use, I am reasonably confident that USN&WR does not calculate law school scores in a materially different way from my model.
Copying the data USN&WR uses proved more difficult. I sometimes had the same data that USN&WR did use, sometimes had the data that USN&WR should have used, and sometimes had to guess what data it might have used. Insofar as possible, I used the same data that USN&WR publishes along with its ranking of law schools. In many categories, however, USN&WR does not disclose what law schools told it. In those cases, I made reference to the ABA data that law schools were supposed to have copied onto their USN&WR questionnaires. Only with regard to one datum—the cost of living index that USN&WR applies to each school's reported total expenditures on instruction and administration, one of the components of the Overhead/Student indicator—did I have to generate my own numbers.
The following table offers some details about the methodology used by, and the sources of the data used in, the USN&WR rankings and my model thereof:
| Data | Weight in Ranking | USN&WR's Source | Published by USN&WR? | Model's Source |
| Peer Rep | 25.00% | USN&WR survey | yes | USN&WR rankings |
| Bar Rep | 15.00% | USN&WR survey | yes | USN&WR rankings |
| Emp at 9 Mos | 14.00% | ABA data schools reported to USN&WR | yes | USN&WR rankings |
| LSAT Median | 12.50% | ABA data schools reported to USN&WR | no | ABA data |
| GPA Median | 10.00% | ABA data schools reported to USN&WR | no | ABA data |
| Overhead Exp/Stu | 9.75% | ABA data schools reported to USN&WR | no | ABA data |
| CoL Index for Overhead Exp/Stu | N.A. | Bought by USN&WR from 3rd party | no | estimated |
| Emp at 0 Mos | 4.00% | Reported by schools to USN&WR | yes | USN&WR rankings |
| Stu/Fac Ratio | 3.00% | ABA data schools reported to USN&WR | yes | USN&WR rankings |
| Acceptance | 2.50% | ABA data schools reported to USN&WR | yes | USN&WR rankings |
| Bar Pass Rate | 2.00% | ABA data schools reported to USN&WR | yes | USN&WR rankings |
| Fin Aid Exp/Stu | 1.50% | ABA data schools reported to USN&WR | no | ABA data |
| Library Resources | .75% | ABA data schools reported to USN&WR | no | ABA data |
Although figuring out all the details of the methodology used by USN&WR took some time and effort, it did not prove unnecessarily difficult. I wish I could say the same about figuring out the data used by USN&WR. There, the hurdles proved not only higher, but inexplicably so.
As the above table indicates, my search for the data used in the rankings sometimes called on me to do no more than refer to the figures published by USN&WR. In many cases, however—and for no apparently sound reason—USN&WR declined to disclose the data it used. Why does USN&WR publish schools' 25th and 75 percentile LSAT and GPA scores but not publish the medians it actually uses in calculating its rankings? Interested parties could perhaps use the median LSATs and GPAs published by the Law School Admissions Council, but that data is hardly easy to download. More crucially, why doesn't USN&WR publish the data about expenditures and financial aid that schools report to it and the ABA? That data, worth over 10% of each school's score, appears in no public source. As I'll discuss in more detail later, USN&WR could greatly improve its rankings simply by letting all of us know more about the data that it uses. In other words, every entry in the above table's "Published by USN&WR?" should read, "yes."
Earlier posts about the 2007 USN&WR law school rankings:
No comments:
Post a Comment