"Financial Aid" Revised in U.S. News Methodology

In my prior post, I described one change to this year's U.S. News and World Report law school rankings methodology: 2% got shifted from the Emp0 indicator to the Emp9 one. I here briefly describe another change of note: dropping the "other direct expenditures" addend to the measure of financial aid. I grant that only USN&WR ranking geeks will find this stuff of interest. Hang in there, though; future posts in my series on the 2007 rankings will offer new ammo for status-seekers, hint at scandal, and suggest reforms that might mitigate our obsession with the USN&WR law school rankings. Devoting this much attention to statistical arcana cannot, after all, be healthy.

The USN&WR methodology includes a measure of expenditures/student called "Financial Aid" ("Fin Aid," for short). Despite its name, though, financial aid makes up only one part of that indicator. In last year's law school rankings—the "2006" rankings—Fin Aid included three sub-measures: financial aid, other direct expenditures, and indirect expenditures. This year's rankings included only two: financial aid and indirect expenditures.

Why the change? Because here, as with most other indicators it tracks, the USN&WR relies on the American Bar Association's classification of quantitative data about law schools. This year the ABA stopped collecting data under the heading of "other direct expenditures."

It might help to know a bit more about the ABA's practices. Each fall, the ABA collects financial and other quantitative data from the law schools it accredits. USN&WR asks law schools filling out its questionnaire to copy over much of the same data. Thus, for instance, USN&WR question 93 asks a responding law school to copy over data from ABA Fiscal Form I.I—Financial Aid. So it goes for all of the addends to USN&WR's FinAid indicator.

In the fall of 2006 the ABA stopped asking schools to report data under the heading of "other direct expenditures." I'm not yet sure why; the ABA has not yet responded to my inquiries about its fiscal reporting practices. Regardless, and whether as a matter of policy or preference, USN&WR followed the ABA last fall in disregarding the other direct expenditures of responding law schools. (In point of fact, the USN&WR questionnaire retained a question about other direct expenditures but forced schools to answer "0" to it.)

That is not quite to say that other direct expenditures dropped entirely out of the 2007 rankings. In calculating a school's Fin Aid (as well as the other financial indicator, "Expenditures on Overhead/Student"), USN&WR takes the average of the last two year's figures. In the most recent law school rankings, then, each school's Fin Aid indicator included both (financial aid + other direct expenditures + indirect expenditures)/student from the fall 2004 survey and (financial aid + indirect expenditures)/student from the fall 2005 survey. That Fin Aid measure, once rescaled into a z-score, counted for 1.5% of each law school's overall score in the 2007 USN&WR rankings.

What impact did dropping "other direct expenditures" from Fin Aid have on the law school rankings? In brief, it looks as if it helped those relatively few schools that formerly had reported "other direct expenditures" to the ABA and USN&WR. Why? Because, judging from the ABA data, it looks as if those schools moved the sums they had formerly reported under Fin Aid's "other direct expenditures" to one of the subcategories of data comprising the Overhead indicator. So what? Whereas Fin Aid counts for only 1.5% of each school's score in the USN&WR rankings, Overhead counts for a whopping 9.75%.

What schools benefited most from the change? For various complicated and sticky reasons, I am hard-pressed to say. "Complicated," because I have no direct proof about how whether law schools reclassified other direct expenditures to inflate their Overhead indicators. "Sticky," because I know only what fiscal numbers law schools reported to the ABA—not what they reported to USN&WR. A school might have told the ABA one thing and USN&WR another. More on that latter problem, though, anon.

Another post about the 2007 USN&WR law school rankings: Change to U.S. News Law School Rankings Methodology.