Each fall, the ABA sends its Annual Questionnaire to each of the law schools it accredits. The eight-part questionnaire covers a wide variety of topics and asks for detailed answers. The questionnaire closes with a statement that the Dean of each responding school must sign, and in which he or she attests, "I hereby certify that the information provided within to be a complete and accurate representation of this law school." The ABA's accreditation power gives it plenty of clout to make sure that law schools take the questionnaire seriously.
After having collected the Annual Questionnaires from all of the law schools that it accredits, the ABA compiles and re-releases the data to those law schools in the form of statistical "take-offs" (as the documents are called). So, for instance, each law school receives a report showing what it and the other law schools reported on the Fiscal Form—data such as what each law school paid out in instructor salaries, summer salaries, administrative salaries, fringe benefits, at so forth.
These take-offs go to the Dean of each law school, and carry notices describing the data as confidential. Why? It is not clear. Generally, a firm protects trade secrets from competitors. By design, in contrast, the ABA's take-offs effectively guarantee that each law school learns detailed information about its competitors. Who remains that must not learn, for instance, the number of volumes and microfilm equivalents held by each law school's library? Not USN&WR, evidently; law schools volunteer to it that and a great deal more data that the ABA labels, "confidential." Suffice it to say, and speaking as someone who researches and writes about intellectual property, the ABA does not have a very strong argument that its statistical take-offs qualify as trade secrets. I suspect, in fact, that the ABA has nothing more than making money in mind (see below).
Nonetheless, I've hewed to the ABA's request. As discussed in a prior post, my model relies on the ABA take-offs to fill in data that USN&WR fails to disclose. Though it has gotten in the way of my attempts to describe the USN&WR rankings, I've taken pains to not disclose the ABA data. In a better world—one that reforms might make reality—the ABA would give all of us access to the same data that USN&WR uses in its rankings.
The ABA might promote another, more immediate interest by making public the data it collects from law schools: accuracy. While poring over the ABA data to prepare this year's model of the USN&WR rankings, I ran across some puzzling features in ABA form F15-0506 (compiling data that member law schools had reported in the Fiscal section of the ABA's 2004 Annual Questionnaire, distributed and collected in the fall of 2005). A phone call to David Rosenlieb, the ABA's Data Specialist, confirmed what I'd suspected: F15-0506 contained errors. After confirming that my Dean had authorized me to access the data, he very helpfully sent me the actual figures. I understand that he will soon send each of the ABA's member schools a corrected version of F15-0506.
That the ABA falls short of perfection should shock nobody. It has a lot on its hands and we all make mistakes. Rather than condemning the ABA's data management processes, therefore, we ought to help improve them. Imagine if the ABA were to adopt an open source approach to the data it collects from law schools, making it publicly available on the ABA website. We all might then cast helpful eyes on the data, double-checking it for accuracy and completeness. The ABA would thereby better serve both its member law schools and its ultimate constituency, the public. Granted, making the Annual Questionnaire data free available would make it hard for the ABA to keep demanding that each law school pay $1430 to receive the take-offs. But it would also save the ABA money by precluding the need to mail voluminous reports (and corrected versions thereof!) to nearly 200 law schools. Lastly, but by no means least importantly, by adopting an open source approach to law school data, the ABA would make it easier for all of us to double-check the USN&WR rankings.
Earlier posts about the 2007 USN&WR law school rankings:
- Change to U.S. News Law School Rankings Methodology
- "Financial Aid" Revised in U.S. News Methodology
- How USN&WR Counts Faculty for Rankings
- Whence Come the LSATs and GPAs Used in the Rankings?
- Gains and Losses Due to USN&WR's Use of Reported Median LSATs and GPAs
- How to Model USN&WR's Law School Rankings
- Why to Model USN&WR's Law School Rankings
No comments:
Post a Comment