Huzzah for U.S. News and World Report! The most recent edition of its law school rankings includes the median LSAT and GPA of each school’s entering class. Finally. I have long argued that USN&WR should publish all of the data that it uses in its rankings. How else can the rest of us (read: rankings geeks) understand how—and, indeed, whether—the rankings work? Though USN&WR remains short of that ideal, disclosing median LSATs and GPAs represents a major step towards making the rankings more transparent and, thus, trustworthy.
USN&WR started the trend towards transparency last year, when it began publishing the “volume and volume equivalents” measures that it uses in its law school rankings. That input counts for only .75% of a school’s score, however. Median LSATs and GPAs together count for 22.5% of a school’s score, in contrast, making their disclosure by USN&WR all the more helpful.
There remain only two categories of data that USN&WR still uses in its law school rankings but does not disclose: overhead expenditures/student (worth 9.75% of a school’s score in the rankings) and financial aid expenditures/student (worth 1.5%). It isn’t evident why USN&WR declines to publish those inputs, too, though perhaps the financial nature of the data raises special concerns. If USN&WR cannot bring itself to publish overhead expenditures/student and financial aid expenditures/student, however, it should abandon those measures. They serve as poor proxies for the quality of a school’s legal education and if we cannot double-check the figures we cannot trust their accuracy.
[Crossposted at Agoraphilia and MoneyLaw.]
Tuesday, May 22, 2012
U.S. News & World Report Improves Transparency of Law School Rankings
Tuesday, January 10, 2012
Are STEM Degrees Already Subsidized More?
Alex Tabarrok argues in his ebook, Launching the Innovation Renaissance, that graduates with STEM (science, technology, engineering, math) degrees are more likely to create innovations that benefit the rest of society – and therefore they are relatively more deserving of educational subsidies than students in other disciplines. Here’s how Alex puts it:
Most importantly, graduates in the arts, psychology and journalism are less likely to create the kinds of innovations that drive economic growth. Economic growth is not a magic totem to which all else must bow, but it is one of the main reasons we subsidize higher education.I think Alex is right; if we’re going to subsidize education, we should subsidize education that generates external benefits for society at large.
The potential wage gains for college graduates go to the graduates — that’s reason enough for students to pursue a college education. We add subsidies to the mix, however, because we believe that education has positive spillover benefits that flow to society. One of the biggest of these benefits is the increase in innovation that highly educated workers theoretically bring to the economy.
As a result, an argument can be made for subsidizing students in fields with potentially large spillovers, such as microbiology, chemical engineering, nuclear physics and computer science. There is little justification for subsidizing sociology, dance and English majors.
But I’m wondering if, in fact, we might already subsidize STEM degrees more than other degrees. Consider the following three factors that make STEM courses more costly to teach:
1. STEM professors are typically paid higher salaries. See, for example, this report from the Chronicle of Higher Education. The last table shows salaries by discipline, as a percentage of the average salary of English professors. Across all disciplines, the average salary is 13.4% higher than an English professor’s. But Engineering professors earn 25.2% more, Computer & Information Sciences 28.4% more. Mathematics is below average at 7.2%, but overall, STEM professors appear to get paid a good bit more than the average. Meanwhile, Fine Arts, Education, Communications, Philosophy, and Psychology are all below the average. (This makes sense, because STEM professors probably have better outside job opportunities and thus a higher opportunity cost.)
2. It’s easier to teach non-STEM courses in large lecture halls, whereas STEM courses often require smaller class sizes to be taught effectively. (I don’t know this with certainty, but I’ve been told as much by university administrators.)
3. When STEM courses are taught in large lecture halls, they require a larger number of teaching assistants to give the students the attention they need. (Again, I don’t know this with certainty, but it’s what I’ve been told.)
Putting 1-3 together, it seems pretty likely that STEM education is more costly to produce. And yet colleges and universities typically charge all students the same tuition regardless of major. True, STEM students may be charged nominal lab fees, but I doubt such fees make a large difference in percentage terms.
So when we consider how much students are charged relative to cost, it looks like STEM students might be getting the larger subsidy. Of course, I don’t know how high the optimal subsidy would be, so it’s possible the current subsidy isn’t large enough.
It’s also worth noting that professors in Law and Business Administration earn the highest pay differentials of all (59.5% and 50.9% above the average English professor), which would imply that these fields are getting among the highest relative subsidies. Econ professors also earn a high differential of 41.2%. Again, this is presumably driven by outside job opportunities. Unless we believe Law, Business Admin, and Econ generate strong positive externalities, maybe we should be charging students more to major in those fields.
Is there some factor I’m missing that would diminish the relative subsidy to STEM (or at least STE) degrees?
Wednesday, April 20, 2011
Martin Interviews Bell for Kosmos Online
Chris Martin recently interviewed me for Kosmos Online, a project sponsored by the friendly folks at the Institute for Humane Studies. Our podcast covered the Socratic method, measuring class participation, and other teaching techniques used in law school and other settings. You can listen here or read a rough transcript here.
(Note that this semester, at the suggestion of my students, I decided to try an approach to grading classroom participation different from the one I described in my interview with Chris. So far, so good. The experiment has yet to finish, though, so I’m not sure what technique I’ll use next year.)
Wednesday, March 09, 2011
Unions, Education, and Simpson's Paradox
I know, I know, I never blog anymore. I hope to change that soon. Maybe.
But in the meantime, I had to draw attention to this post by IowaHawk. As Tyler Cowen observes, the post is marred by some unnecessary partisan vitriol. But the central point is nevertheless fascinating. Although unionized Wisconsin outperforms un-unionized Texas in educational performance when you look at overall figures, it turns out that Texas outperforms Wisconsin for every major ethnic group. Now, this might sound impossible, until you realize that Texas has a substantially higher percentage of blacks and Hispanics, who tend to get lower test scores regardless of the state.
I'm not trying to make political hay out of this; I'll leave that to others. No, the reason I find this so interesting is that, as a math nerd, I'm excited at any sighting of Simpson's Paradox in the wild. The same phenomenon can happen when, for instance, a hospital with better doctors gets a disproportionate number of the sickest patients. See here for a previous, albeit artificially constructed, appearance of Simpson's Paradox on this blog.
Wednesday, October 27, 2010
Three Kinds of Diversity
Diversity comes in many flavors. I here compare three types—diversity of skin color and sex, cultural diversity, and ideological diversity—and offer some observations about the distinctive costs and benefits of each. I conclude that, holding all else equal, a group of people having diverse colors and sexes will enjoy modest institutional gains at low cost, while a group touting ideological diversity runs the risk of high transaction costs but wins a shot at great intellectual gains. Groups with high cultural diversity fall in between those two extremes.
Diversity of skin color and sex appears on the face of a group, thus offering ready proof that its selection, such as through hiring and promotion, was not tainted with invidious discrimination. Holding all else equal—assuming, specifically, that the racially and sexually diverse group does not possess above-average cultural and ideological diversity—the costs of intra-group transactions remain low. Thus, for instance, might a facially diverse group of culturally and ideologically similar people get along very smoothly. Think, here, of an elite law school where every professor has absorbed Ivy League norms and all lean moderately left. They might bicker, of course; law professors specialize in that. But such a culturally and ideologically uniform group is not likely to host nasty public fights about ballot initiatives or the like.
Are there downsides to pursuing diversity of skin color and sex in hiring and promotion? Not if you can find enough well-qualified candidates, and not if you avoid discriminating against candidates for blameless having an uninteresting color or sex. Happily, it is not too hard to satisfy both conditions, these days.
Cultural diversity proves harder to document, and runs some risk of increasing intra-group transaction costs. Someone brought up solely within the confines of respectable East Coast institutions will have to work a bit to understand a peer raised Mormon, in Utah's backcountry. So, too, might differences of sexual orientation (which like cultural differences generally do not appear on a person's face) sometimes lead to innocent misunderstandings. Holding equal for other sorts of diversity, however, cultural differences offer many charms and few serious costs. Most of us, and especially those of us in academia, enjoy meeting friendly people with exotic backgrounds. When we share ideologies, moreover, meeting fellow travelers who differ from us suggests that our most heartfelt values transcend race, sex, and culture—a comforting, if somewhat smug, idea.
Ideological diversity, standing alone, proves at least as hard to document as cultural diversity—it does not appear on a person's face nor even, typically, in a person's dress or hairstyle—and much more likely to raise intra-group transaction costs. Religious differences prove largely intractable, though in polite society we tend to keep them private. Political differences, at least in American institutions, threaten to burst out into loud and public disagreements, however. Such frank exchanges can help each side to hone its arguments, of course, and thus offers the prospect of considerable gains both to the disputants and the group that harbors them both. But if local norms do not temper the tone and proper boundaries of ideological debate, transactions costs can easily soar, making it hard for a group to manage even run-of-the-mill functions efficiently.
In sum: diversity of skin color and sex offers few costs and modest benefits; cultural diversity creates slightly higher transaction costs but compensates with intriguing charms; and ideological diversity presents a high risk/high return strategy for institutions devoted to generating new and useful ideas.
[Crossposted at Agoraphilia, and MoneyLaw.]
Tuesday, April 27, 2010
The Hand Rule
Judge Learned Hand famously opined that if the burdens of preventing an accident outweigh its cost multiplied by its probability, it does not constitute carelessness to avoid those burdens. Doesn't that little gem make you want to break out in song? I've got just the thing: The Hand Rule, a little ditty I recently composed and played for some students at Chapman Law School.
Though I've yet to record The Hand Rule, I can offer you a .pdf of the lyrics and chords as well as a PowerPoint, complete with pictures of Learned Hand, to accompany the performance (both uncopyrighted). Here's a sample of a verse and the refrain:
In the case of Carrol Towing Co., Learned Hand set forth to showSilly? Yes, but it gets students to pay attention and remember what they learn. So goes the modus operandi of the Law and Fun school.
The meaning of "reasonability."
Defendant failed to leave in charge, a man to watch its unmoored barge.
And plaintiff's cargo met calamity.
"Negligence!" plaintiff complained and on appeal, Judge Hand explained,
The proper scope of liability.
Learned, learned, Learned. Learned in the law was he.
Learned Judge Hand, Learned, he judged so learnedly!
So learn what the Hand Rule teaches: "There's no liability,
If the burden of the cost exceeds the loss times the probability."
[Crossposted at Agoraphilia and MoneyLaw.]
Thursday, July 16, 2009
A Model of the 2010 USN&WR Law School Rankings
As in every year since 2005, I this year again built a model of the law school rankings published by the U.S. News & World Report ("USN&WR"). Figuring out the rankings—the "2010" rankings, as USN&WR's calls them—proved especially trying this time around. USN&WR changed several parts of its methodology this year and the ABA, which distributes statistical data on which my model depends, fell far behind its usual publication schedule. Finally, though, the model ended up generating scores gratifyingly close to those that USN&WR assigned law schools. Here's a snap-shot comparison of the results:
For details about how and why I modeled USN&WR's law school rankings, as well as for similar snap-shots, see these posts from 2005, 2006, 2007, and 2008.
Perhaps in later posts I'll offer some reflections on what this year's model of the USN&WR rankings teaches. For now, I'll just offer this happy observation: The close fit between USN&WR's scores and the model's scores suggests that law schools did not try game the rankings by telling USN&WR one thing and the ABA (the source of much of the data used in my model) another. Even a skeptic of law school rankings can find something to like in that.
[Crossposted at Agoraphilia and MoneyLaw.]
Sunday, January 18, 2009
Physical (Re)Education 101
My co-blogger and host, Glen, might object, but I'd argue that "agoraphilia" covers not just "love of markers," but also "love of the great outdoors." Surely, the meaning of "agora" reaches at least that far. Allow me, then, to update you on my adventures under the open sky; here, specifically: surfing.
I've continued the training program that I began during my sabbatical, about year ago, and have gotten fairly competent on my 6'10" hybrid fish. I've also put a lot of time into figuring out my newest board, a 7'2" Channel Island M13, and had some great rides. These days, I surf those two boards almost exclusively. My wave count has improved a lot and I'm starting to get a handle on pumping for acceleration, cut-backs, and sometimes even throwing a little spray. I've yet to get any air, though.
I'm a little worried that if I go back to longboarding I might suffer a relapse, and put my hard-won shortboarding skills at risk. The last few times I've taken out my longboard—a performance-shaped 9-footer—I've thus limited myself to fin-forward paddling and takeoffs. I'm getting OK at spinning it, but I still haven't figured out how to cross-step back to regular footing. Instead, I just finish out my (usually short) rides goofy-footed.
Today, though, I just might let myself indulge in some good old fashioned, straight-up longboarding. I'm heading to Church's with my Marine buddy (who can use his pass to park us right on the beach) and we're expecting a mackin' swell. He always longboards, and it'll be nice to hang with him outside, picking off the bigger sets. Besides, I think I've earned a gravy session; yesterday, for the first time ever, I surfed Lowers! The swell had not started showing, so it wasn't crowded, and I got some good rides on the small but well-shaped waves. It took me over ten years of preparation, but I finally can say I've shortboarded one of the world's premier breaks!
I might say more about that preparation—which has included special trail running and gym exercises of my own devising—in a later post. And as long as I'm covering physical training, I should probably throw in an update on my semi-prone bike, which I recently took out of storage, upgraded, and put through some test rides. Right now, though, I've got to saddle up for the first big winter swell of the season!
Wednesday, January 07, 2009
Anarchist Chess
My boy, Kai, joined his school's chess club this fall and has ardently plunged into learning the game. He enjoys setting up the board next to his mom's favorite reading chair and playing both sides of lively matches, complete with violent captures and Star Wars sound effects.
At only five years old, Kai has a long way to go before he masters chess. Yet I think he has already figured out something that a lot of adults don't understand. Running from his chess board to my side, the other day, Kai announced with glee, "I was playing myself and I captured my king but I just kept playing without him!"
Wednesday, May 07, 2008
Forget about Memorization
A recent article in Wired, Want to Remember Everything You'll Ever Learn? Surrender to This Algorithm, offers both a fascinating sketch of Piotr Wozniak's single-minded pursuit of memorization and all that you need to know if you want to match his incredible achievements. It boils down to this: Use Wozniak's program, SuperMemo, if you yearn to remember lots of data.
Alas for my students, however, SuperMemo probably won't help much with the study of law. As first-year students quickly discover—often to their evident chagrin—you cannot learn the law simply by memorizing it. SuperMemo might work for, say, drumming conjugations of French verbs into your head, but it won't help you figure out whether a promise to forego demanding payment of a debt constitutes illusory consideration. In that, the law resembles physics: Learning the rules couts for far less than figuring out how to apply them to particular facts.
I don't know of any sure-fire way to master that sort of learning. We law profs muddle along with a mixture of classroom demonstrations, abstract theorizing, rough rules-of-thumb, and hands-on practice. I'd love to offer my students something like a SuperMemo program for mastering the law, but I doubt that so complicated a task could easily fit into a neat algorithm.
[Crossposted at Agoraphilia, MoneyLaw, and College Life O.C..]
Thursday, March 27, 2008
Musing about Correlations
In the course I'm currently teaching, the exams usually consist of two sections: multiple choice and short answer. I just finished recording the midterm grades, and for curiosity's sake I calculated the correlation between the two sections. For one class, the correlation between multiple choice and short answer was 0.59; for the other class, it was 0.70.
The question is, what correlation should I hope to get if the test has been optimally designed? Two things occur to me:
1. I don't want a correlation of zero, because I expect smarter and better prepared students to well on both parts (and dimmer and less prepared students to do poorly on both parts). A correlation of zero would raise the worry that my test doesn't really measure ability and preparation.
2. I don't want a correlation of one, because that means the sections are effectively redundant. Multiple choice is much easier to grade, so why bother grading the short answers if the multiple choice section tells me everything I need to know? If the two sections measure different things, correlation should be less than perfect.
So the correlation should be in between zero and one. But that's a pretty wide range. How much correlation should I aim for? Is there even a right answer to the question?
Friday, November 02, 2007
What Makes Education Special?
Megan McArdle has been doing yeoman’s service lately in defending school vouchers. I particularly liked this passage:
What I want to know from Ezra [Klein], and other liberal policy wonks ... is: why is education special? I have a model for what goods the government should buy versus what goods the government should actually provide directly; it has to do with geography, non-excludability, and transaction costs. But what is your model for saying that education is in a special class of goods that are rival and excludable, have ordinary levels of transaction costs, and yet nonetheless need to be provided directly by the government?Megan’s not arguing against government involvement here. She is conceding the need to make sure kids get educated, and if that requires some form of government involvement, so be it. The question she’s raising is, what makes education something that government should provide directly instead of subsidizing?
There are plenty of other goods and services that we want to make sure people have; food, shelter, and clothing come to mind. But with food, do we expect the government to run the farms and grocery stores? No. If some people can’t afford food, we subsidize them with welfare checks or (if we want to make sure they’re really buying food) with food stamps. Do we expect the government to provide housing? Well, we’ve had our experience with public housing, and it’s been almost uniformly recognized (even by liberals) as a disaster. So now advocates for the poor push for housing vouchers, which poor people can use to pay some or all of their rent. Do we expect the government to provide clothing? No. Again, we give poor people welfare checks, which they can use to buy clothes. If we were concerned that they were buying too much booze and too few shoes, we’d probably give them clothing coupons. (Maybe we already do.)
So, to repeat Megan, what makes education special? Education is not especially different from many other goods and services. It has no unusual features that make it unlikely to be provided by a private market in an efficient fashion. The only issue is that some people might not be able to afford it. So why not just give people money (in the form of vouchers, to make sure they spend it on education) and let them go to their provider of choice?
One more hack at this. If government had never gotten involved in education in the first place, and we were just now contemplating the creation of a government program, would we come up with anything like the status quo? Would we say, “Hey, why don’t we have government build and run a whole bunch of schools?” Or would we go the much more natural and obvious route of subsidizing kids so more of them could attend private schools? And if I’m right that it’s the latter, then why are voucher opponents so fixated on preserving the public school system?
Friday, September 28, 2007
Choo-Choo-Choose Me
Neal ponders the following sentence in his son’s school rulebook:
Choose to make good choices.Neal wonders about the seeming redundancy of the sentence; why not just say, “Make good choices”? He surmises that the sentence fits a “Choose to…” template common among educators these days: “Choose to do your homework,” for instance. Another explanation, he suggests, is the desire to place extra emphasis on the notion of choosing, which could be too subtle in the lone word ‘choices.’ Oonae, in the comments, says the phrasing makes sense “if you think that making good choices refers to a series of specifics while choosing to make good choices refers to the acquisition of an attitude or disposition.”
While I find the educator-ese tone of the sentence irritating, I actually appreciate the underlying sentiment. On all three interpretations, the motivation is to get students to embrace their own agency, understanding that they make choices that have consequences. This stands in sharp contrast to another pedagogical tradition, sadly more popular, that places emphasis on factors outside students’ control that affect their success: race, socioeconomic background, family circumstances, and so forth. Of course, the reality is that both internal choices and external forces affect outcomes. But focusing on the latter seems more likely to create a victim mentality than to encourage success.
So I was a little surprised at how Oonae followed her insightful comment above: “But I hate it anyway. It’s stupid. Ersatz autonomy. Didacticism masquerading as empowerment, and I don’t know which of these is worse anyway.” What is so ersatz about the autonomy involved? I suppose it’s ersatz if, in fact, all the students’ good “choices” will actually be forced on them. And given the thickness of the rulebook Neal was reading, that may be true to some extent. But in the context of the “choose to” statements in question, it seems more often we’re talking about genuine choice contexts, with non-coercive consequences, such as whether to do one’s homework. You can choose not to do it, and you can get an F as a result. What’s so bad about encouraging students to think of themselves as autonomous agents, capable of deciding what kind of lives to live and people to be?
Wednesday, July 04, 2007
Hello, Jonah
As I mentioned earlier, I plan to perform a song with each of the lectures that I give at the upcoming IHS Liberty & Society Seminar. You might call it a silly indulgence, but I peg it as simply good, clean fun. Done right, songs constitute good pedagogy, too. I've tried to write mine so that they reinforce the themes of my lectures.
I've already blogged about two of the songs that I plan to perform, Monster, and The Interstate Commerce Blues. I wrote a third song, Hello, Jonah, to accompany my last lecture, Living a Life of Liberty. Both the song and the lecture offer a particularly apt message for the 4th of July: Friends of liberty find it difficult, but rewarding, to both live in this imperfect world and live up to their principles.
Verse 1:
Well, hello, Jonah! And—whaddya' know?—
I see Geppetto and Pinnochio!
It's gettin' crowded, from head to tail.
Guess we're all livin' in the belly of the whale.
At least it's cozy, and nice and warm,
And it gives us shelter from the ragin' storm,
N' though I smell trouble, I can say at least,
I'm not deeper in the belly of the beast.
Refrain 1:
We stumble through this world of darkness, pain, and doubt.
We suffer in this prison, we struggle to get out.
With strivin', strength, and study, we might finally find a way.
To flee the night and finally see the day.
Verse 2:
Hey, now, Geppetto, what's that you've got?
As ideas go—hah!—I'd say that's hot.
Count me in! I like your plan,
To burn the belly of leviathan.
Thank you, Jonah, I liked your prayer.
But will you now please get over here.
You've got to labor, and not just wish,
To get out of the belly of the fish.
Refrain 2 (same as refrain 1 but for last line):
. . .
To join the fight to win the light of day.
Verse 3:
Hurray, brothers! We're finally out!
Hallelujah sisters! Raise a joyful shout!
From hell below we've now been hurled.
But we're still livin' in the belly of the world.
Refrain 3 (same as refrain 1 but for last line):
. . .
To seize the light and make a brighter day.
Thursday, March 01, 2007
The Marginal Cost of Teaching
This post began as a comment on Tom’s post below on the (small) virtues of a large teaching load, but it grew long enough to justify a post of its own. Tom makes the intriguing claim that he “enjoy[s] a downward sloping marginal cost curve” for teaching. Downward-sloping MC curves are pretty rare in the world, so I immediately began to speculate on the possible explanations.
As a first approximation, I would think the MC is flat. Teaching a class involves a fairly high fixed cost of preparation plus lecturing, which is invariant to the number of students; and also an added cost per student, mostly in the form of grading. The latter is the marginal cost. If each student’s exam/paper takes about as much time (in expected value) as any other, then MC should be horizontal.
Tom might be thinking that the fixed costs of teaching (prep and lecture) get spread over a larger number of students as the class size increases. This is true, but it means you have a downward-sloping average cost, not marginal cost.
Another possibility is that grading gets faster as you go along: with each exam you grade, it becomes a little easier to grade the next because you’ve seen the more common errors, you don’t have to recall what the original questions were, etc. This happens with me up to a point – certainly the first few exams go much more slowly than the rest – but the MC still flattens out at some point. But Tom’s grading technology may differ from mine!
Another possibility is that I’m looking at the wrong margin; I’m thinking of the MC of added students, whereas Tom meant the MC of added classes. If you’re teaching more than one section of the same class, some preparation costs are fixed (I write the same lecture notes for all sections) while others are variable (I have to deliver the same lectures more times). But even here, it seems like the MC should be horizontal after the first section.
And all of the above analysis focuses on the time cost instead of the true opportunity cost, which is the value of the lost time in alternative uses. Let’s suppose it would have been spent on research. Assume research is subject to diminishing returns; to simplify drastically, suppose that five days of research will yield 40, 20, 10, 5, and 3 pages of output. If teaching takes away one of those days, the cost is 3 pages; if more teaching (either from extra classes or extra students) takes away a second day, the cost is 5 pages; if yet more teaching takes away a third day, the cost is 10 pages; etc. Thus, once we take account of opportunity cost, in combination with diminishing returns to research, it’s apparent that teaching exhibits the traditional upward-sloping MC curve.
[UPDATE: Tom later amended his post to replace "marginal" with "average."]
Teaching Law Yesterday
Law schools have in recent years tended to give their tenure-track faculty lighter and lighter teaching loads. We can foresee the day when most law professors will routinely teach but one class a semester—and only intimate seminars, at that. But before we charge into that brave (but very laid back!) new world, we law profs might do well to recall what we stand to lose.
I can tell you about "old school" law school from firsthand experience. Last semester I taught two Contracts I sections—over 120 students. This semester I have those same students for Contracts II, plus a fair-sized Entertainment Law course, giving me 9 credit-hours to cover.
(Why am I enjoying that blast from the academic past? Long story. The short of it: Please wish our dear colleague, Prof. Frank J. Doti, well.)
Teaching three courses and around 150 students costs me a fair amount of time and attention, of course. I have less time for scholarship than I normally would. With Utah L. Rev. expecting a paper by April 1, I face a bit of a challenge. Soon, I'll also face a lot of grading: midterms for Contracts II and finals for all my classes. Whew.
But let me emphasize the positives to a heavy teaching load: 1) I can help a lot of people learn about the law; 2) I enjoy a downward sloping average cost curve; 3) I have a bigger audience for my inner (or is that "outer"?) ham; and 4) I feel like I'm earning my pay.
Don't underestimate the appeal of that last benefit. We law profs have landed pretty good gigs. Our professional qualifications—good grades, scholarship, teaching—attest to our prodigious mental powers. How easily can we coast through our academic careers, meeting only the bare duties of our jobs! Sloth thus tempts. Teaching an overload certainly offers ample balm for that worry.
[Crossposted to MoneyLaw.]
Monday, July 11, 2005
L&S: Stupid Study on Smart States
During the evening lecture, I read this article over James’s shoulder:
The smartest state in the union for the second consecutive year is Massachusetts.This method of calculating educational performance is, for lack of a better word, crapola. Some of the factors listed – graduation rates, reading proficiency, math proficiency – are plausible measures of educational output. But the other items – expenditures per student, class size, pupil-teacher ratios, and physical violence – are putative inputs into education.
The dumbest, for the third year in a row, is New Mexico.
These are the findings of the Education State Rankings, a survey by Morgan Quitno Press of hundreds of public school systems in all 50 states. States were graded on a variety of factors based on how they compare to the national average. These included such positive attributes as per-pupil expenditures, public high school graduation rates, average class size, student reading and math proficiency, and pupil-teacher ratios. States received negative points for high drop-out rates and physical violence. [emphasis added]
By including these factors in a measure of a state’s “smartness,” the study designers have assumed they actually make a difference. In fact, there’s evidence that some of these things (I’m thinking particularly of expenditures per student) are not that important. But even if the designers' assumptions are correct, the study design is still flawed, because they are double-counting. If higher spending does improve education, then the results of higher spending should show up in the real output measures, like reading and math proficiency. Likewise for class size, pupil-teacher ratios, and physical violence. There’s no need to include them separately.
I eagerly await the news reports noting that the states’ “smartness” as measured in this study is correlated with higher educational expenditures and lower pupil-to-teacher ratios (which, of course, is true by design), and offering this as evidence of the need to expand education funding and hire more teachers.
Friday, May 06, 2005
Meta-Exam Questions
Here’s an actual question from an exam I gave earlier this semester:
Professor Nerdburger is deciding what kind of exam to give. He considers multiple choice and essay exams just as good in terms of measuring students’ performance, so he only cares about minimizing the total time-cost of creating and grading exams. The opportunity cost of his time is $60/hour (or $1/minute). It takes one hour to create an essay exam, and it takes four hours to create a multiple choice exam, regardless of the number of students. It takes 20 minutes to grade each student’s essay exam, and it takes 5 minutes to grade each student’s multiple choice exam. (a) Find the total cost function for each type of exam. (b) State the type of exam Professor Nerdberger should choose, depending on the number of students in the class.The numbers are arbitrary, of course, but the problem actually does describe – at least to a first approximation – how I think about what kind of exam to give. Essays have a low fixed (preparation) cost but a high variable (grading) cost, whereas multiple choice exams have the reverse. The variable cost is dependent on the number of students in the class. Thus, the more students there are in a class, the more sensible it is to have a multiple choice exam.
[Answer: TC(essay) = 60 + 20q; TC(mc) = 240 + 5q. Professor Nerdburger should give an essay exam for any class with fewer than 12 students, and a multiple choice exam for any class with more than 12 students. For a class of exactly 12, it doesn’t matter.]
However, this approach does leave out two main factors that I consider. First, if exam questions can be recycled from semester to semester, then the fixed preparation cost can be spread out over time. Instead of a decision-rule that depends on number of students in a class, the appropriate rule (when questions can be recycled) should depend on the number of students over time. Since that number is likely to be very large for any professor who plans to keep his job, this factor strongly shifts the balance toward multiple-choice tests.
But second, most professors (including me) disagree with Prof. Nerdburger’s judgment that essays and multiple-choice questions are “just as good in terms of measuring students’ performance.” Written answers convey some information about student learning that Scantron answers cannot. This factor points toward having more essays. As a result, professors have to make comparisons of dissimilar values. Unlike Prof. Nerdburger, who has the luxury of measuring everything in terms of time spent, we have to weigh the better information attributable to essays against the greater expenditure in time to grade them.
And then there’s the possibility of other question types. In most of my classes, I end up using a mix of multiple-choice and short-answer questions, since short answers convey most of the information I would glean from essays, but they are faster and easier to grade (though not as fast multiple choice).
Sunday, August 17, 2003
Pseudo Cum Laude
Number 2 Pencil relates the sad, sad story of a young woman who would be her class’s valedictorian if not for her inability to pass a basic math proficiency exam. Eugene (thanks for the pointer) quotes Joanne Jacobs’s summary of the story: “They were giving her As for being a good kid. But they weren't teaching her.”
All this reminded me of a key passage in one of my favorite books, The Bonfire of the Vanities. I’m cutting it down substantially so this post won’t be ridiculously long… but as the saying goes, read the whole thing. And by thing, I mean the book.
“Actually, I’m calling to inquire about one of your students, a young Mr. Henry Lamb.”
“Henry Lamb. Doesn’t ring a bell. What’s he done?”
“Oh, he hasn’t done anything. He’s been seriously injured.”
…
“Henry Lamb … Oh yes, I know who you mean. Well, that’s too bad.”
“What I would like to find out, Mr. Rifkind, is what kind of student Henry Lamb is.”
“What kind?”
“Well, would you say he was an outstanding student?”
…
“… I gather you’re not from New York. … Then there’s no reason you should know anything about Colonel Jacob Ruppert High School in the Bronx. At Ruppert we use comparative terms, but outstanding isn’t one them. The range runs more from cooperative to life-threatening.” …
“Well, how would you describe Henry Lamb?”
“Cooperative. He’s a nice fellow. Never gives me any trouble.”
“Would you describe him as a good student?”
“Good doesn’t work too well at Ruppert, either. It’s more ‘Does he attend class or doesn’t he?’”
“Did Henry Lamb attend class?”
“As I recall, yes. He’s usually there. He’s very dependable. He’s a nice kid, as nice as they come.”
“Was there any part of the curriculum he was particularly good – or, let me say, adept at, anything he did better than anything else?”
“Not particularly.”
…
“Will Henry Lamb graduate, or would he have?”
“As far as I know. As I say, he has a very good attendance record.”
…
“Let me ask you this. How does he do on his written work?”
…
“Written work? There hasn’t been any written work at Ruppert High for fifteen years! Maybe twenty! They take multiple-choice tests. Reading comprehension, that’s the big thing. That’s all the Board of Education cares about.”
“How was Henry Lamb’s reading comprehension?”
“I’d have to look it up. Not bad, if I had to guess.”
“Better than most? Or about average? Or what would you say?”
…
“…these kids haven’t reached the level where it’s worth emphasizing the kind of comparisons you’re talking about. We’re just trying to get them up to a certain level and then keep them from falling back. You’re thinking about ‘honor students’ and ‘higher achievers’ and all that, and that’s natural enough, as I say. But at Colonel Jacob Ruppert High School, an honor student is somebody who attends class, isn’t disruptive, tries to learn, and does all right at reading and arithmetic.”
“Well, let’s use that standard. By that standard, is Henry Lamb an honor student?”
“By that standard, yes.”
“Thank you very much, Mr. Rifkind.”