Friday, December 21, 2007

Explaining the RAND Experiment

At Overcoming Bias, David Balan ponders the counterintuitive results of the famous RAND health insurance study. The RAND study gave one group of patients more generous coverage, and a second group of patients less generous coverage. Not surprisingly, the former group consumed a lot more health services. Surprisingly, the two groups had indistinguishable health outcomes. This is bizarre because, although we would expect health services to have diminishing marginal returns, we would not expect them to have zero marginal returns.

But Balan has a different problem with the diminishing returns explanation:
The problem is that some of the other evidence from the RAND study is not really consistent with this story. It seems that the marginal care consumed only by people with more generous insurance is not just low-value stuff. The marginal treatments consumed only by those with more generous insurance, in the opinion of expert doctors, looks a lot like the infra-marginal treatments consumed by everybody. But if that's true, doesn't it have to mean that all health care is of little value? If the marginal care looks just like the infra-marginal care, and the marginal care is of little value, then doesn't the infra-marginal care have to be of little value too?
I don’t find this objection compelling. The concept of diminishing marginal returns does not require that marginal and inframarginal units differ from each other, only that they differ in their effects. If my firm experiences diminishing returns to labor, for instance, that does not mean the additional workers I’ve hired are not as good as the previous workers in terms of their strength or skill. I could hire perfectly identical workers and still experience diminishing returns because of overcrowding, insufficient capital support, and so on. Similarly, perfectly identical medical treatments might have diminishing returns because the most good is done by the initial treatments, little good by subsequent treatments.

What requires an explanation is not that marginal returns to (even identical) health services fall, but that they seem to fall to zero. For that, I’m guessing three factors are at work.

1. As many have noted (most recently here), additional health services can have adverse consequences. Additional hospitalization can mean greater exposure to disease, for instance. If health services are characterized by diminishing marginal health benefits and increasing (or even constant) marginal health costs, the net marginal effect eventually goes negative – even ignoring the cost of providing the service.

2. We’re only looking at two different groups, one of which consumed much more health care than the other – about 40% more as I recall. If the first additional 20% generated positive net returns, and the next 20% generated negative net returns, then it might appear that the whole extra 40% had close-to-zero impact. We don’t know how a hypothetical third group receiving an intermediate amount of care would have fared; they might have done better than both groups. (If the RAND study did look at more than two groups, I haven’t heard about it.)

3. There are health outcomes that aren’t captured in standard health indicators like mortality and disability. Inconvenience and discomfort, for instance. Say that patient A and patient B had the same irritating skin rash. In both cases, the rash would have gone away on its own after a month, but patient A went to the doctor and got a treatment that made the rash clear up in a week. At the end of the year, patients A and B might well have the same measured health outcomes. But that doesn’t mean A’s additional treatment had zero marginal benefit.

3 comments:

Anonymous said...

The great increases in the average lifespan in the last hundred year can be attributed to better sanitation, the advent of antibiotics, & the widespread availability of nutritious foods. Maybe vitamin enrichment of foods has something to do with it as well. The heyday for most of those things is past. Germs mutate to make antibiotics ineffective, sped up greatly by doctor's overprescription of them. The world is evermore polluted due to human activity, making food, water & air more vulnerable to contamination. What goes up, like lifespan, may start to come down, if it hasn't already.

I agree that stays in hospitals entail hidden risks for sickness from pathogens. But the risk is there from the start of your hospital stay. It may be worse at the start because you are admitted very sick and your body hasn't had enough time to adapt to a new and foreign environment. This should be possible to verify empirically.

Anonymous said...

I have to agree with the counterintuitivity of the majority of health surveys and experiments. The three concepts that you wrote about are what I've noticed over the years of cringing while reading health studies. My thoughts:

1. Additional health services can and do indeed produce undesirable consequences. Diseases that are contacted at the hospital are called "nosocomial infections." These are viral and bacterial. The reason we acquire these so frequently is that when we are hospitalized we are usually under an undue amount of stress--psychologically and physically, which most definitely lowers our immune system. If we are ill due to an infection in the first place, our immunity is challenged on a more frequent basis. Put these factors together along with gender, age, and even race and you're pretty much a walking disease time-bomb. People who receive health care due to a trauma, let's say an open head injury, are less susceptible to nosocomial infections due to the fact that they are usually (not always) on the younger side and are able to fight infections more effectively. The fact that most head injuries lead to either doctor-induced or naturally induced comas (which protects the brain while it tries to heal itself) would rule out the natural psychological lowering of the immune system due to stress. My personal belief is that there are too many variables with health care, especially hospitalizations, for any type of conclusive study-- especially when it comes to insurance. Is insurance coverage a major consideration in hospitalizations? ABSOLUTELY! Insurance coverage is a psychological stress for a huge number of people. How can we possibly do an "experiment" or "study" with all these variables?

2. Two different groups is a ridiculous indicator of any study. Just my opinion. Take it or leave it. There are too many variables within each group. How can a + b = c in a study like this? Another thing--society would love to "group" us to determine how we can fight diseases, economic challenges, etc. We need a little of both grouping and looking at individuals in particular in order get a balance. How can this happen in an experiment like this? I'm not sure if these things were taken into consideration when the groups were put together as I'm not one of the "powers that be."

3. All health outcomes are varied due to all sorts of reasons. What would work for Patient A may not work for Patient B. We obviously all have different DNA (with the exception of identical twins,)therefore different chemical make-ups.

Anonymous said...

The folks who conducted the RAND study concluded that the group receiving more medical care did not just get more useless medicine; they also got more health-productive medicine, but the benefits were offset by additional harmful medicine.

There were many groups in the experiment. There was the group that had 100% coverage, or "free" care. There were groups that faced cost-sharing of 25/50/75/95 percent up to a high deductible. There was also an HMO group enrolled in Group Health in Seattle. Outcomes for all six groups were similar. (The HMO group also spent much less than the "free" group. Thus an interesting implication is that HMOs do as good a job of eliminating unnecessary services as high-deductible insurance.)