WHO's Healthcare Rankings, Part 3

[NOTE TO NEW READERS: This series of blog posts culminated in a Cato Institute Briefing Paper, which discusses all of my criticisms of the WHO healthcare rankings.]

This is probably my last post on this topic, but no promises. I want to draw attention to the margins of error associated with WHO’s healthcare rankings, which media reports on the rankings typically neglect to mention.

If you look at the study that produced the rankings for “overall health system attainment” (this is the one that ranked France, Canada, and the U.S. 6th, 7th, and 14th, respectively), an 80% confidence interval puts the U.S. rank anywhere from 7th to 24th. France is anywhere from 3rd to 11th, Canada from 4th to 14th. Here is a blown-up section of the study’s Figure 2, which shows the intervals graphically:

The U.S. is not named specifically on the horizontal axis, but it is the country just to the left of Iceland. Notably, its interval is a good bit wider than those around it. Obviously, there is considerable overlap among these intervals, and we cannot say with great confidence that the U.S. doesn't rank better than both France and Canada.

But these intervals result only from errors associated with random sampling in the construction of the statistics. They do not consider differences that could result from different weightings of the factors that compose the attainment index. As I argued in the two previous posts, there is good reason to think the proper weight for three of these factors is, in fact, zero. The authors of the study did not calculate rankings based on that weighting, but they did consider some other possible factor weights. Here is a blown-up section of the relevant graph (the study’s Figure 5):

According to the study, this figure shows that “for only a small number of countries was there any substantive change in rank” as a result of different factor weights. But looking at the section above, one country stands out as having an especially wide interval. That country is the one just to the left of Iceland – once again, the U.S. In other words, the ranking of the U.S. healthcare system is especially sensitive to the choice of weights in the index.

And, it should be noted, the rank resulting from any given factor weighting will itself have a margin of error resulting from random sampling. That means the two different sorts of intervals shown above ought to be considered jointly, resulting in yet even wider ranking intervals. More reason, I think, to regard the WHO healthcare rankings as unreliable at best.

[UPDATE: See also Part 1 and Part 2.]