Think Tanks

Think Tanks are valuable sources of information. No doubt. But at the same time, I think most people can admit that they can also obscure debates on issues because they do their job too well.

In a way, I'm reminded of a previous diary that grumbles a bit about the complexity of stats and how it would seem that the same raw numbers can made to support opposing arguments....gotta hand it to those think tanks.

Megan McArdle has two posting on the issue: the first here and the follow up here .

First, the statement...ala "New Rules" by Bill Maher:

You are not allowed to argue in favor of school choice if the only evidence you can come up with is two links from Cato. You are not allowed to argue against global warming if you are relying entirely on a report from CEI. You are not allowed to talk about the recording industry based only upon press releases from the Progress and Freedom Foundation. And you are definitely, definitely not allowed to talk to me about the minimum wage if the best evidence for your position comes from EPI.

So basically, she seems to be saying that when we are debating the pros and cons of a hot button issue, we cannot and should not simply pull a "VOILA" and back up our favored positions with a simple link to research done by think tanks whose life purpose is to argue in favor of that issue. That can be a tall order, indeed. Think Tanks do all the hard work for us. It's too easy. Of course, on that note, I can honestly say that sometimes I look at research from from a think tank arguing the opposing position and I see the flaw in the what they are saying...in can be methodological, one of omission or simply something that seems relevant but really isn't. This is quite different than plugging ones ears and simply assuming they are lying.

The context of Megan's first post is on a previous post on minimum wage and her critique of the Economic Policy Institute (EPI), a think funded by unions and labor interests to argue in favor of issues from that POV. Click through and read the rest if you'd like. But the idea is simply that she doesn't hold their predictable opinions on such a matter in very high regard.

On Megan's second post, she seems to preempt the argument to the contrary with some balance. In it, she says that she doesn't rely on think tanks that argue her preferred position on an issue. But at the same time, she clarifies what I think is a good assessment of think tanks and their value:

I think they do a lot of good work. But the political policy ones do their best work when they are trying to decide policy within a movement; that's when you start seeing real innovative work. They are also very good at providing critiques of academic work in their areas of interest.

When they turn to fighting outsiders over, say, the minimum wage, the quality of their work sharply degrades. They have limited ability to change their policy position, because the donors will revolt; if they can't get an answer the donors will like, they don't ask the questions. They also only hire scholars who agree with them. That already biases their work, but then you have to contend with the groupthink problem: when everyone at the office agrees with you that your opponents are idiots, and you socialize mostly with other people in the movement, your thinking gets a tad lazy.

So if the only support for your positions comes from movement think tanks (plus maybe a few marginal academics), your position is probably extremely weak. Indeed, if someone from the other side were pulling the same trick, you would be the first to notice this. Independent studies commissioned by think tanks are especially suspect. You can't check their calculations, and survey design is easily manipulable to get the answer you want.

Concluding advice,

When debating an issue, it's better to use think tanks as a secondary source when their numbers are backed up from a more neutral source like the Census Bureau, the GAO, the CBO or the JEC...among others.
__________________________