« Global Warming Skeptics Strike Back | Main | Policing in the Industrial Age »

April 12, 2006

Science and Incentives

McQ at QandO points to another issue that affects my discussion of global warming from yesterday: incentives. Scientists are rarely, if ever, just handed money and told they can use it to do whatever research they'd like to do. Instead they are expected to approach donors with proposals for what they intend to look for and what they expect to find. This leads to problems.

When someone is looking for a particular result, they tend to find it. This has been observed in studies where graduate students are asked to observe a particular activity. Some students are told that they should see a certain phenomonon, while others are told only to record what they see. The students who are told what they will see inevitably see more of that phenomon than those who are sent in without any preconceived notions. It will therefore come as no surprise, then, that scientists tend to find precisely what they expect to find when they conduct experiments. This is why peer-review is so important, and why consensus is so damaging to science. Part of what helps science forward is humanity's perverse need to prove other people wrong, something scientists get to do and even pretend to be helpful by calling it peer-review. But when everyone agrees that you're supposed to find X when you perform experiment Y, we can miss out on errors. (Please note that there is no evidence of which I am aware that this process is a conscious one: the vast majority of scientists set up their experiments and observe them with the intention of recording what happens even if it doesn't concur with what they expect to see. Finding what they're looking for is a result of the subconscious, not proof of venality.) So when a scientist comes to a source of funding and says he wants money to see if, as he expects, secondhand smoke causes cancer, the odds are pretty good he'll find what he's looking for.

This problem is exacerbated by the fact donors will not keep paying money for science that doesn't produce the results they're looking for. The American Lung Association is not going to give you money if your last study showed that cigarette smoke may not be as dangerous as is commonly believed, and Phillip Morris isn't going to pay you a dime if you're in the habit of noting the hazards of secondhand smoke. There is therefore a rather significant fiduciary incentive for scientists to find certain results if they want to continue to be paid for their research. Combine this knowledge with the above-noted tendency to find what you're looking for, and the tendency of the press to note when scientific studies are released by interest groups is seen to be a wise one. Although it would be wiser if they were to note it every time a study was released, and not only when studies are released by organizations the press considers suspect (the American Lung Association has the same tendency towards bias as Phillip Morris albeit in the opposite direction).

We find other problems with how the government tends to fund much scientific research, as John Stossel notes in the article flagged by McQ. If something is a big problem, it's easy to get funding for it. If a scientist using government funds discovers that there really is no need for government to get involved with an issue, that means bureaucracies don't get to expand and people don't get extra powers for dealing with the threat. As Emory University psychologist Claire Coles notes in the article, "If you go to an agency and say, 'I don't think there's a big problem here, I'd like you to give me $1 million,' the probability for getting the money is very low." Again, that doesn't necessarily mean that any one piece of government research is incorrect, only that the incentives run in only one direction: to find problems that government can then fix. This is not a good way to incentivize science, because sometimes there really are situations where government intervention is not warranted.

Again, I want to emphasize that none of the above proves that any particular piece of science is good or bad. Scientists generally are honest people trying to come up with the right answers. But people respond to incentives, even without realizing it, and the incentives for science in this day and age are to find bad things, from nuclear winter to global warming and a host of lesser subjects. That is a good thing to keep in mind when evaluating science, not to debunk it, but only to view it with the proper degree of skepticism.