Wednesday, May 30, 2007
You may remember the story in the media from the end of 2004. A study was done which used French records of grape harvests in Burgundy to reconstruct the temperatures from 1370 to 2003. And they announced that “the summer of 2003 appears to have been extraordinary, with temperatures that were probably higher than in any other years since 1370.”The study was duly printed in Nature magazine and the rest of the media passed it on with the usual global warming spin that is now common for the media.
Douglas Keenan, over at Informath.org was asked to look at the report. First he read the article to get the gist of what they were saying. His job was simple. He was only supposed to evaluate the math used to draw the conclusions.His response was recently published in Theoretical and Applied Climatology. And he concluded that the original article “greatly overestimated the temperature of 2003 and greatly underestimated the temperatures of the warmest years in the instrumental record prior to then.”Keenan’s article in the journal only tells half the story. It notes the study was flawed but doesn’t tell what he went through to find this out. After reading the article he wrote the authors, Isabelle Chuine and others, asking for the data so he could evaluate it. They were not happy about that. On his web site Keenan writes: “The authors, though, were very reluctant to let me have the data. It took me eight months, tens of e-mails exchanged with the authors, and two formal complaints to Nature, to get the data.” He says “It is obviously inappropriate that such a large effort was necessary.”From the data he could see that the paper estimated the summer 2003 temperature “higher than the actual temperature” by 2.4 degrees centigrade. He concluded: “This is the primary reason that 2003 seemed, according tot he authors, to be so tremendously warm.” I would think so. In addition they said that previously hot years like 1945, 1947 and 1952 were “much lower than the actual temperature.” Combine these two errors together and you get the Nature magazine article. “[T]he authors had developed a method that gave a falsely-high estimate of temperature in 2003 and falsely-low estimates of temperatures in other very warm years. They then used those false estimates to proclaim that 2003 was much hotter than other years.”At this point the story gets very interesting. Keenan says the math done was so flawed that anyone could have noticed it. And he wondered why it was that the peer reviewers who supposedly read papers for Nature allowed this flawed study to be published. “I asked Dr. Chuine what data was sent to Nature, when the paper was submitted to the journal. Dr. Chuine replied, “We never sent data to Nature.”
Keenan understood that the real story here is not that the study was flawed, the math erroneous and the conclusions wrong.
What is important here is not the truth or falsity of the assertion of Chuine et al. about Burgundy temperatures. Rather, what is important is that a paper on what is arguably the world’s most important scientific topic (global warming) was published in the world’s most prestigious scientific journal with essentially no checking of the work prior to publication.Moreover—and crucially—this lack of checking is not the result of some fluke failures in the publication process. Rather, it is common for researchers to submit papers without supporting data, and it is frequent that peer reviewers do not have the requisite mathematical or statistical skills needed to check the work (medical sciences largely excepted). In other words, the publication of the work of Chuine et al. was due to systemic problems in the scientific publication process.The systemic nature of the problems indicates that there might be many other scientific papers that, like the paper of Chuine et al., were inappropriately published. Indeed, that is true and I could list numerous examples. The only thing really unusual about the paper of Chuine et al. is that the main problem with it is understandable for people without specialist scientific training. Actually, that is why I decided to publish about it. In many cases of incorrect research, the authors will try to hide behind an obfuscating smokescreen of complexity and sophistry. That is not very feasible for Chuine et al. (although the authors did try).
Finally, it is worth noting that Chuine et al. had the data; so they must have known that their conclusions were unfounded. In other words, there is prima facie evidence of scientific fraud. What will happen to the researchers as a result of this? Probably nothing. That is another systemic problem with the scientific publication process.
Now this creates a problem and this problem extends to the Intergovernmental Panel on Climate Change as well. Keenan notes that “Once something has been published in a peer-reviewed journal — particularly a prestigious journal — it tends to be considered as established, possibly even heralded as ‘truth’.” After this happens the IPCC uses this alleged “peer-reviewed research” for their studies: “Yet Yet most peer-reviewed research is not properly checked prior to its publication. In other words, most of the research that is relied upon by the IPCC, and thus government policy makers, has never been properly checked. That probably seems incredible; it is unfortunately true.”I suggest the problem is that when people get the results they want they tend not to check very carefully. Only when the results are contrary to their assumptions do they investigate thoroughly. The implication for the warming controversy would be that material by skeptics gets combed over very carefully looking for every flaw, no matter how minor. In the meantime the studies that confirm the thesis are given passes in spite of glaring errors.Since the IPCC does no studies of climate and merely reports what has been published then studies like this one on grapes gets included in the mix and conclusions are drawn on studies that shouldn’t be used. But even that is not the end of the process. After a paper is written based on studies that apparently can escape any real peer-review the politicians start rewriting the conclusions that are released to the media. And this is called a “scientific consensus” by politicians and the press.
posted by CLS at 5/30/2007 11:56:00 AM