I agree that excluding bad data from further analysis is an accepted and common practice. I do not agree that locking down or losing the raw is standard and accepted.

When you have observations that you believe are not “right” for some reason you can exclude the data but you had better explain your reasoning behind assigning that data point “flier status”.

In any event Shannon’s point in the article is the IT practices observed by CRU crew are deplorable. Such code as the CRU code would have caused a swarm of SEC auditors to descend upon my previous client.

They’ve never claimed 99%. Basically, I think their claims of whatever percentage they use is just so much hand-waving (it seems to be based on gut feelings and hunches masquerading behind some pseudo-scientific math for cover) but to use as your premise that we’re at 99% and will move from there is already stretching things about the “certainty” of global warming.

Based on what criteria? Your personal opinion? Your gut feelings? What about 50%, or 20%, or 2%? What if it’s 100% certain that 3% of global warming is caused by humans – is that actionable?

And if it is, how much money is warranted to spend on it? How much action would you get against that 3% for a billion dollars? A trillion dollars? One hundred trillion dollars? What amount is justified?

Finally, since it appears certain that global warming is a bad thing, shouldn’t we take steps to curtail it, even if it’s not “our fault”?

This is the point re AGW that failed my “smell test” from the very beginning. Somehow I was supposed to believe that a change in climate would have wholly negative, extreme and unambiguous consequences for everyone everwhere to the end of time. More floods! More droughts! More hurricanes! More and more and more of everything that’s bad – everywhere. There was no mention of anything good or better or simply different. That bears no relation to my observations of the real world. When one area goes through a drought another might get perfect rain. And while one area has the highest snowfall on record another might have their mildest weather in years. Yet global warming was somehow destined to bring unmitigated disaster to every corner of the globe simultaneously. Tell that to all the people in frozen Siberia and other places. Basically, proponents were making the absurd claim that, miraculously (just as climate science was discovered), we are at the point of optimal climate in the whole entire history of mankind. No change in either direction would benefit anyone. I see no evidence of that claim being a fact. It’s just an assertion. Who really says, on balance, a warmer earth is worse in all respects? History suggests otherwise. Only someone who is preaching, instead of observing how natural systems actually function, could make an absolutist claim like that.

One more point about this: since it appears certain that global warming is a bad thing,

Interesting, your choice of the word “appears”. After the revelations from CRU, appearance may be all there is. And “certain” is not exactly certain, at this point. And “bad thing” is an essentially meaningless term. I’ve got a little mold in my shower. That’s a bad thing. Should I see if I can get a UN panel to tax the entire world to pay to clean it up. Without a scale (that’s accurate and reliable) “a bad thing” is not a sufficiently precise measure to justify the kinds of actions these people are talking about.

Percival Lowell thought he saw canals on Mars.

1. Crappy code is not good, but it doesn’t make the results automatically invalid

Well, bad math doesn’t automatically make results invalid but that’s the way to bet. More to the point, we’ve been told this high quality work that has play a major role in making the science “settled” to the point we justifying reengineer the entire planetary economy and tech base. I don’t think that “might not be invalid” is a high enough standard to base such momentous decisions on.

2. Sometimes there are good reasons to throw out data – the collection could have been done wrong, making even a good model fail.

In this case, the data was used to create the models. CRU was trying to establish the history of climate so that the history could be used in creating simulations of future climate. They had no valid reason to toss out any data just because it didn’t give them the answer that they wanted.

3. Scientists aren’t policymakers, either. The policymakers know their word isn’t gospel, and they aren’t taking it entirely uncritically.

Quite clearly, you don’t follow the news. All the policy makers are rather explicitly taking the climatologist word as gospel and are planning on making extraordinary sacrifices in order to forestall the cataclysm that gospel says is coming. People who don’t believe that the scientist’s word is gospel are likened to holocaust deniers. That says rather a lot about the trust politicians place in the climatologist.

4. This isn’t the only data supporting the AGW hypothesis.

So, now it’s a hypothesis? I thought it was “settled” and an established fact. Although, the CRU data is not the only data it is important data that was used to create all the climate simulation upon which we will now, and I can’t repeat this enough, reengineer the entire planetary economy and tech base. It is the only major source of data about centuries long climatic trends. All the models that use CRU data will have to be scrapped.

5. Work done 20 years ago isn’t going to be 100% compliant with today’s best practices. Pretty much every scientific result even obtained has some degree of sloppiness associated with it. It’s not appropriate to assume malicious intent because they didn’t correct for something they couldn’t have known about.

I didn’t assume malicious intent. I assumed they were amateur software developers who committed a very common mistake of letting their code evolve by accretion into something they could not manage.

6. The relative validity of a theory doesn’t hinge on one result – everything needs to be replicated, as this will be

Here’s an idea: Let’s reproduce it and then try to falsify it BEFORE we use it to reengineer the entire planetary economy and tech base.

7. It’s not an either/or situation.

When you’re talking about passing laws and fining, imprisoning or even killing people who break those laws, then yes there is an either/or threshold. Either the scientific work has enough validity to justify taking political action or it does not.

8. Even if we move from a 99& chance global warming is human caused to a 60% chance global warming is human-caused, it’s still actionable information.

What action? Reengineer the entire planetary economy and tech base action or put a little extra money into research sort of action? Are we talking about action like going to the beach over the weekend or action like invading Normandy?

Judging the economic consequences decades down the road is not and will never be science. Every attempt by scientist and others to make long range predictions about any human phenomena have failed completely and have often produced horrific outcomes (see eugenics.)

9. Finally, since it appears certain that global warming is a bad thing, shouldn’t we take steps to curtail it, even if it’s not “our fault”?

It is far from certain that global warming is a bad think. Bjorn Lombard took all the global warming models as true, took their combined most likely predicted climate changes and then attempted to calculate the probable impact. In his estimation the benefits of the most likely degree of global warming will offset the negatives for most people.

If global warming is largely natural, then we should take much different actions to adapt to it.

As far as the code quality issues go – you can’t decide who wants to do global warming research and who doesn’t.

I never said I did. I just simply want them to use professional tools and standards and to produce high quality and verifiable software. I want the same standards applied to software that we apply to every other scientific tool.

If you’ve got a better idea, do tell.

My better idea is to wait and make policy that reengineers the entire planetary economy and tech base until all the science has been hashed over and reproduced.

There’s incompetence, malice, and bad luck in every field, but that doesn’t mean the whole field is invalid

No, what makes a field invalid is its lack of predictive power. It doesn’t matter how nice people in a field are if they can predict outcomes. We’re supposed reengineer the entire planetary economy and tech base based on simulations whose predictions we cannot test.

These revelations at CRU have simply revealed that climatologist and their allied politicians have vastly oversold the predictive power of the field.

Thanks for posting this. You hit each point right on the head. I got a really good laugh out of it, especially the part about QA sexually violating code – that’s pretty close from my first-hand experience in QA.

A lack of storage space isn’t one of them. If the data is bad, the data is bad. Yet somehow this data seeped into the next iteration of the data set, so it couldn’t have been that flawed.

I almost wish I was taking a course from some of these folks, so that when they marked on my test show all your work I could come back with why, you don’t. They should be able, given just the raw data & their programs, to recreate the final dataset.

Especially since you admit that the code was worthless – “crappy” implies “crap” implies “something to be flushed away”. Since Code Is Law (h/t Lessig), a worthless piece of code is EQUIVALENT to a worthless process for achieving results. If the process does not work, and can not work, then that alone should invalidate its results. “Automatically”.

You are a poor scientist.

On the plus side, you do earn points with me for not pulling rank with a “Dr” Gunn. Maybe some more schooling would do you good.

It may or may not be “bad”. Some areas may be adversely affected, some may benefit. The fact of the matter is that climate changes. Always have and always will. It’s folly to try to stop it. A better solution is to focus policy and technology to best adapt to changing climate.

One thing that impresses me about the discussion is how much this unsystematic code construction resembles biological systems. We share 90% + of our genes with yeast and the law of unintended consequences rules everything. Natural selection chooses what works. That is not a good system when you expect the results to be consistent.

1. Crappy code is not good, but it doesn’t make the results automatically invalid.
2. Sometimes there are good reasons to throw out data – the collection could have been done wrong, making even a good model fail.
3. Scientists aren’t policymakers, either. The policymakers know their word isn’t gospel, and they aren’t taking it entirely uncritically. A scientists word doesn’t equal law, so don’t act like the data integrity is crucially important because policymakers can’t and won’t think for themselves and get other opinions.
4. This isn’t the only data supporting the AGW hypothesis. Arctic ice, polar bear habitats, and a variety of other data support the AGW hypothesis.
5. Work done 20 years ago isn’t going to be 100% compliant with today’s best practices. Pretty much every scientific result even obtained has some degree of sloppiness associated with it. It’s not appropriate to assume malicious intent because they didn’t correct for something they couldn’t have known about.
6. The relative validity of a theory doesn’t hinge on one result – everything needs to be replicated, as this will be.
7. It’s not an either/or situation. Science is about assigning probabilities to the various hypotheses out there. At no point does any hypothesis get annointed as the one true and eternal truth. It looks, to this observer, like continued work will continue to strengthen the AGW hypothesis, even if the probability contributed to AGW by this work was greater than it should have been.
8. Even if we move from a 99& chance global warming is human caused to a 60% chance global warming is human-caused, it’s still actionable information.
9. Finally, since it appears certain that global warming is a bad thing, shouldn’t we take steps to curtail it, even if it’s not “our fault”?

As far as the code quality issues go – you can’t decide who wants to do global warming research and who doesn’t. Interest dictates what is studied, so unless you can find a way to make only “good” coders interested in studying climate change, there’s going to be come crappy code that continues to be written, just as there are some crappy experiments done in every other scientific endeavor. That’s why peer-review exists and why experiments are repeated. Even if you did, those people would be criticized 20 years from now for things they did. It’s just the way science works. If you’ve got a better idea, do tell.

There’s probably been some bad actors in climate research, even after giving them the benefit of the doubt. There’s incompetence, malice, and bad luck in every field, but that doesn’t mean the whole field is invalid, even in finance. Collateralized debt securities weren’t a universally bad idea. It was assuming you knew more about the risks and overleveraging that was the bad idea.

In fact, most software engineers are incapable of understanding – nevermind coding – the math required to define the models involved in many serious researchers’ work.

This is true but it also true that programmers don’t understand accounting principles when they write banking software, they don’t understand tax code when they write tax software, they don’t understand art when they write graphics software and they don’t understand business methods when they write business logic.

Programmers don’t have to be experts on the subject being modeled in the program, they just have to understand the logical relationships that the experts want.

Professional programmers have a lot of skills and habits that amateurs, even very intelligent amateurs simply do not have. No one who has never worked in software development understands how to manage such projects.

I have over 30 years of programming experience. And I have an advanced degree in applied mathematics. For 10 years I was manager of a research group developing software for earth modeling. The end users were geologists and geophysicists working for petroleum companies. The algorithms we wrote often required understanding complex mathematical and physical processes. I preferred to hire geoscientists, mathematicians, physicists, and mechanical engineers, instead of university trained computer programmers. I found that the scientists I hired learned good programming practices, while getting up to speed on the unique math and/or physics involved, faster than the already trained programmers could learn the science. Even though most university trained programmers do study mathematics as part of their degree requirements. Heck, if I was still working, I still might hire a climatologist instead of a programmer depending on grades and what I learned during the interview process.

The Climate code is not merely hideously ugly, it is inoperable.

As a 20+ year programmer, your points hit the nail right on the head to me.

This is indeed even bigger than their fake peer review and their running from FOIA requests.

Thanks for posting this,

Rob