Ginny pointed out something very important in the comments to this post:
One of the arguments in Jonathan Rauch’s “In Defense of Prejudice,” is another dirty secret is that, no less than the rest of us, scientists can be dogmatic and pigheaded. “Although this pigheadedness often damages the careers of individual scientists,” says Hull, “it is beneficial for the manifest goal of science,” which relies on people to invest years in their ideas and defend them passionately. And the dirtiest secret of all, if you believe in the antiseptic popular view of science, is that this most ostensibly rational of enterprises depends on the most irrational of motivesambition, narcissism, animus, even revenge. “Scientists acknowledge that among their motivations are natural curiosity, the love of truth, and the desire to help humanity, but other inducements exist as well, and one of them is to ‘get that son of a bitch,’” says Hull. “Time and again, scientists whom I interviewed described the powerful spur that ‘showing that son of a bitch’ supplied to their own research.” Shortly after I taught that essay we went to a family celebration, where one of my husband’s cousins, a geology ph.d. who worked for Exxon, explained to me that he was grateful Exxon had let him work for ten years before he showed he was right and he had found something useful. (I’m no scientist, if he explained it, I didn’t understand it.) But he phrased his explanation in just that manner: Those guys thought I was crazy and wrong; I was determined to show them I was right. In other words, what kept him going was his desire to show those sons of bitches. Of course, there are happier attitudes to have for ten years, but, then, the rest of us can be happy that some of those guys figured out better ways to find oil and to get it out of the ground.
The scientific method is a mechanism for the evolution of thought. Evolution depends on conflict and stuggle as its motive engine. Conflict requires competitive personalities. Those personalities are not always the easiest to deal with. QED, most good scientists are jackasses.