Why There?

I have had my attention* directed to the recent publication of some rather interesting predictions about global warming and tropical storm activity in the Philosophical Transactions   of the Royal Society A (Mathematical, Physical and Engineering Sciences). My first reaction was: why Philosophical Transactions A? Especially for two researchers from Georgia? Then I looked at the journal’s internet masthead:

Philosophical Transactions A is expanding and most journal issues will be dedicated to the publication of Theme Issues in four subject clusters:

                             

• Nano-science nanoengineering and quantum computing
• Environmental change and renewable energy
• Dynamical systems and complexity
• Biophysics, biological mathematics and medical engineering

The reason that the choice of journal raised my hackles is that the Royal Society’s Transactions is not the first choice for a meteorological article of such startling significance. It has a middling-low Impact Factor, and most scientists** strive to get their research published in as prestigious a journal as possible in order to win the publish-or-perish games that are the lifeblood of Academy politics.

Before you read this post, you are going to need a little background if you are not a scientist or an Academic engineer. Derek Lowe had some good pieces  on the subject of Journal Impact Factors a while back, and I encourage you to read them first. For a detailed definition of the Impact Factor, go here.

For the rest of this piece, I took that latest publicly available IFs from 2005. I doubt things changed that much for 2006, and we’ll know about 2007 soon enough. If you are interested in the raw data, go here and click on the first link for the Excel file of the 2005 IFs.

So why did publication in Philosophical Transactions A (PTA from here on down) raise my concern for a piece that has political overtones? The journal has an impact factor of 2.224. To give you a sounding on what that means, I’ll have you old fogies remember back to 1989, and the press release of cold fusion by Fleischman and Pons. It all sounded very promising. Then my P-Chem teacher snorted in derision: but they’re publishing in the Journal of Electroanalytical Chemistry.

At the time I didn’t understand what Dr.  T meant. Once in grad school, I quickly discovered the pecking order of journals, and came to the conclusion that if I thought I’d just discovered Cold Fusion, I’d be submitting to Science and / or Nature post haste. Getting a pub in Science or Nature is to a scientist what getting on the front page of the Wall Street Journal is to a businessman or woman. Not only that, they have access to tough, top tier peer reviewers, which lends an aura of legitimacy to a publication making startling claims. This is directly related to their impact factors – 29.273 for Nature and 30.927 for Science.

Back to Drs. Fleischman and Pons: the JEOC has an Impact Factor barely below that of PTA 2.222. It raised a lot of concern at the time that they would not submit to a more prestigious journal, and those concerns were subsequently borne out. So you can see why I look askance at the submission of this tropical storm paper to PTA when the Journal of Climate has a 2005 IF of 3.402 and the Bulletin of the American Meteorological Society has an IF of 3.055. Moving even lower, Climate Change, which one would think would be a natural fit for this kind of publication, has an IF of 2.479. But reviewers there are going to knowledgeable and tough. The only reason to submit to PTA that I can see is to either dodge the tough reviews in the first place, or get something into publication that has been rejected by a better journal (a time-honored and perfectly legitimate practice, if the rejection was more about subject matter fit and importance of the work, rather than methodological issues). But generally one works one’s way down in the specific field after rejection by Science or Nature. One does not look for other general science journals, especially ones with lower IFs than ones in the field.

But impact factors themselves don’t tell the whole story. As Derek said:

Here’s a broadside in the British Medical Journal, pointing out (among other things) that the individual papers inside a given journal follow a power-law distribution, too. It’s glossed over by the assignment of a single impact factor to each journal, but the most-cited 50% of the papers in a given journal can be cited ten times as much as the lesser 50%.

                           
The less interesting papers are getting a free impact ride, while the better ones could have presumably been playing off in a super-impact league of their own, if such a journal existed.

I went and conducted a crude Google scholar search on the keyword Meteorology, restricting the journals to PTA and Nature. Over the recorded history in Google, PTA had 362 cites, most of which were historical, with a few hard articles, but very few. Nature had 596 cites, most of which were editorials, but with many more hard science articles in the first 100 or so in the list. What I’m getting at is that meteorological articles in PTA are likely to be in that lower 50% getting a free ride because PTA is a mid-to-lower tier journal read by scientists of all types, whereas the lower 50% of weather-type articles in a journal such as the Journal of Climate are likely to be of higher quality than the PTA’s best meteorological articles because an article in JC is going to be read by a lot of people with a lot of specific knowledge who can and will yell about BS publications. None of the Chemists and Physicists who read PTA will have that type of knowledge.

Between PTA’s stated goal of carving a niche in the publishing world by printing theme issues (thus increasing the danger of printing an even higher ratio of stuff in the lower part of the power law curve in order to fill out a thematic issue to which few good papers have been submitted) and their dearth of previous meteorological pieces, I’m skeptical of how much honest criticism this paper received before publication. For a confused layman trying to make sense of scientific arguments, this is a clear sign that the article must be viewed more carefully regarding the validity of its claims.

One piece of the peer review dynamic to keep in mind is that as a submitter you can request that certain reviewers be included or excluded from the review process. Professional rivalries can and do color reviews, and this is a legitimate exercise. The Editor has final control over who reviews however, and the higher the IF of the journal, the more easily the Editor can call up the top guys and gals in the field for an impartial review of extraordinary claims. Lower tier journals tend to go along with the “please send this to…” list.

Not to say that the claims in this tropical storm paper might not be true (in this case, I highly doubt that they are, but I don’t want to stray into argumentum ad journalum, here), but an IF for the journal in question does give one a good idea of the level of scrutiny in the peer review.

* The genesis of this post was a piece by Tree Hugging Sister at the Coalition of the Swilling.

** The reclusive genius J.W. Gibbs notwithstanding.