Junk Science Warning Signs: Part II

Another aspect of being a discerning customer of scientific information is that the careful consumer looks at the source of the information. Not as an ad hominem on any particular researcher, but with an eye to how much quality control went into the peer review, as I mentioned in my last post.

This is directly related to an old post of mine: “Why There?“, and that is a question that you should ask yourself every time a new publication hits the lay press:

I’ll have you old fogies remember back to 1989, and the press release of cold fusion by Fleischman and Pons. It all sounded very promising. Then my P-Chem teacher snorted in derision: but they’re publishing in the Journal of Electroanalytical Chemistry.
_
At the time I didn’t understand what Dr. T meant. Once in grad school, I quickly discovered the pecking order of journals, and came to the conclusion that if I thought I’d just discovered Cold Fusion, I’d be submitting to Science and / or Nature post haste. Getting a pub in Science or Nature is to a scientist what getting on the front page of the Wall Street Journal is to a businessman or woman. Not only that, they have access to tough, top tier peer reviewers, which lends an aura of legitimacy to a publication making startling claims. This is directly related to their impact factors – 29.273 for Nature and 30.927 for Science.
_
Back to Drs. Fleischman and Pons: the JEOC has an Impact Factor barely below that of PTA 2.222. It raised a lot of concern at the time that they would not submit to a more prestigious journal, and those concerns were subsequently borne out. So you can see why I look askance at the submission of this tropical storm paper to PTA when the Journal of Climate has a 2005 IF of 3.402 and the Bulletin of the American Meteorological Society has an IF of 3.055. Moving even lower, Climate Change, which one would think would be a natural fit for this kind of publication, has an IF of 2.479. But reviewers there are going to knowledgeable and tough. The only reason to submit to PTA that I can see is to either dodge the tough reviews in the first place, or get something into publication that has been rejected by a better journal (a time-honored and perfectly legitimate practice, if the rejection was more about subject matter fit and importance of the work, rather than methodological issues). But generally one works one’s way down in the specific field after rejection by Science or Nature. One does not look for other general science journals, especially ones with lower IFs than ones in the field.

I made mention in the previous post that another reason to keep your eye on the journal’s regular audience is that if the article is on a subject not regularly studied by the audience, the peer review is going to be a little sketchy. Sometimes, if the editor is not careful, the whole journal can become sketchy. There are some really low tier synthetic chemistry journals that might just as well be titled The Journal of Mostly Irreproducible Results (cough Tetrahedron Letters cough).

Once the editor goes bad, though, the whole journal is in trouble. And imagine my surprise to stumble across evidence that the editor of the journal most favored in recent year by my favorite Academic crank Otto Rössler is in trouble.

I first learned of the trouble from a post on this blog post by Philip Davis Рwritten before I wrote my piece on R̦ssler.

If you didn’t click the link, the post describes a publishing scandal very similar to the editorial shenanigans that I described in detail in my post about Otto Rossler, involving the very same journal, but this time surrounding the Editor-in-Chief, Mohamed El Naschie* rather than Rossler. It seems that El Naschie has published 322 papers in his own journal. When one considers that even a crank like Rossler has managed about 50 less in a 40 year career, one’s mind begins to boggle.

As Phil notes in a follow-up post, El Naschie has been replaced as EIC, probably a case of too little, too late.

It does not surprise me that the Editor-in-Chief and founder of Chaos, Solitons and Fractals was involved in scientific controversy. It surprises me even less that this controversy is based on behavior similar to Rossler’s: self-publishing articles that would be subjected to intense peer review in a journal with a high reputation. Acceptable behavior is dictated by behavior at the top, and Rossler was just following his boss’s lead.

One thing that Davis said that is really, really good for the layman to keep in mind is this:

A journal is a community of individuals, and membership in each community is conferred with the successful transfer of a manuscript. If this gift is accepted, the author receives a symbolic transfer of prestige back from that community. Prestige is legitimated when it is recognized by the broader scientific community. If I have never heard of your journal, then being an author means nothing to me. If your article is published in a controversial journal, then that association is transferred as well.

This is another way of looking at what I was saying in the previous post on Rössler. Of course people want to be good scientific citizens. Of course scientists want to publish in order to share their discoveries with others so that others don’t repeat their work, but go on to advance science even farther by building on it. But that does not mean that the publishing scientist does not want to obtain the greatest amount of standing in return for their effort.

In my literary days I was a historicist. Historicists admit that there is a social aspect to every human endeavor. Unlike post-modernists, however, historicists never jumped the shark and claimed that the entire process to anything (especially science) is entirely social.

So, lets be clear that I’m agreeing with Davis as a historicist, not as a post-modernist. The social aspect of journal publishing outlined by Davis is a proxy for a process of physical reproduction of the experiments in a new publication. In the perfect world, all reviewers of a new manuscript would be required to reproduce the results in a new paper as part of the peer review process to provide a firewall against scientific fraud. In practice, there is not enough time in the day.

So top tier journals unleash the big dogs as peer reviewers, and those top scientists are likely to have performed experiments similar to the ones in a hypothetical new publication. The bottom of the barrel often performs no peer review beyond the editor and perhaps one other reviewer, usually also on the editorial board. In the case of CS&F, even this minimal review was skipped for the editor’s publications.

When I dismantled Rössler’s reputation in that previous post, I had not fully researched Chaos, Solitons and Fractals beyond finding that quote by Wen Zhen describing the unethical editorial practices that led to the inflation of the journal’s Impact Factor. Since then, I’ve dug a little deeper into the matter, and I’ve seen a lot of quotes similar to this one:

For you see, the most simple minded, stupid, and yet pervasive index of journal quality is the Impact Factor, peddled by none other than (you guessed it) Thomson Scientific, which determines the quality of the journal by the brilliant and subtle method of dividing the number of citations are received from publications indexed by Thomson by the number of articles in the journal (in other words, the sort of thing a monkey would come up with). As a result mostly of citations to itself, CFS has a higher impact factor than any mathematics journal, even though it is worthless pseudoscience. (Journals in other fields consistently have higher IFs than mathematics, since they write more, shorter, papers, and thus tend to cite each other more often).

One of the legitimate complaints about Impact Factor is that it does not take into account the size or the publishing norms of the field. Mathematics is a small field that publishes infrequently, so going by simple number of papers cited, journals in the field are going to get lower IFs than Biology journals which is larger and depends on rapid communication to grow the knowledge base**.

But once I turned my, well, not baleful eye, more an exasperated and pissed off hairy eyeball, to CS&F, well, as my friend MWT said, the whole thing stated to look like digging out a tumor with a spoon. The entire journal is tainted from the crank-editor-in-chief down to Rossler.

Let’s start at the beginning, shall we? Ah, screw that, I’m tired of this cluster of nuts. I, like Eric, want to write about something fun for a change. But when the LHC turns on again, the whole crew we’ve been writing about will get another burst of publicity, and I want to have everything laid out in black and white pixels so I can lump it all together in one big meta-post and mail it to the major media outlets to let them know that a group of bloggers who actually, you know, went to school and studied stuff, rather than studying how to write about stuff, actually did their work for them, thank you very much.

So here, in as concise a post as I can manage, is the evidence that anyone who has published in CS&F should take those papers right off of their CV and hide them, and anyone, such as Rossler, whose publication list is riddled with cites to that journal, should be laughed out of public discourse.

First of all, I became aware of the greater scandal with CS&F via the n-Category Cafe, a fine science blog that you all should read:

It is sad that some academic institutions and, in larger extent, some publishers back those people up. For example, Elsevier has a journal called Chaos, Solitons and Fractals, included unfortunately in the A+ category in quality by the Australian Academy of Sciences, in a powerful commercial citation factory called Current Contents and with high “impact factor” over 3. It is not that in Chaos etc. there are no good papers, some are normal regular hard science. But, a significant and very visible percentage of papers there belong to one and the same group of people including the very editor, certain El Naschie, a person with many bogus affiliations, and writing in recent years papers with practically no arguments but high predictions based on numerology, coincidences and fancy pictures combining Lie algebras, chaos theory and so on, at the layman level.

Please allow me one digression here – funding and other goodies are doled out by governments based on a mixture of quality and quantity of publications of the scientist in question. One of the main indicators of quality is the IF, so the game mentioned by Wen Zhen has serious implications on the distribution of public monies, when non-sensical, self-referential papers in CS&F are graded as A+ by a government funding agency. THIS is why cranks need to be driven out publicly and quickly – scarce research dollars should not be spent on them, but rather on legitimate research.

Those of you familiar with the Walter L. Wagner story will see a depressing repetition here. El Naschie was trained as an engineer, and claims a Ph.D. from Cambridge. Unfortunately, Cambridge no longer claims him:

One further aspect is whether El Naschie’s PhD thesis, claimed to have been accepted at University College London in the 1970s, exists or not. No clear information on this has yet emerged.

Beyond his Ph.D., El Naschie claimed affiliation with Cambridge for a long time with no clear basis in reality:

El naschie keeps publishing junks in CSF for a quite long time and kept unnoticed by mentoring system of Elsevier which seems very odd. While it was so obvious from the far beginning that we have a crackpot.

The same applies to Cambridge university which allowed him to publish his articles for nearly ten years 1993-2001 using its affiliation, while, for sure, he wasn’t a staff member there. It is far from reality to imagine that people in Cambridge have been fooled for that long time.

But what, exactly, is he publishing? Well, as Derek Lowe noted, the physicists and mathematicians who have actually read his work have a pretty low opinion of it:

While I’m not qualified to referee his works, those who are report that his papers don’t make much sense – “undisciplined numerology larded with impressive buzzwords” is one review at the UT site. (That’s a phrase I’m going to have to remember for future use; it’s bound to come in handy).

The Quantum Pontiff also notes the skepticism of reviewers interviewed by Nature about the scandal:

Scene three: tensions rise. Peer reviewed or not peer reviewed, that is the question:

Most scientists contacted by Nature comment that El Naschie’s papers tend to be of poor quality. Peter Woit, a mathematical physicist at Columbia University in New York, says he thinks that “it’s plain obvious that there was either zero, or at best very poor, peer review, of his own papers”. There is, however, little evidence that they have harmed the field as a whole.

Hmmmm. The extent of harm is often hard to judge in a delayed feedback loop. The University of Frankfurt is reportedly initiating an investigation of El Naschie’s claim to be affiliated with them, when his actual affiliation is:

a private association, called the “Frankfurter Förderverein für physikalische Grundlagenforschung” (Frankfurt association for the support of basic research in phyiscs).

It is interesting to note that Thompson really set itself up in this interview with El Naschie concerning the article On a fuzzy Kahler-like manifold which is consistent with the two slit experiment which was published in the other journal caught up in this scandal (CS&F and the International Journal of Nonlinear Science have been accused of requiring authors in either journal to cross-cite to drive up the IMpact Factor).

In fact, I have never recognized the traditional lines of demarcation between the sciences, not even between theoretical physics and engineering, let alone pure mathematics and applied physics. Thus, the melting of math, physics, and experimental realism may have appealed to similarly-inclined researchers and thus led to the high citation rate of this particular paper.

However, in any event, one should not forget that my approach in this paper, namely geometrizing physics, is in a direction where the majority of theoretical physicists working on the Minkowski-Einstein program are involved, and that the two-slit experiment which I attempt to resolve in the same paper is arguably the most famous and most difficult problem in quantum mechanics. There are also possible applications, as yet undreamed, for this experiment in nano and quantum technology. This may also have contributed to the high citation rate.

This attack on “traditional lines of demarcation” is quite a large red flag for crankhood – most real scientists are actively multidisciplinary, but also very, very cautious about going to far afield form their core training without being very careful that they are not making fundamental mistakes. And when the man boasts about high citation numbers? Well, just take a gander at this Google Scholar page, which gives the actual papers citing one of his more famous papers in Google Scholar at 38 cites. I sure didn’t find any cites without El Naschie listed as an author. To give you a calibration, I have not published in the open scientific literature in about 10 years since becoming one of the evil minions of Industry and changing career paths, but my most-cited paper has been cited 345 times in 11 years, and only about 5 of those cites are from papers I authored.

Elsevier’s practice of bundling journals so that librarires have no choice but to buy crud such as CS&F if the library wishes to purchase Elsevier’s more prestigious journals has not made the publishing house any friends in Academia. The biology community is not happy with their decision to publish the journal Homeopathy, which lends a patina of legitimacy to that most unscientific of “disciplines”.

I’ve been loath to jump on the bandwagon excoriating Elsevier, when I know that their journals do provide a publishing outlet when the channels at the Society-sponsored journals are full – there is just not enough space in the more “altruistic” formats to publish everything that comes out of Academia, and private publishers do perform a useful function in filling the gap. No one complains about Nature Publishing Group turning a profit, because they maintain their quality standards. The issue is not one of private versus “altruistic”, and Academics would do well to remember this before going off into tirades that betray their ignorance of economics. The issue is that Elsevier is behaving as a near-monopoly in a way that leads to one of two possible conclusions – either they are asleep at the wheel, or they are publishing and bundling known crap, and degrading the prestige of their other journals in the procerss purely out of greed.

One of those two conculsions is correct, but I do not have the time or resources to ascertain which one. And in fact, both lead to the same unhappy end for Elsevier, unless they clean up their act. The only conclusion I can come to for certain is that anyone who regularly published in CS&F is tainted. I know that I would be taking any publications in CS&F off of my CV, if I had any.

* For the scientifically uninitiated, that “lecture” is chock full of vague buzzwords, banal observations, and scientific non-sequiturs. What exactly this branch of mathematics has to do with nanotechnology, I have no idea, and I speak as someone trained in several branches of nanotechnology.

** Biology, let’s be very frank here, is the easiest of the hard sciences, or at least the one that requires the least math. People who balk at chemistry or physics at junior and senior levels of college because of the sharp increase in applied math at that level tend to drop into biology as a consolation prize. In science there is a very definite hierarchy of perceived difficulty based on math content (and hey, I’m a P-chemist, so I say that there is a high correlation between perception and reality here :p). Taking my tongue partially out of my cheek, biology is also the science that’s going to give you the greatest bang for your buck in terms of innovation at this point in history, so it is not exactly a bad thing that biology is heavily populated right now. But take a poll of 100 doctors and ask how many started as Chemistry majors and switched to biology when P-Chem loomed large. The driver for migration into biology is not perceived utility, but perceived ease of obtaining a degree with decent grades. Even when biology begins to become less fruitful 50 years hence, it’s still going to attract more students than high math disciplines.