Sunday, April 1, 2012

Two criticisms of the Harrit et al paper that are no longer valid [Updated]

UPDATE: Oystein and Mohr have responded. Oystein took my post as an acknowledgement that Harrit et al wasn't actually peer-reviewed. It wasn't. I do agree that Bentham isn't exactly what debunkers and skeptics would call an "established journal" - akin to Nature or Science - but that doesn't mean it wasn't reviewed. 9/11 truth is anti-establishment, so criticising it for not being accepted yet by the very establishment it's challenging is stupid and circular. My point was, you shouldn't assume from the fact that the editor-in-chief resigned that there were issues with the review. It could be that she was pressured to resign or that she was driven to resign by her own ideological commitments. The scientific community is unfortunately not immune to things like groupthink, cognitive dissonance and peer-pressure. And that was my point. I wasn't attacking peer-review per se, I was attacking idealism and elitism regarding the scientific review and editorial process. It annoys me when people try to dismiss or discredit research they don't like by calling into question its peer-review, because that's the sort of thing you'd expect from a meticulous lawyer or a nitpicky bureaucrat, not a scientist. I'd rather the discussions just be about the data.

Chris Mohr seems to agree with what I'm saying:
While I am familiar with the arguments about whether the Bentham paper was properly peer-reviewed or whether there was a proper chain of custody for the WTC dust in that study, I have always considered these to be of very secondary interest to me. Knowing some of the people at least peripherally involved in that study, I have always asserted that Harrit/Jones et al did everything in their power to preserve the integrity of their dust samples (including rejecting some whose sources were more questionable), and that perfectly good science gets reported in the Bentham journals, whatever its peer-reviewed status.
As to whether or not I accept the validity of Millette's samples, well I don't doubt that they were collected professionally, but I'm still very much open to the possibility that they may have been tampered with. That's why I said, "red/gray chips, or at least particles purporting to be them". My argument was, assuming Millette's samples are genuine, and assuming the chips in his samples are what Jones et al found, then that undermines criticisms regarding the chain of custody of Jones' samples. Those assumptions may be false, but if they are then that's more of a problem for the debunkers than it is us.



The vast majority of debunker responses to the discovery of active thermitic material over the past three years have been one of the following mantras: "Paint!", "Peer-review!" and "Chain of custody!". Dr James Millette's report on the red/gray chips has apparently given the "Paint!" mantra new life, but let's look at the other two...

"Peer-review!"

Even before the Harrit et al paper was published, debunkers were disimissing the journal it was published in as a "vanity publication" or a "pay-to-publish" journal. They ignore the reasons why the authors chose that journal. The main reasons were: 1. It's free and open access. There's no paywall preventing people from downloading the paper. And 2. The paper is very long - 25 pages with 33 coloured pictures, many of which fill an entire page. A paper like that would never be accepted by a top journal like Nature or Science. Bentham, however, was willing to publish such a paper.

Then of course, the journal's editor-in-chief resigned and the debunkers have since highlighted that fact at every opportunity. As CSI's Dave Thomas wrote in the 9/11 10th anniversary edition of the Skeptical Inquirer:
The article’s publication process was so politicized and bizarre that the editor-in-chief of the Bentham journal that featured Jones’s article, Marie-Paule Pileni, resigned in protest.
Debunkers of course assume that this must mean the paper was crap. It doesn't seem to occur to them that she may have been pressured to resign, or that her emotional reaction may be due to the implications of the paper's conclusions rather than the quality of science. Scientists and journal editors are only human. They are affected by the same conflicts of interest and ideological biases as anyone else. That's the major flaw of the peer-review process. The U.S. Supreme Court agrees...

U.S. Supreme Court opinion, Daubert v. Merrell Dow Pharmaceuticals (1993):
Another pertinent consideration is whether the theory or technique has been subjected to peer review and publication. Publication (which is but one element of peer review) is not a sine qua non of admissibility; it does not necessarily correlate with reliability, and in some instances well-grounded but innovative theories will not have been published. Some propositions, moreover, are too particular, too new, or of too limited interest to be published. But submission to the scrutiny of the scientific community is a component of "good science," in part because it increases the likelihood that substantive flaws in methodology will be detected. The fact of publication (or lack thereof) in a peer-reviewed journal thus will be a relevant, though not dispositive, consideration in assessing the scientific validity of a particular technique or methodology on which an opinion is premised.
In other words, what the Supreme Court is saying here is, don't put "peer-review" on a pedestal! The process is not without its flaws, especially when it comes to controversial issues such as global warming or 9/11.

Publication and peer-review are just the first stage of the scientific process . The stuff that comes after - replications, rebuttals, debates, symposia etc. - are much more important than the initial publication. Pretty much every paper ever published has been either expanded upon or refuted later on. And so you could look back on any paper and say, in retrospect, based on what we know now, that that paper, if it had been submitted now, wouldn't be acceptable. But that doesn't mean that it shouldn't have been published back when it was. This is the nature of science.

Consider, for example, NASA's 2010 claim of finding arsenic-based life, or CERN/OPERA's 2011 claim of faster-than-light neutrinos. In both cases, the publication and announcement of these finds attracted a great deal of controversy, and both claims were recently refuted by replication, but that doesn't mean the initial results were fraudulent, or that the claims were unscientific.

The fact that skeptical scientists have attempted to independently replicate and rebut the findings of Harrit et al means the hypothesis has progressed to the next stage of the scientific process - meaning any criticisms of its initial peer-review are now null and void. For years, debunkers have basically said, "we're going to dismiss this paper because we don't believe it was properly peer-reviewed". But now, this position has been undermined. As I pointed out before, the JREFers' support of Dr Millette's study was an acknowledgement of the nanothermite hypothesis' scientific legitimacy. They can no longer argue on the basis of editorial controversy that the claims of Harrit et al should not be taken seriously, because they DID take them seriously!

"Chain of Custody!"

In a September 2009 debate between Dylan Avery and Pat Curley, Pat questioned the chain of custody of Steven Jones' dust samples. When Dylan noted that the chain of custody is documented in the Harrit et al paper, Pat's response was:
I don't think you're gonna find that this stuff was hermetically sealed, that it was labeled at the time that it was taken - all the sorts of things that police would do with something that they're using as evidence.
Pat does sort of have a point here about the samples not being collected professionally. But as I wrote in response at the time:
By questioning the chain of custody you are effectively accusing the scientists and the citizens of conspiring to fake evidence by manufacturing high-tech energetic nanocomposites that only a handful of labs in the world can even make and adding them to samples! That sounds like a crazy conspiracy theory to me! And yet you find the idea of the government tampering with evidence ridiculous! Someone get Pat a tin foil hat!
Now that red/gray chips, or at least particles purporting to be them, have been found in professionally collected samples independent of Steven Jones', debunkers can now be assured that these red/gray chips, whatever they are, did not enter Jones' samples via accidental contamination, and were not intentionally added by 9/11 truth activists. So criticisms regarding the collection and chain of custody of Jones' samples are now null and void.

These two debunker arguments have now been undermined by the debunkers themselves. If they were intellectually honest, they'd stop making them. But I doubt they will. Their "Peer-review!" and "Chain of custody!" mantras function as sort-of quasi-ad-hominems. Debunkers are more interested in discrediting the research than they are in having a genuine scientific discussion about it.