When Science Gets Ahead Of Itself

Science certainly can get ahead of itself, but I don’t think the author chose a particularly good example of it. The beginning of this article is mostly fine — the history is laid out, and the whole idea of “science requires confirmation, and Planck didn’t confirm BICEP2, so that’s just too bad” is spot on.

It’s this:

The BICEP2 results were announced in a press conference before they had gone through the referee process. That meant the hard-core examination of the data and their analysis had not yet been subject to a peer review by someone (or a bunch of someones) who was not part of the team. It would have been the referee’s job to be merciless in his or her criticism, catch potential problems and, hopefully, make the paper better. Peer review is an awesome process and it’s one critical reason behind science’s powerful capacity for finding the true voice of the world.

Scientists often complain about how the media blow science stories out of proportion or get the details of those stories wrong. But in this case, by press-releasing their results before this full scientific process was completed, the international media machine was engaged by the scientists themselves.
…
And while, of course, everyone was careful to include “if these results are confirmed,” the point is — within weeks — people were already noticing problems with dust and the BICEP2 results related to dust.

I think there’s a fairly significant sin of omission here.

Here’s a hint: all these people who “were already noticing problems with dust and the BICEP2 results related to dust” — where did they get the detailed information required to notice these problems? The press release? No, there was a preprint uploaded to arXiv on the same day the press release was issued.

So what would have been different without the press release? Not a lot, I think. Scientists would have latched on to the results pretty quickly, and there would have been blogging and tweeting and other communication that doesn’t involve the traditional media, who might have taken a tad longer to pick up on the story. From a science perspective, though, the impact is that maybe the science discussion gets started a day later and accelerates a little more slowly. Otherwise, Clarice, the press release is INcidental. I think it only mattered to the mainstream press and scientists outside of the field who were not in a position to add anything to the discussion, and served to reduce confusion and inaccuracy in some articles (a positive, not a negative). Such as [ahem] avoiding saying “gravity waves” instead of “gravitational waves”.

The advent of arXiv is a huge difference between the actions of Pons and Fleischmann of cold-fusion infamy, who also issued a press release before peer-review. The difference is that now there is a preprint server, widely used, where physicists upload papers before peer-review and publication. Making a preprint widely available has become common practice. I can’t believe the objection is to structured discussion or dissemination of work among/between scientists before formal peer review, because that would include not only preprints, but any kind of conference or colloquium presentation that occurred before publication — those are not peer-reviewed either. That would be a rather radical objection, and I don’t think that was the intent — it’s not what I take away from the article.

If there’s a possible criticism here, it’s perhaps that the paper’s authors needed to temper their predictions based on criticism, but I don’t really have the expertise to go through and find and comprehend whatever changes they made between the original submission and the final, published article. (Oh, did I mention that the paper actually made it through peer review? Yes, it was published in June.) So perhaps they did. Whatever objections were present didn’t stop publication, nor is it clear that they should have. In fact, I think it’s likely that the widespread discussion meant that reviewers had access to more objections and discussion that took place in public than if there had been no preprint and press release. Remember, peer review isn’t a guarantee of correctness. It always comes down to verification, and more verification.

As a scientist who spends a lot of time explaining science to the public, I just wish the BICEP2 press-released team had waited. I wish they’d have let the usual scientific process run its course before they made such a grand announcement. If they had, odds are, it would have been clear that no such announcement was warranted — at least not yet — and we’d all be better off.

And I think the process did run its course, quite properly, and eliminating a press release would have changed nothing of importance. I think the author needs to metaphorically pick up the flag here. Incidental contact, no foul.