I’ve subscribed to Bob Park’s What’s New for the better part of the past decade; it’s a mailing list that’s still a mailing list rather than a blog, which makes it old-fashioned, to the extent that email can be old-fashioned. It’s quick commentary on what’s new and controversial, much like a blogger would do.
This past week, as part of his continuing commentary on things possibly related to global warming he mentioned something which I’m not sure the science media quite “gets,” and serves as a decent example of how science progresses:
Researchers at Dalhousie University in Halifax, Nova Scotia say phytoplankton are disappearing from the ocean. Strictly speaking it’s not really a science story — yet. There’s no independent confirmation, and until that happens scientists don’t get too excited. But Dalhousie is a respected school, and you can bet a lot of scientists are looking at sea water today.
This is how it usually goes — any new finding serves as a springboard for more investigation. A single experiment is usually not given an extraordinary amount of weight if the result is something new and unexpected, and the experiment represents a relatively small amount of data (results from large collaborations at accelerator labs are generally afforded more weight because they are gathering tremendous amounts of data). This is especially so if it appears to contradict previous experiments. Science is cautious this way. You always want to get more data, and maybe have someone else repeat the experiment, or possibly do a more advanced experiment which would only work if the foundational work is correct. That’s how you gain confidence in the results.
The unspoken part of this is that the results were properly published — there was a press release, but that was coordinated with the publication in Nature. This was not something just put out on the web or shouted from a rooftop — they followed the important first steps of the process by going through peer review.
I was thinking about this when I later read a story in the NY Times: Rumors in Astrophysics Spread at Light Speed, in which a number of recent stories are discussed, in which results were aggressively interpreted. But while the thrust of the story seems to be about how fast information can spread and the author’s disappointment that none of the rumors he keeps hearing seem to pan out, I got a different message. I saw confirmation of the tendency for the media to pick up the ball and run with it, in their rush to be first (or not be left out) with little regard for checking the facts, combined with the author not reading or listening very carefully. In the extrasolar planet example, the TED talk speaker is pretty clear he’s talking about size, and he does call them candidates. If you don’t understand the jargon, how about checking with someone first? One would hope the lesson of climategate would not be lost here — an earlier case where misunderstood jargon was reported, only to have it turn out that there was nothing to see — but I fear that lesson has already been forgotten, since the blame went to the scientists (for using the word “trick”) but seemed to pass the media by. The Higgs at Fermilab? That was a rumor posted on a blog, and the linked gawker story reports it as such.
These spread at the speed of light, in part, because nobody put the brakes on. Nobody said, “Hey, wait a tic. Maybe we should get someone else to weigh in on it.” This is the cautionary tale of Pons & Fleischmann going to the popular press before their paper had been peer-reviewed, let alone published. That was more than 20 years ago.
Hardly a week goes by, for example, that I don’t hear some kind of rumor that, if true, would rock the Universe As We Know It. Recently I heard a rumor that another dark matter experiment, which I won’t name, had seen an interesting signal. I contacted the physicist involved. He said the results were preliminary and he had nothing to say.
Smart guy. Very.
My view is that journalists shouldn’t just be relying on the restraint of scientists to remind them that preliminary results are preliminary. What if the scientist had commented? Would you run the story, knowing full well that it had not passed peer-review nor had it been independently confirmed? What is so hard about these caveats and disclaimers scientists take for granted, and come up over and over again, when discussing science results? Is the collective journalistic memory so short that scientists (or their lawyers) have to start reading a statement before they ever make a comment?
Please understand that the following result is preliminary and should not be taken as the final word. For anyone unfamiliar with the field, an effort must be made on the reader’s part to see where this fits in with the prevailing models of the day. There is a chance that it could be wrong or have only limited applicability to broader problems being investigated by other research teams. Further investigation may confirm our findings, or show that our results were anomalous or contained errors.
Scientists already know this. Journalist should know this.
ZapperZ has also commented on the NYT story
