Are You a Good Witch or a Bad Witch?

Star dust casts doubt on recent big bang wave result

[O]ther things apart from gravitational waves, such as dust, can emit polarised photons.

To minimise the chances of this effect causing a false signal, the BICEP 2 team pointed their telescope at a patch of sky far away from the Milky Way’s dusty disc. Then they used models of the dust in that part of the sky to estimate its effect on the polarisation. They found that this could account for no more than about 20 per cent of the signal that they attributed to gravitational waves last month.

But Mertsch says the models they used didn’t account for dust shells produced as the expanding remnants of supernovae slam into surrounding gas and dust.

So the results may not hold up. This is bad, right? We get all worked up about a result, and it turns out it might not be correct.

Except it’s not. Procedurally, this is good. This is exactly the way science is supposed to work. You do your best attempt at the research and looking for confounding effects that might be giving you a false signal. You have it peer reviewed, and you publish. As I’ve talked about before, that’s just the first hurdle. After that comes feedback from other scientists, including attempts at replication. And that’s where we are now — some attempts at replication are already ongoing, and here we have someone who has new information that might affect the conclusion. This is how science moves to get things right.

The bad part is that the first results get hyped, because they’re new, and there’s no restraint for waiting for the weight of evidence to pile up. Sometimes the first results won’t stand up to scrutiny. I recognize that this might weaken the confidence some have in science, but in reality it should strengthen it: other scientists are willing (enthusiastic, even) to stand up and say, “Wait a minute!” when all the ducks don’t line up. There’s no conspiracy to conform. Scientists questioning loudly trumpeted results like this gives me confidence that all of science is subject to the same kind of feedback.