One of the things I was thinking about this past week, in between talks (or possibly during) at ICAP was the connection with the “impossible” drive that was arguably not actually validated by NASA in the news recently. I think the seed of this was planted during a talk about trying to measure the electron’s electric dipole moment (EDM), in order to rule out some of the extensions to the Standard Model (SM). There’s no connection in the physics, but it’s the concept of ruling out certain measurements that struck me.

You might hear the phrase that absence of evidence is not evidence of absence, and that’s true, as far as it goes. If scientists have not measured the strength of gravity a million km above the sun’s north pole, for example, it would not mean that there is no gravity there. But the aphorism doesn’t work when you have done testing and can reasonably expect to get a result if some model is true. Then your absence of evidence really isn’t an absence — you’ve measured something, and gotten a null result or a small result, which rules out a larger value.

Let’s say you wanted to make a determination of the existence of mermaids. This being a physics analogy, it wouldn’t be enough for a straight up-or-down statement of their existence — you’d have a model of the conditions under which they’d be found. Someone else might have a competing model, saying they existed, but under somewhat different circumstances. Then we could go out and search for the mermaids. We search the right kind of islands at the right time of day, and find nothing. Repeat as necessary, because statistics. That’s not going to absolutely rule out the existence of mermaids, but it puts a limit on how many mermaids are statistically likely to be out there. Depending on the conditions under which we searched, it might place stricter limits on one model over another — if another model said that mermaids existed in a somewhat different environment, our search of that “space” might not have been as thorough.

There is (not unsurprisingly) an xkcd cartoon related to this

We rule out phenomena, at increasingly better levels of confidence, the longer we properly observe and don’t see anything. (“Properly” because looking with blinders on, or the lens cap in place, doesn’t count. e.g. creationists will never find “transitional” fossils because they refuse to look.) Scrutiny in duration and/or in precision, yielding null results, pushes the limits back of where any new discovery might be.

We can also see this if we go back to the days of physics before quantum mechanics, over a hundred years ago. Albert Michelson had remarked, in 1894, “The more important fundamental laws and facts of physical science have all been discovered, and these are now so firmly established that the possibility of their ever being supplanted in consequence of new discoveries is exceedingly remote…Our future discoveries must be looked for in the sixth place of decimals.” Lord Kelvin had (supposedly) announced in 1900 that “There is nothing new to be discovered in physics now; All that remains is more and more precise measurement.”

The sentiment was wrong, of course: there was new physics lurking. But one part of this was correct: that new physics was lurking in the “sixth place of decimals” or beyond. One or two anomalies aside, the new physics wasn’t found where we had already looked — that’s physics we still use to this day, in the realm of what we typically observe — it was found as the tools got better.

Which is another reason why the “impossible” drive draws so much scrutiny. We’ve been down this road before, many times and not seen anything, which is why a claim that something is there (and was there all along) is met with so much skepticism. This doesn’t say that there can be new physics. What it says is that any new physics is going exceedingly likely to be found in the uncharted waters. Mermaid sightings were claimed in remote places, not the local beach.