Friday’s XKCD was one of Randall’s better ones, and I see that Matt has already commented on it

>

It goes right to the heart of one of the greatest philosophical difficulties of science. All we can do is measure correlation. We can never be assured that we’re not just getting lucky and that in fact the fundamental-seeming physical laws we deduce are just flukes.

And this is true — science is inductive, and we draw conclusions rather than prove. What distinguishes science from superstition is what happens next. Correlation is the start, but can easily be wrong, which is the basis of the logical fallacy post hoc, ergo propter hoc (happened after, therefore was caused by). If one is not cognizant of this, one might notice that the US never used nuclear weapons until after women got the right to vote, and think there’s meaning to the correlation.

So we ask for more. What we can do is set up conditions where if the phenomenon does happen to be a fluke, that the odds of it being so are really, really small. (Flip a coin and get heads 10 times in a row? Doesn’t mean you have special powers. Do that significantly more often than once per thousand attempts and we’ll talk). That’s the power of statistics, and why a single event is not enough to demonstrate causation. But even then, there are potential pitfalls. Two correlated effects might be caused by a common factor. If you don’t consider this possibility, you might conclude that buying a Lexus causes people to vote Republican.

But even that isn’t enough. We also want there to be a plausible mechanism that we can model, and use that to predict other behavior. Then you test — can you turn the effect on and off, and do it in such a way that eliminates other explanations? And the tests must be rigorous, with specific predictions and carefully executed experiments. It’s only after that testing that the suggestive winking of correlation can be reasonably concluded as causation.