I see the phrase “absence of evidence is not evidence of absence” tossed around often-especially in terms of religious debates. This phrase, however, isn’t actually true. There are cases in which absence of evidence IS evidence of absence. These are cases where there is a low prior probability. To illustrate this, I’ll use a common example where the evidence is exceedingly high and the prior probability is low.

I’m going to use a tool known as Bayes’ Theorem. This is an equation that let’s us calculate the probability of an event taking into account the evidence. It’s essentially the mathematical basis for the famous Carl Sagan quote: “Extraordinary claims require extraordinary evidence”. The equation for only two options is:

\[ P(h|e)=\frac{P(e|h){\times}P(p)}{P(e|h){\times}P(p)+P(e|-h){\times}P(-p)} \]

That is, the probability that our hypothesis is true given the evidence (P(h|e)) is dependent on the probability of the event without any evidence for this specific instance (P(p) called a “prior probability”), and how well the evidence fits with what we would expect to see if our hypothesis is true (P(e|h)).

Now, to the example with a lot of evidence and a low prior probability. In 2011, an estimated 0.5% of the general US population used cocaine. This means the prior probability of any given person is really low. Let’s assume we have a drug test with a 99% accuracy. This means the evidence if it shows up positive is really good for someone having actually done the drug. Now, let’s pop in the numbers:

\[ P(h|e)=\frac{(0.99){\times}(0.005)}{(0.99){\times}(0.005)+P(0.01){\times}(0.995)} \]

This turns out to be about 33%. That is, with a 99% accurate drug test, if a person tests positive for cocaine, there is about a 66% chance of it being a false positive. So, given a low prior probability, even with really good evidence, the hypothesis is likely to not be true.

There are indeed cases where absence of evidence is evidence of absence.

No Comments »