More Damning

As I putting together my long-delayed eighth It All Adds Up column, I suddenly realised that Ben Stein's witless bleating actually has a name: "Prosecutor's Fallacy".

It's always worth explaining the fallacy, because it's so commonly held.  Stein's particular variant of it boils down to the following: "Almost no rapists are famous economists, therefore almost no famous economists are rapists."  The problem with this argument becomes entirely obvious when you point out that very few French people are famous economists.  Despite that fact, however, Stein seems entirely happy to believe Strauss-Kahn regarding the latter's country of origin.

A more mathematical (though very simple, I promise) demonstration after the jump.

Every year I was in Durham, I gave my first year probability students the same question in tutorial which demonstrated why Prosecutor's Fallacy is, indeed fallacious.  Essentially, the question runs like this.  Suppose there is a disease suffered by only one in every ten thousand people.  Despite its rarity, its severity is such that a very accurate test has been developed to tell whether someone is suffering from it.  The test is 99% accurate; that is, if you have the disease, there's a 99% chance the test will be positive, and if you don't have the disease, there's a 99% chance the test will be negative.

Suppose you get the test and, horror of horrors, it comes back positive.  What is the chance you actually have the disease.

Even very smart people (including my Panel Talk co-host) tend to assume the answer is 99%.  It's a 99% accurate test, after all.  But that isn't true, and you can see that it isn't true very simply, by putting some numbers into the pot.

Let's assume we run the test on one million people, and that within that one million people, one hundred have the disease (this is consistent with the one in ten thousand figure).  We can split this group into four smaller groups.  First of all, there are 999 900 people who don't have the disease.  Of them, 989 901 will get back an accurate test result, telling them they are indeed disease free.  Unfortunately, the remaining 9 999 people will all get back an incorrect positive result.

On the other hand, of the one hundred people who do have the disease, 99 will be told so by the test.  One unfortunate individual will actually get back a negative test result despite actually being infected (sucks to be them).

So what does that mean?  Well, it means that for 99 people who get a positive result because they're infected, there are 9 999 people who get a positive result because the test went wrong. In other words, if you've received a positive result, there's a less than 1% chance you actually have the disease.  Indeed, if everyone who got a positive result requested a second test, then even then about 100 of the 9 999 disease-free  people would receive a second positive (compared to 98 of the 99 infected people who would get another positive, once again there would be one infected person who would get a negative result), so your chance of actually being infected is still (just) less than a half.

Although the numbers are very different, this is exactly the situation Stein is trying to process.  One need only replace "has a disease" with "forces others to have sex with them", and Stein's test "Are you a famous economist?"  It doesn't matter how accurate his test is, up until we know how many rapists walk the earth, and how many famous economists too, Stein is just a monkey flinging shit around; mainly at a woman who lacks a platform with which to defend herself.