(I actually haven’t read this book but the title of it is apposite)
This is another great post by Eliezer Yudkowsky over at Less Wrong:
I am teaching a class, and I write upon the blackboard three numbers: 2-4-6. “I am thinking of a rule,” I say, “which governs sequences of three numbers. The sequence 2-4-6, as it so happens, obeys this rule. Each of you will find, on your desk, a pile of index cards. Write down a sequence of three numbers on a card, and I’ll mark it “Yes” for fits the rule, or “No” for not fitting the rule. Then you can write down another set of three numbers and ask whether it fits again, and so on. When you’re confident that you know the rule, write down the rule on a card. You can test as many triplets as you like.”
Here’s the record of one student’s guesses:
4, 6, 2 No 4, 6, 8 Yes 10, 12, 14 Yes
At this point the student wrote down his guess at the rule. What do you think the rule is? Would you have wanted to test another triplet, and if so, what would it be? Take a moment to think before continuing.
The challenge above is based on a classic experiment due to Peter Wason, the 2-4-6 task. Although subjects given this task typically expressed high confidence in their guesses, only 21% of the subjects successfully guessed the experimenter’s real rule, and replications since then have continued to show success rates of around 20%.
The study was called “On the failure to eliminate hypotheses in a conceptual task” (Quarterly Journal of Experimental Psychology, 12: 129-140, 1960). Subjects who attempt the 2-4-6 task usually try to generate positive examples, rather than negative examples—they apply the hypothetical rule to generate a representative instance, and see if it is labeled “Yes”.
Thus, someone who forms the hypothesis “numbers increasing by two” will test the triplet 8-10-12, hear that it fits, and confidently announce the rule. Someone who forms the hypothesis X-2X-3X will test the triplet 3-6-9, discover that it fits, and then announce that rule.
In every case the actual rule is the same: the three numbers must be in ascending order.
But to discover this, you would have to generate triplets that shouldn’t fit, such as 20-23-26, and see if they are labeled “No”. Which people tend not to do, in this experiment. In some cases, subjects devise, “test”, and announce rules far more complicated than the actual answer.
Yudkowsky says that this is usually classified under the bias called Confirmation Bias, but thinks that it isn’t specific enough to describe this failure of reasoning. So he calls it the Positive Bias.
BUT… If one thought like a Bayesian, then one shouldn’t fall prey to not figuring out the pattern. That’s the ideal, anyway 😉 Referring to my previous post about what makes a good explanation, this Positive Bias is an example of not being precise enough. And what is the hallmark of precision, according to me? Finding out what sort of data a hypothesis excludes is more important than what it predicts. Any and every idea that you come up with to try to explain some phenomena should have some examples of evidence that it excludes.
Here’s the interesting thing. What if you’re a student in the experiment above, and you do start listing examples that you think break the rule… and nothing you posit breaks the rule? What if there aren’t examples that break the rule? What would you conclude about the pattern? There isn’t one! This is a good way of determining randomness; something that, say, people who believe in fate or god would never want to confront. The Positive Bias seems to be one of the ways the brain tricks itself into thinking there is order or a plan to random events. And if there’s a plan, then there must be some sort of agent behind that plan. Therefore god. Or the government (i.e. conspiracy theories).
Of course, to not fall prey to my own Positive Bias, I would like to conduct the above experiment on friends in real life 🙂 Maybe my atheist friends would be more likely to list patterns that they think break the rule and my theist friends wouldn’t. That would indicate a correlation.