The Watson Selection Task – a logical puzzle

The Watson Selection Task is a logical problem designed to show how bad we are at making logical decisions.  Watson first used it in 1968 – and found that only 10% of the population would get the correct answer.  Indeed around 65% of the population make the same error.  Here is the task:

The participants were given the following instructions:

Here is a rule: “every card that has a D on one side has a 3 on the other.” Your task is to select all those cards, but only those cards, which you would have to turn over in order to discover whether or not the rule has been violated.  Each card has a number on one side and a letter on the other. Give yourself a couple of minutes to work out what you think the answer is – and then highlight the space below where the answer is written in white text.

The correct answer is to pick the D card and the 7 card

This result is normally quite unexpected – but it highlights one of the logical fallacies that we often fall into:

A implies B does not mean that B implies A

All cats have 4 legs (Cats = A, legs = B, A implies B)
All 4 legged animals are cats (B implies A)

We can see that here we would make a logical error if we concluded that all 4 legged animals were cats.

In the logic puzzle we need to turn over only 2 cards, D and 7.  This is surprising because most people will also say that you need to turn over card with a 3.  First we need to be clear about what we are trying to do:  We want to find evidence that the rule we are given is false.

If we turn over the D and find a number other than 3, we have evidence that the rule is false – therefore we need to turn over D.

If we turn over the 7 and find a D on the other side, we have evidence that the rule is false – therefore we need to turn over the 7.

But what about the 3?  If we turn over the 3 and find a D then we have no evidence that the rule is false (which is what we are looking for).  If we turn over the 3 and find another letter then this also gives us no evidence that the rule is false.  After all our rule says that all Ds have 3s on the other side, but it doesn’t say that all 3s have Ds on the other side. Are mathematicians better at this puzzle than historians?

Given the importance of logical thought in mathematics, people have done studies to see if undergraduate students in maths perform better than humanities students on this task.  Here are the results: You can see that there is a significant difference between the groups.  Maths students correctly guessed the answer D7 29% of the time, but only 8% of history students did.  The maths university lecturers performed best – getting the answer right 43% of the time.

Making different mistakes You can also analyse the mistakes that students made- by only looking at the proportions of incorrect selections.  Here again are significant differences which show that the groups are thinking about the problem in different ways.  DK7 was chosen by around 1/5 of both maths students and lecturers, but by hardly any history students.

You can read about these results in much more depth in the following research paper Mathematicians and the Selection Task – where they also use Chi Squared testing for significance levels.