ABSTRACT
Creating and testing hypotheses represents a crucial feature not only of progress in science, but also in our daily lives in which we set up assumptions about reality and try to test them. However, the lay scientist stands accused of processing his or her hypotheses in such a way that he or she is biased to confirm them. “Confirmation bias” means that information is searched for, interpreted, and remembered in such a way that it systematically impedes the possibility that the hypothesis could be rejected – that is, it fosters the immunity of the hypothesis. Here, the issue is not the use of deceptive strategies to fake data, but forms of information processing that take place more or less unintentionally. In this chapter we are going to study how far the accusation of a confirmation bias in hypothesis testing is justified. But let us first try to solve two problems:
First, assume somebody presents you with the following task:
I made up a rule for the construction of sequences of numbers. For instance, the three numbers “2-4-6” satisfy this rule. To find out what the rule is, you may construct other sets of three numbers to test your assumption about the rule I have in mind. I gave you one set of three already, and for every three numbers you come up with, I will give you feedback as to whether it satisfies my rule or not. If you are sure you have the solution, you may stop testing, and tell me what you believe the rule to be.
