1Received October 20, 2004
2Revised May 24, 2005
3Accepted June 8, 2005
4On line July 11, 2005
5How do people deal with conflicts between their beliefs and observation? This is an important question, both for theoretical and practical reasons. These kinds of questions have received more attention in the field of Artifical Intelligence (e.g. Gärdenfors, 1988) and philosophy (e.g. Harman, 1986), but have been relatively neglected in psychology. However, there have been recent studies that have looked at belief revision within a larger framework of deductive reasoning (Elio and Pelletier, 1997, Politzer and Carles, 2001). These studies use the following basic paradigm. A reasoner is presented with a conditional premise (If P then Q), which is described as being true. Then, P is presented as being true, which induces the conclusion that “Q must be true”. However, following this, it is stated that “Q is false”. How is this contradiction resolved? There are two main strategies for doing so, the first involves revising belief in the conditional premise by asserting that P does not necessarily lead to Q, the second involves putting into doubt the minor premise that “P is true”. Initial studies indicate that, when faced with this kind of contradiction, a majority of adults tend to revise the conditional premise (Elio, 1997, Elio and Pelletier, 1997, Dieussaert, Schaeken, De Neys and d’Ydewalle, 2000, Politzer and Carles, 2001, Revlin, Cate and Rouss, 2001) with both familiar and symbolic contents. Such results seem to be explained by a recent theory (Legrenzi, Girotto and Johnson-Laird, 2003, Johnson-Laird, Girotto and Legrenzi, 2004). However, with conditional premises that refer to familiar content, the tendency of revisions is modified by the extent to which people believe that the conditional relation is true. Politzer and Carles (2001) found that the tendency to revise the major premise increases when the premise is judged to be less certain. In other words, the tendency to abandon a hypothetical relation depends on the level of belief spontaneously accorded to the relation.
6In these studies, the belief relation is presented to participants who are told to consider it to be true. This does mirror some real-life situations in which relational beliefs of the kind “If P is done, then Q will happen” are presented as being true by outside authority, which may then be more or less supported by an individual’s own beliefs. However, in many cases people will construct such beliefs on the basis of their personal experience. One particularly salient example of this is the case of a researcher who arrives at a causal hypothesis after conducting a series of experiments that provide support for this hypothesis. As we well know, there does not seem to be a strong tendency for researchers to abandon their own hypotheses when faced with reported evidence contradicting this belief. There do not appear to be any studies that have looked at this, but there are good reasons to expect that self-construction of beliefs would produce more resistant beliefs, if only because individuals should be more ready to trust to their own experience rather than accepting another’s evaluation.
7The basic question that we wish to examine in this study is the impact of self-construction of a relational belief on belief revision. Our basic hypothesis is that self-constructed beliefs should be more resistant to revision than beliefs that are simply presented as true. Even if this hypothesis may appear obvious, it has not been tested to our knowledge. We also examined another factor related to the effect of direct experience with a relational belief. We specifically looked at the relation between the amount of experience that an individual requires in order to establish a relational belief and its potential revision. We did not have a definite hypothesis here.
8In order to look at this hypothesis in a relatively controlled environment in which subjects could actively construct the conditional relation, we created a simple computer system which presented 5 boxes in a row on the top of a screen, with 5 boxes below them in one-to-one correspondence with the top boxes. The middle boxes in both the top and bottom rows were clearly labeled as being of particular interest (the middle box on the top row was labeled box A, and the middle box on the bottom row was labeled box B, while none of the other boxes had any label at all). Using a mouse, participants could click on any of the top boxes and one of the bottom boxes would turn red. In the case of boxes 2, 3, 4, and 5 (from the left), clicking on the top box would always result in the corresponding bottom box turning red (i.e. the bottom box 2 would always turn red when the top box 2 is clicked, the bottom box 3 would always turn red when the top box 3 is clicked, and so on). In order to provide some variability in the system, clicking on the first top box resulted in systematic cycling between lower box 1 and lower box 3 turning red (i.e. if lower box 1 turned red when the top box 1 was clicked, then the next time top box 1 was being clicked lower box 3 would turn red, and vice versa). For each participant, the computer recorded the number of times that each box was clicked. A control condition was used in which participants were simply told that it was always true that if box A was clicked on, then box B would turn red (in this condition, subjects did not manipulate the computer). Our main hypothesis was that the tendency to revise the if-then conditional would be greatest in the control condition.
9A total of 60 undergraduate students from the University of Quebec in Montreal(30 males, 30 females; average age = 28 years, 2 months) were randomly assigned to one of three groups.
10A computer program was created using Microsoft Visual Basic. On the screen were two rows of 5 grey boxes placed in one-to-one correspondence, with one row higher than the other. The middle box in the top row was labeled box A, while the middle box in the bottom row was labeled box B. None of the other boxes were labeled. A computer mouse could be used to click on any of the top boxes, without restriction. Clicking on the first box to the left of the top row initially resulted in the first box to the left of the bottom row turning red. The next time this box was clicked on, the third box of the bottom row turned red, with subsequent tries systematically cycling between the first and third boxes. Clicking on any of the four remaining boxes on the top row always resulted in the corresponding box on the bottom row running red. Lower boxes would turn red for approximately 1 second, after which the display reverted to original form (see Figure 1). The program recorded the number of times that each box was clicked on by the participants.
Figure 1 : Computer program used in the experiment : by clicking on top boxes 2, 3 (box A), 4 and 5 (from the left) corresponding bottom boxes turn red, and by clicking on the top box 1 (from the left), bottom box 1 or 3 (box B) turn red.
11All participants were interviewed individually. They were shown the computer screen with the rows of boxes. In the control condition they were told that it was always the case that if box A was clicked on, then box B would turn red, while in the two experimental conditions, they had to test the computer program in order to discover by themselves the relation between the boxes of the first row and those of the second row.In these experimental conditions,participants were told to test the program until they felt certain that they understood how it worked (in particular, the one-to-one correspondence between A and B). At this point, the participants in the three groups were asked to indicate what would happen if box A was clicked on and to rate the certainty of their response. The first experimental group was given a certainty rating scale from 0 (“not at all certain”) to 10 (“completely certain”), for this question. Results of this group showed minimal variation in certainty levels. Thus, a second experimental group was asked how much they would bet that if box A was clicked on, that box B would turn red. Participants in the three groups were then told that a third (unknown) person using the same computer program had clicked on box A on the previous day, but that this time the first box to the left had turned red (this part corresponds to the introduction of a conflict between their belief and their observation). They were then asked how they would account for this by giving at least one explanation (involving a revision of their belief).
12Belief revision responses were of three types. The first type consisted of putting into doubt the major (if-then) premises (i.e. by stating that clicking on box A did not always make box B turn red). The second type consisted of putting into doubt the minor premises (i.e. by stating that it was not true that someone had clicked on box A without box B turning red) . A third response put into doubt both the major and minor premises. We first looked at the percentage of participants in each of the three conditions who were ready to put into doubt the basic if-then conditional either by itself or in combination with the minor premise. The percentage of participants who responded in this way was 40% for the first experimental group, 50% for the second experimental group and 80% for the control group. We combined the two experimental groups and performed a Chisquared analysis in order to compare conditional revisions in the control and experimental groups. This showed that there was a significantly larger percentage of revisions of the conditional in the control condition, X2(1) = 6.65, p < .001. In fact, the difference between the control group and both the first experimental group, X2(1) = 3.96, p < .05, and the second experimental group, X2(1) = 6.67, p < .001, were individually significant.
13Our initial hypothesis was thus verified. We then looked at the relationship between the number of times that participants in the experimental group tested each of the 5 boxes, their degree of certainty that clicking on box A would turn box B red, and their tendency to subsequently revise this conditional belief.
14We first looked at ratings of certainty on a 0 to 10 scale given by participants in the first experimental group. Participants in this condition were instructed to test the mechanism until they were sure how it worked, thus we did not expect much variation on this dimension. In fact, 17 of 20 participants in this group gave a rating of 10, 1 gave a rating of 9 and 2 gave a rating of 8. A Wilcoxon signed-ranks test showed that the mean rating was not significantly different from 10. Although there was a positive correlation between the number of tries of box A and certainty ratings, the low level of variation in certainty ratings (coupled with further analyses that showed that the effects of certainty levels disappeared when number of tries was partialed out) showed that certainty levels were not useful indices of belief revision. In the second experimental group, participants were asked to indicate how much they would bet that clicking on box A would make box B turn red, in the hope that this might provide a more nuanced measure of certainty. However, there were no significant correlations between bets and number of tries of box A or between bets and belief revision. It is probable that variation in bets simply indicated individual differences in risk taking rather than measurable differences in certainty. Given this, we eliminated certainty ratings from the rest of the analysis.
15We then looked at the relationship between the number of times each of the 5 boxes were tested and the tendency to revise the conditional belief. Table 1 gives the correlations between number of trials on each of the 5 boxes and revision of the conditional belief. As would be expected there are strong correlations among number of trials for each of the 5 boxes. There is also a significant correlation between trials of box A and belief revision (-.36). That is, the more times participants tried box A, the weaker was the tendency to reject the conditional subsequently. However, even more striking is the fact that the correlation between trials of box 1 (the first box on the left and the only box for which there was any variation in result) and belief revision is also highly significant (-.51), contrary to what was found with the remaining 3 boxes.
Table 1. Correlations between number of trials (clicks) of each of the five boxes and tendency to revise the conditional.
16Examination of the patterns of correlations among the 5 boxes does indicate that the correlations between box 1 and the other four boxes are generally lower than those among the latter. A principle components analysis of these correlations does indeed show two factors. The first loads strongly onto the four boxes to the right of box 1. The second loads strongly onto box 1, with smaller contributions of boxes 2, and A. This clearly suggests that while there is a general effect of the number of tries of all the boxes, which reflects the overall experience with the apparatus, there is something specific about how box 1 is approached. We then looked at the way that trials on box A and box 1 were specifically related to belief revision, by first examining partial correlations. The correlation between trials on box 1 and belief revision, with trials on box A partialed out, was -.39, while the correlation between trials on box A and belief revision, with box 1 partialed out was -.07. This clearly shows that investigation of box 1 contributes more to the nature of belief revision than does investigation of box A, when their shared variance is taken out.
17These results show that there are two factors that have an impact on belief revision in this situation. The first is some measure of overall experience with the conditional relation which involves testing out both box A and box 1. The second is the specific contribution of testing out box 1 over and above this common level. We can roughly distinguish two models of how this latter factor might play out. One model is that the absolute number of trials of box 1 determines the effect of belief revision, in which case the shared variance would simply reflect the fact that there is a correlation among numbers of trials for all the boxes, including box A. Another model is that box A trials do contribute to belief revision, and that the relative proportion of box 1 trials to box A trials also contributes to belief revision. We constructed a ratio score which consisted of box 1 trials divided by box A trials. We then performed two stepwise regressions on the belief revision scores. In the first, we simply included trials on box A, box 1 and the ratio score, and allowed the program to chose which variables were entered. Only trials on box 1 were significant, and this accounted for 25.7% of the variance. In the second, we forced trials on box A as the first factor. After this, only the ratio score was significant, and this alone accounted for 27.0% of the variance. The second model consisting of trials on box A and the ratio score accounted for 39.7% of the variance, much higher than the model consisting of box 1 alone. This analysis supports the idea that it is not just the absolute level of trials of box 1 that explains the effect of belief revision, rather it is the joint contribution of trials on box A and the relative number of box 1 trials compared to the number of box A trials that determines this effect.
18In this study, we compared belief revision in two basic situations using a computer generated system that was completely unknown to the participants. The first involved simply telling them that a given causal relation (“if box A is clicked then box B turns red”) was true. In the second, participants were instructed to test the system until they were certain of its functioning. The results of the first experimental group clearly show that participants did just that, since they uniformly claimed to have a very high degree of certainty that “if A then B” was true.
19The results of this manipulation clearly show that when people are faced with a conflict between a conditional belief and third-person observation, they are more ready to abandon a belief that has been communicated by authority than one that has been self-constructed from personal experience. While the situation employed in this study used an artificial environment, the co-existence of a multitude of theories with disconfirming evidence generated by researchers other than the originators of these theories in fields such as psychology does support the idea that the results of this study can be reasonably generalized to more ecologically valid contexts.
20More interesting in many respects are results that show that belief revision is affected by the quantity of direct experience with the artificial environment used here in two different ways. Before looking at this, it is worth repeating that participants received direct instructions to test this environment until they were satisfied that they understood how it worked. In addition, the (“if A then B”) conditional relation that was used in the belief revision task was always true, and thus generated no direct disconfirming evidence. The results of the first experimental group clearly show that this generated very high levels of subjective certainty that clicking on box A leads to box B turning red. Thus, the clear effects of experience cannot be attributed to variation in subjective certainty. In other words, we can fairly conclude that, in accordance with the instructions, the participants in the experimental conditions ended their trials with a firm conviction that “if box A is clicked then box B will turn red”. Nonetheless, despite these very high levels of certainty, there are strong effects of quantity of evidence obtained by participants on the tendency to conserve the relational belief when faced with a conflict between this belief and third-party observation.
21How does direct evidence affect belief revision? There is first an initial effect of the number of times that box A was directly tested. The more trials of box A that were used to arrive at a high degree of certainty, the less tendency there was to subsequently reject the conditional belief. It should be noted, to make this point even clearer, that the correlation between trials of box A and belief revision among the 17 participants who gave the maximal certainty rating (10) that the conditional belief was true, r(17) = -.47, p = .06, was actually higher than that observed with the total experimental sample, r(40) = -.36, p < .05, although the smaller number of participants makes the former correlation only marginally significant. Thus, in some manner, additional experience with the conditional relation makes this more resistant to subsequent modification in a way that cannot be explained by subjective certainty about the truth of the relation. It is of course possible that some other method of measuring subjective certainty might provide a link between experience and belief revision, but our attempt using a betting paradigm failed completely in doing so.
22The system that we presented to our participants also included one variable element. Most strikingly, the strongest single predictor of whether the conditional relation (“if A then B”) would be modified subsequently or not, is the number of times that this variable element was tested. Our analyses suggest that participants who examined box 1 (which cycled systematically between the first and third places) relatively more often than they examined box A tended to subsequently deny the conditional belief less often.
23Thus, in the particular universe defined by the simple computer system used here, belief revision is determined by two factors, (1) direct experience with the conditional belief, and (2) experience with the sources of variability in the system. Each of these contributes separately to the tendency to choose to revise a conditional belief in the face of potentially disconfirming evidence, and appears to act in a way that is not captured by explicit evaluations of certainty.
24This suggests that at some level certainty is a multi-layered construct, involving both an evaluation of the consistency of a specific belief and an evaluation of the consistency of the system in which the belief is embedded. In other words, the “epistemological status” of a belief is determined not only by direct experience with the specific belief, but also by some systemic certainty that provides a level of implicit certainty that can be stronger than direct certainty of the belief in question.
25Preparation of this manuscript was supported by a grant from the Natural Sciences and Engineering Research Council of Canada (CRSNG) to the second author.