Humans are notoriously inconsistent reasoners. When making inferences and judgments they show both an ability to produce “logical” answers along with a well-documented tendency to use heuristic shortcuts, such as the belief bias effect (Evans, Barston, & Pollard, 1983; Markovits & Nantel, 1989). The prevailing explanation for such inconsistency postulates that people have access to two competing inferential systems. These models are known as dual process models (De Neys & Glumicic, 2008; Epstein, 1994; Evans & Stanovich, 2013; Kahneman, 2011; Sloman, 1996; Stanovich, West, Ackerman, Kyllonen, & Roberts, 1999). Although there are many different instantiations of this model, most suppose a distinction between processing systems that give rise to relatively automatic responses (Type 1) and those that are more effortful (Type 2; Evans & Stanovich, 2013). Type 1 responses tend to be fast, and thus form an early, default response, whereas answers requiring more cognitive capacity tend to be slower. This asymmetry in speed underlies the dual-process theory explanation for many reasoning phenomena including belief-bias (Thompson, Pennycook, Trippas, & Evans, 2018): Reasoners default to an answer based on experiential, belief-based information, which is generally responsible for nonlogical responses and may not engage the slower working memory intensive system. Thus, under time pressure, reasoners are more likely to respond based on conclusion believability (Evans & Curtis-Holmes, 2005). These models suppose that human reasoning is the product of an internal interaction between these two systems.

One of the critical questions related to the variability in human reasoning concerns the factors that determine individual differences in reasoning. One such factor that is suggested by dual process theories is cognitive capacity. Increased cognitive capacity allows both better Type 2 reasoning and increased ability to inhibit Type 1 responses. However, another, different, source of individual differences is provided by the dual strategy model of reasoning (Markovits, Brisson, & de Chantal, 2015, 2017a; Markovits, Brunet, Thompson, & Brisson, 2013; Markovits, Lortie Forgues, & Brunet, 2012; Verschueren, Schaeken, & d’Ydewalle, 2005). This model postulates the existence of two forms of reasoning strategy, which mirror a basic debate in the reasoning literature. Briefly, mental model theories suggest that reasoning involves a process that generates a representation of premises which can then be scanned for potential counterexamples to a putative conclusion (Johnson-Laird & Byrne, 2002). Probabilistic theories suggest that inferences are made based on an estimation of the likelihood of a putative conclusion, given the premises (Evans, Over, & Handley, 2004; Oaksford & Chater, 2003). Each of these theories claim to provide exclusive and incompatible models of reasoning. By contrast, the dual strategy model based on an original idea by Verschueren et al. (2005) suggests that these basic processes are not exclusive, but are instantiated in two different strategies. The statistical strategy produces rapid likelihood estimations based on associative access to probabilistic information. The counterexample strategy is more focused and requires the generation of a representation of key elements of the premises with a corresponding search for potential counterexamples.

Previous studies have developed a series of simple deductive reasoning problems that have been used to characterize people as primarily counterexample or statistical reasoners. This uses a series of deductive reasoning problems that correspond in their logical form to the affirmation of the consequent inference (P implies Q. Q is true), coupled with statistical information that suggests that the putative conclusion (P is true) is highly probable or not (while showing very low or higher levels of counterexamples to the conclusion). People who consistently deny the putative conclusion irrespective of the statistical information are classed as counterexample reasoners, while those who accept the putative conclusion more often for the highly probable than the less probable problems (in a way that reflects the relative strength of these conclusion) are classed as statistical reasoners (between 65% and 75% of reasoners can be classed in one of these two categories). An initial study using this diagnostic instrument has shown that strategy use can sometimes change in response to factors such as time pressure and appears to be under some degree of metacognitive control (Markovits, Brunet, Thompson, & Brisson, 2013). Studies have also shown that people who are categorized as either statistical or counterexample continue to reason in ways that are consistent with this categorization. More specifically, people classed as statistical reasoners continue to be particularly sensitive to the underlying probabilistic structure in other forms of reasoning problems, while counterexample reasoners are more influenced by the presence of potential counterexamples (Markovits, Brisson, & de Chantal, 2017a; Markovits, Brisson, de Chantal, & Singmann, 2018a).

Critically, although the use of a counterexample strategy often leads to greater production of the classically “logical” response, in some cases this same strategy leads to greater levels of “nonlogical” responding than a statistical strategy would (Markovits, Brisson, & de Chantal, 2017a). In addition, a recent eye-tracking study has shown that when reasoning with syllogisms for which the logical validity of putative conclusions is in contradiction with their empirical truth, counterexample reasoners attend more consistently to problem premises and less to the conclusion than do statistical reasoners (de Chantal, Newman, Thompson & Markovits, 2019). These results suggest that a critical difference between these two strategies is more in the way that information is processed than in any form of logical capacity. More specifically, the counterexample strategy produces more focused reasoning that processes a smaller quantity of information that is more directly related to key aspects of an inferential problem, while a statistical strategy uses fewer intensive processes that treat a wider range of information with less attention paid to each. This, in turn, suggests that strategy use should be related to individual differences in a wider variety of judgments, which do not involve logical validity. Further studies have indeed shown that strategy use is related to individual differences in the processing of emotional stimuli (Markovits, Trémolière, & Blanchette, 2018b), accuracy of processing three-dimensional mental rotations (Markovits, 2019), and susceptibility to a variety of social biases (Gagnon-St-Pierre, Doucerin, & Markovits, 2020).

In the following, we examined the relationship between reasoning strategies and so-called logical intuitions, which comes from some of the more intriguing recent results in reasoning. These results suggest that there exists a form of very rapid reasoning that can produce the logically correct response to certain limited forms of reasoning problems (De Neys, 2012; Handley, Newstead, & Trippas, 2011; Morsanyi & Handley, 2012). For example, when people produce belief-based responses that are in conflict with the rule-based response, a variety of indicators, such as galvanic skin responses, response confidence, and response latency, indicate that they are nonetheless sensitive to the conflict (De Neys, 2012; De Neys, Vartanian, & Goel, 2008; Simon, Lubin, Houdé, & De Neys, 2015). In other words, it appears that information about the validity of the conclusion is available quickly enough to interfere with conclusion believability (Handley et al., 2011), leading to the hypothesis about logical intuitions.

The existence of such logical heuristics suggests that even when deprived of cognitive resources, at least some people should be capable of responding logically. One of the clearest ways of doing this is to restrict the time allotted for making inferences. Recent studies (Bago & De Neys, 2017; Newman, Gibb, & Thompson, 2017) have indeed shown that at least some reasoners are able to produce very rapid responses that reflect logical (rule-based) parameters.

These results, along with those previously cited, provide strong confirmation for the idea that at least some adults have access to a form of rapid responding that allows producing “logical” responses, at least to a limited subset of inferences. One of the important questions that these results raise is the question of what kinds of individual differences are related to such reasoning. Indeed, a recent study by Frey, Johnson, and De Neys (2017) has found that there may be a subset of reasoners who are less able to make intuitive judgements that are consistent with logical criteria, which clearly suggests the existence of individual differences.

A major question, then, is what could characterize these differences? Our first hypothesis is that strategy use is indeed one such individual difference. As previously discussed, there is evidence to suggest that a key distinction between the two strategies is the way that they differentially attend to the premises and the conclusion (de Chantal et al., 2019). Use of a rapid form of rule-based heuristic clearly depends on being able to preferentially attend to the premises of an inference to generate a conclusion based solely on these, rather than to a potentially conflictual conclusion. Under time pressure, such attentional differences should be more important. Since counterexample reasoners preferentially do this, it would be expected that they would show a higher level of very rapid logical responding than statistical reasoners. In fact, a recent study that examined mental rotation found that strategy use was a more important factor under time constraints (Markovits, 2019), consistent with the analysis. Thus, in the following studies, we specifically examine the effect of severe time constraints on rule-based reasoning. Our general hypothesis is then that counterexample reasoners will produce higher levels of logical responses under extreme time constraints than will statistical reasoners.

Study 1

Our first test of this prediction used two kinds of inference. One of these were adapted from a previous study (Markovits, de Chantal, Brisson, & Gagnon-St-Pierre, 2019). These involved two forms of conditional inferences—the logically valid modus ponens inference (MP: if P then Q, P is true) and the logically invalid affirmation of the consequent inference (AC: if P then Q, Q is true), along with two equivalent forms of syllogistic inferences that follow the same logical pattern (all A are X, X is B, A is B) and (all A are X. B is X, A is B). For convenience, we refer to these as AA (affirmation of the antecedent) and AC (affirmation of the consequent) inferences. In addition to the AA and AC inferences, we included another form of logical judgment that has not, to our knowledge been examined in adults. This is the recognition that premises might simply not be logically related to a putative conclusion (i.e., they are non sequiturs). Thus, the premises “A implies B. B implies C” are unrelated to the putative conclusion that “A implies D.” Ideally, reasoners should judge that this conclusion does not follow from the premises. Studies with very young children have shown that this kind of judgment often leads to acceptance of the conclusion if both premises and conclusions have the same valence, with there being a clear developmental increase in the recognition of the incompatibility of the two (Markovits, Schleifer, & Fortier, 1989). A more recent study has suggested that the way that adults respond to simple inferences when severely time constrained resembles responses of much younger reasoners who have unlimited time (Markovits et al., 2019), meaning that we might expect time pressure to increase acceptance of these conclusions. We thus constructed a series of inferences where the conclusion and the premises were not logically related. These were constructed so that the conclusion was an empirical consequence of the minor premise, but did not follow logically from the major and minor premises—for example, “All birds can fly. Butterflies have wings. Butterflies can fly.”

We presented inferences with two different time constraints; a first group received 5 s, while the second group received 3.5 s for each inference. The 5-s condition was chosen to allow sufficient time to process problem premises, with the possibility of deploying both intuitive and possibly more complex processes to make inferences. Pretests indicated that 3.5 s was closer to the limits of participants’ capacity to adequately process problem premises. The pattern of differences between the two time delays provides additional evidence for the existence of a consistent individual difference primarily related to intuitive responding. Participants also received the strategy assessment task which is used to categorize reasoning strategy. We specifically hypothesized that (a) the overall level of logical responding would decrease as the time constraint becomes more severe, since the longer time might allow higher probabilities of all participants being able to attend to problem premises, while the shorter time would make it less likely that participants would be able to attend to premises, and (b) that counterexample reasoners would show higher levels of logical reasoning at both time constraints, since they should be able to better attend to problem premises than statistical reasoners irrespective of time constraints.

Method

Participants

A total of 206 participants recruited from an online participant pool (124 females, 82 males; average age = 25 years, 1 month) took part in this experiment. Participants were each paid 2 pounds. Based on our prior work, we estimated that this would give us a sample of around 150 participants who could be classified as either statistical or counterexample reasoners, and thus give use adequate (.8) power to detect small to medium effects of ηp2 = .05 (computations based on MorePower; Campbell & Thompson, 2012).

Materials and procedure

Participants were first given a brief explanation of the inference tasks. They were told that they would be given three statements, of which the last was a conclusion made on the basis of the first two, and that their task was to indicate whether the conclusion was logical or not. Participants were given an example of a valid conclusion using a transitive relationship. They were then given an example of an invalid conclusion also based on transitive relationships. They were then told that they would be given a brief time to respond either yes or no. They were also told that an indicator would turn red when they would have only 2 seconds to respond. In one condition, participants were given 5 seconds to respond, while in the other condition they were given 3.5 seconds. Participants were given a practice phase to get familiar with the time limit.

They were subsequently given 15 inferences, in a random order. These consisted of 5 inferences of the form AA, 5 inferences of the form AC, and 5 nonlogical inferences. All the inferences used the same set of major premises (see Table 1 for details).

Table 1 Set of inferences used in Studies 1 and 2
Full size table

Following the time-limited inferences, participants were presented with a 13 problem strategy assessment, identical to that used in previous studies (see Markovits et al., 2013, for details). Each problem described a causal conditional relation involving a nonsense terms or relations that included frequency information concerning the relative numbers of not-p.q and p.q cases out of 1,000 observations. Participants were then given an inference corresponding to the affirmation of the consequent inference (P implies Q, Q is true. Conclusion: P is true) and were asked to indicate whether the conclusion could be logically drawn from the premises or not. Of the 13 items, five had a relative frequency of alternative antecedents that was close to 10% (each individual item varied between 8% and 10%), five had a relative frequency that was close to 50% (each individual item varied between 48% and 50%), and three had a relative frequency of alternative antecedents that was presented as 0% (these last were presented in order to provide greater variability in problem types). The following is an example:

“A team of geologists on Kronus have discovered a variety of stone that is very interesting, called a Trolyte. They affirm that on Kronus, if a Trolyte is heated, then it will give off Philoben gas.

Of the 1,000 last times that they have observed Trolytes, the geologists made the following observations:

910 times Philoben gas has been given off, and the Trolyte was heated.

90 times Philoben gas has been given off, and the Trolyte was not heated

From this information, Jean reasoned in the following manner:

The geologists have affirmed that: If a Trolyte is heated, then it will give off Philoben gas.

Observation: A Trolyte has given off Philoben gas.

Conclusion: The Trolyte was heated.”

Participants were presented with two buttons, one which stated that the conclusion was invalid, while the other stated that the conclusion was valid. They were asked to click on the appropriate button when they were certain of their response. The key index here is the number of times (out of a total of five) that the conclusions to the 50% problems were accepted as valid, and the comparative number of times (out of 5) that the 90% conclusions were accepted as valid.

Results and discussion

We initially examined the proportion of missing responses. There were 0% missing at 5 seconds, and 1.6% missing at 3.5 seconds. Participants were categorized into reasoning strategies using the same criteria as in previous studies. Those who rejected all the 50% and 90% conclusions were put into the counterexample category. Those whose rejection rate for the 50% conclusions was at least 2 more than the rejection rate for the 90% conclusions were put into the statistical category. All other were put into an “other” category (which previous results indicate consist of people with changing strategies). We then categorized participants into reasoning strategy for each of the time limits (3.5 s: 44 counterexample, 32 statistical, 25 other; 5 s: 41 counterexample, 30 statistical, 34 other). We then calculated the proportion of logically correct responses for the AA, the AC, and the nonlogical inferences as a function of time and reasoning strategy (see Fig. 1).

Fig. 1
figure 1

Proportion of logically correct responses to the AA (affirmation of antecedent), AC (affirmation of consequent), and nonlogical inferences as a function of time (5s, 3.5s) and strategy (counterexample, statistical) in Study 1

Full size image

We initially examined performance on the nonlogical inferences. We performed an analysis of variance (ANOVA), with the proportion of logically correct responses (with missing responses excluded) as a dependent variable, and time (3.5s, 5s.) and strategy as the independent variables. This gave significant main effects of time, F(1, 143) = 9.10, p < .003, ηp2 = .060, and Strategy, F(1, 143) = 21.67, p < .001, ηp2 = .132. Overall, fewer nonlogical inferences were rejected at 3.5 s (M = .66, SD = .36) than at 5 s (M = .82, SD = .32). In addition, fewer nonlogical inferences were rejected by statistical reasoners (M = .60, SD = .36) than by counterexample reasoners (M = .84, SD = .27). The Time × Strategy interaction was not significant, F < 1.

We then examined whether responses to the nonlogical inferences were rejected at greater than chance rates. For counterexample reasoners both at 5 s, (M = .92, SD = .19), t(40) = 13.90, p < .0001, and 3.5s (M = .76, SD = .30), t(43) = 5.66, p < .0001, rejection rates were significantly greater than expected by chance. However, for statistical reasoners this was only true at 5 s (M = .66, SD = .35), t(53) = 3.24, p < .002; at 3.5-s, rejection rates were at chance levels (M = .503, SD = .38), t(44) < 1.

We then examined performance on the AA and AC forms. We performed an ANOVA, with proportion of logically correct responses as a dependent variable and logical form (AA, AC) as a repeated measure, and time (3.5 s, 5 s) and strategy as the independent variables. The findings replicated those of Study 1. There were significant main effects of Form, F(1, 142) = 291.47, p < .001, ηp2 = .672; strategy, F(1, 142) = 38.11, p < .001, ηp2 = .212; and time, F(1, 142) = 4.59, p = .034, ηp2 = .031. Proportion of logically correct responses were higher for the AA inferences (M = .94, SD = .12) than for the AC inferences (M = .45, SD = 27). The proportion of correct inferences was higher at 5 s (M = .72, SD = .20) than at 3.5 s (M = .65, SD = .19). Finally, counterexample reasoners had a higher rate of correct responses (M = .77, SD = .19) than did statistical reasoners (M = .61, SD = .18).

There was, in addition, a significant Form × Strategy interaction, F(1, 142) = 22. 5.27, p < .001, ηp2 = .151. Analysis of the Form × Strategy interaction showed that counterexample reasoners correctly rejected the AC inferences (M = .59, SD = .36) more often than did statistical reasoners (M = .31, SD = .33), with no difference on the AA inferences.

These results provide additional confirmation of the utility of the dual strategy model in predicting individual differences in very rapid responding. They show that there is a difference in the ability of reasoners to distinguish between AA and AC inferences, one that is modulated by reasoning strategy. In addition, they provide novel and clear evidence that at least some people can detect conclusions that are inconsistent with the premises even when they are close to the limits at which the information can be processed. Counterexample reasoners are uniformly better able to detect logical inconsistencies than are statistical reasoners at both time limits used in the study. In fact, when given 3.5 s, statistical reasoners are unable to reliably identify nonlogical inferences, while counterexample reasoners are still able to do this.

Study 2

These results clearly show that people’s ability to make rapid logical judgements is related to reasoning strategy, with counterexample reasoners having a significantly greater ability to make such judgements than statistical reasoners. While these studies are convincing, a further important question is the relation between strategy use and cognitive capacity as individual difference predictors. This is particularly interesting since the standard dual process explanation considers that cognitive capacity is critical for the use of more demanding Type 2 processing, but also suggests that this is not the case for heuristic responding. However, the extremely short processing times that have been examined in the two previous studies, which are close to the limits of participants ability to simply process information, likely do not allow for Type 2 processing. The existence of some degree of logical responding in these studies is thus consistent with the logical intuition hypothesis (e.g., De Neys, 2012), which would in turn suggest no relationship between rapid logical responding and cognitive capacity. However, Thompson et al. (2018) have argued that high-capacity reasoners would be more likely to have automated simple logical relations or to be more likely to represent argument premises in a way that allows them to make fast inferences. They did indeed find that high capacity individuals were most likely to display evidence of logical intuitions.

These results clearly suggest the importance of considering potential interactions between cognitive capacity and reasoning strategies. There is clear evidence that strategy use is correlated with measures of cognitive capacity, both with working memory (de Chantal et al., 2019) and with fluid intelligence (Thompson & Markovits, 2020). This could imply that the observed effects of strategy use might simply be due to the shared variance with cognitive capacity. However, recent results by de Chantal et al. (2019) have shown that logical responding on belief-bias tasks is explained by separate contributions of both working memory and strategy use. This shows that, despite the correlation between strategy use and working memory, they contribute differently to people’s ability to produce logical responses when there is conflict with a nonlogical heuristic response. Similar dissociations between IQ and strategy use have been shown with differing heuristics with unlimited time (Thompson & Markovits, 2020).

In the following study, we examine the relation between very fast and unlimited time responding on the tasks used in Study 1 and measures of working memory and strategy use. Given the dissociations previously mentioned, our working hypothesis for this study was that strategy use would explain differences in the production of very fast logical responses over and above any effects of working memory.

Method

Participants

A total of 291 participants recruited from an online participant pool (174 females, 117 males; average age = 33 years, 10 months) took part in this experiment. Of these, 149 were in the limited time condition, while 142 were in the unlimited time condition. Participants were each paid 2 pounds.

Material and procedure

Each participant received the same instructions as those used in Study 2. They were first given the same set of 15 inferences for which they were given either 4 s to respond or unlimited time to respond. The 4-s time limit was chosen to be highly restrictive, but to have a somewhat lower level of missing responses (although given the limited accuracy of online studies, it was probably functionally similar to the shortest interval in the previous studies).

Following this, participants were given a working memory task based on the Baddeley, Logie, Nimmo-Smith, and Brereton (1985) adaptation of the measure proposed by Daneman and Carpenter (1980). Participants were given a series of sentences each of which involved a person, a verb, and an object. The participants were presented with a sequence of either three or four sentences and were then cued to recall either the person or the object described in each of the sentences. Half of the sentences made sense; the other half did not. To ensure comprehension, the subjects were asked to rate each sentence as making sense or not, immediately after presentation. The test involved four sequences of two sentences (which were used for training and were not scored) and four sequences of three sentences, followed by four sequences of four sentences. The total number of items recalled in correct serial order for the sequences with three and four sentences was then calculated. Finally, participants were given the identical strategy diagnostic used in the previous studies.

Results and discussion

Overall, in the 4-s condition, participants were unable to give responses to 15.2% of problems. This indicated that this time constraint remained close to the limits of people’s ability to process the information in these problems. As before, participants were classed as either counterexample (4 s: N = 57, unlimited: N = 65), statistical (4s: N = 65, unlimited: N = 54) or Other (4 s: N = 27, unlimited: N = 23). Also, as before, participants classed as Other were excluded from further analysis. There was a marginally significant correlation between strategy use and working memory, r = .12, p < .07, with counterexample reasoners having higher working memory scores (M = 10.5, SD = 5.12) than statistical reasoners (M = 9.41, SD = 4.36).

We first examined the effects of time restriction on the three forms of reasoning. We calculated proportion of correct responses (eliminating any nonresponses) to the AA, AC and nonlogical forms for the two conditions (see Fig. 2). We then performed an ANOVA, with proportion of correct responses as a dependent variable and form (AA, AC, nonlogical) as a repeated measure, and condition (4 s, unlimited) as an independent variable. This gave main effects of condition, F(1, 289) = 119.47, p < .0001, and form, F(2, 288) = 158.05, p < .0001, and a significant Condition × Form interaction, F (1, 288) = 15.75, p < .0001. Post hoc tests indicated that the proportion of correct responses was lower in the 4-s condition compared with the unlimited condition for all three forms. Analysis of the interaction showed that the effect of time was significant only for the AC and nonlogical inferences, consistent with the results of Study 1.

Fig. 2
figure 2

Proportion of logically correct responses to the AA(affirmation of antecedent), AC (affirmation of consequent), and nonlogical inferences as a function of time (unlimited, 4s) and strategy (counterexample, statistical) in Study 2

Full size image

We then specifically examined the effects of working memory and strategy use on reasoning with the nonlogical forms. We performed a regression with number of logically correct responses to the nonlogical items as a dependent variable and strategy, condition (4 s, unlimited), working memory, and number of logically correct responses to the AA items and interaction terms involving Condition × Strategy and Condition × Working memory as independent variables. The model accounted for 33.6% of the total variance, F(4, 234) = 21.21, p < .001. All four independent variables were significant predictors; condition: b = 0.42, t(234) = 4.96, p = .0001; strategy: b = 0.13, t(236) = 2.72, p < .01; working memory: b = .02, t(234) = 3.66, p < .001; AA acceptances: b = 0.32, t(234) = 3.24, p < .002. There was also a significant Condition × Working Memory interaction, b = −.02, t(234) = −2.06. p < .05.

To examine the nature of the interaction, we performed separate regressions for each condition, with strategy, working memory, and AA responses as independent variables. For the unlimited time condition, there were significant effects of both working memory, b = .02, t(118) = 3.11, p = .002, and strategy, b = .13, t(118) = 2.26, p = .026, and a marginal effect of AA responses. For the limited time condition, there were significant effects of strategy, b = .07, t(115) = 2.01, p = .046, and AA responses, b = .63, t(115) = 4.14, p < .001, with working memory having no significant effect (t < 1).

This analysis indicates that overall, the unshared variance in both working memory and strategy use contribute independently to the ability to correctly clarify nonlogical syllogisms. However, analysis of the Condition × Working Memory interactions suggests that while both working memory and strategy use contribute separately to responses in the unlimited time condition, when time is constrained, strategy use continues to have an impact, while working memory does not.

We then examined performance on the AC forms. We performed a regression, with number of rejections of the AC forms as dependent a variable and strategy, condition (4 s, unlimited), working memory, and number of acceptances of the AA items as independent variables, with interactions terms involving Condition × Working Memory and Condition × Strategy. The model accounted for 42.6% of the total variance, F(4, 234) = 28.93, p < .001. All four independent variables were significant predictors; condition: b = 0.42, t(234) = 4.39, p < .001; strategy: b = 0.19, t(234) = 3.49, p < .001; working memory: b = .02, t(234) = 3,41, p < .001; AA acceptances: b = 0.41, t(234) = 3.61, p < .001. There was also a significant Condition × Working Memory interaction, b = −.02, t(234) = −2.41, p = .017 and a marginal Condition × Strategy interaction (p = .09).

As before, we performed separate regressions for each condition, with strategy, working memory, and AA responses as independent variables. For the unlimited time condition, there were significant effects of working memory, b = .02, t(118) = 3.62, p < .001, strategy, b = .19, t(118) = 3.56, p < .001, and AA responses. For the limited time condition, there were significant effects of strategy, b = .32, t(115) = 5.28, p < .001 and AA responses, b = .69, t(115) = 2.78, p = .006, with working memory having no significant effect (t < 1).

To get a better idea of the strength of these effects, we performed Bayesian linear regressions (using JASP, Version 10) for the nonlogical and AC forms separately by condition, with working memory, strategy, and AA responses as independent variables, and calculated BFinclusion for the latter two. For the unlimited time condition, the nonlogical forms’ BFinclusion values were as follows: working memory = 32, strategy = 4; the AC forms were: working memory = 210, strategy = 154. Using the usual scale for interpreting effect strength showed that working memory had very strong support for both the nonlogical and AC forms, while strategy had very strong support for the AC forms, but moderate support for the nonlogical forms. For the time-constrained condition, the nonlogical forms’ BFinclusion values were as follows: working memory = .8, strategy = 2.2; the AC forms were: working memory = .6, strategy = 40,163. These results show that when time is severely constrained, strategy use was more strongly related to performance especially on the AC forms, with positive but weak support for the nonlogical forms. There was no evidence that working memory had any impact on either nonlogical or AC forms under time constraint.

Study 3

The results of Study 2 show that while the contribution of strategy use and working memory varies between the AC and the nonlogical forms, strategy use remains a strong predictor even when the contribution of working memory is factored out in both cases. In fact, under severe time constraint, working memory has no appreciable impact on either form of reasoning, while strategy use continued to show a strong relation to performance on the AC forms (although there is only weak evidence for this on the nonlogical forms). In this final study, we attempted to generalize the results to rapid reasoning with syllogisms with believable and unbelievable conclusions, used to evaluate the belief-bias effect (Evans et al., 1983; Markovits & Nantel, 1989). We also used a different measure of cognitive capacity, the short form of Raven’s Matrices, which is a measure of fluid intelligence. Previous studies have already shown that when given unlimited time, working memory and strategy use (de Chantal et al., 2019) and fluid intelligence and strategy use (Thompson & Markovits, 2020) separately predict the tendency to choose the logical response over the heuristic response in belief-biased problems. We thus examined only time-constrained belief-biased reasoning for which participants were given only 4s to respond.

Method

Participants

A total of 150 participants recruited from an online participant pool (45 females, 105 males; average age = 30 years, 4 months) took part in this experiment. Participants were each paid 1.5 pounds.

Material and procedure

Each participant received the same instructions as those used in the initial study. They were first given a set of 12 belief-biased inferences (see Appendix) presented in a random order, for which they were given 4 s per inference to respond. These syllogisms were identical to those used by de Chantal et al. (2019) and varied according to the validity of the inference and conclusion belief. The valid syllogisms were equivalent to modus tollens (MT) inferences (“All A are B, C is not B, therefore C is not A”), and the invalid syllogisms were equivalent to affirmation of the consequent (AC) inferences (“All A are B, C is B, therefore C is A”). The major premises were the same for both valid and invalid syllogisms, and conclusions were designed to be believable or unbelievable (for a full list of the syllogisms, see the Appendix). Affirmation of the consequent syllogisms with unbelievable conclusions and MT syllogisms with believable conclusions had conclusions with believability that were consistent with the logical conclusion and were referred to as consistent inferences. Affirmation of the consequent syllogisms with believable conclusions and MT syllogisms with unbelievable conclusions had conclusions that were inconsistent with the logical conclusion and were referred to as conflict inferences.

Participants were next given 12 items comprising the short form of Raven’s Matrices, which were presented in a single order. Finally, participants were given the identical strategy diagnostic used in the previous studies.

Results and discussion

We first examined performance on the belief-biased syllogisms. Overall, 14.3% of responses were missing, which is in the same range as that observed in the previous studies, and which once again indicated that this time constraint was close to the limits of participants’ ability to process the information. We then calculated the proportion of correct responses to the consistent and conflict syllogisms as a function of strategy (see Fig. 3). Overall, the proportion of correct responses to the consistent syllogisms (M = 0.74, SD = 0.24) were much higher than the percentage of correct responses to the conflict syllogisms (M = 0.27, SD = 0.26). The corresponding levels of correct responses to these identical items under unlimited time in the de Chantal et al. (2019) study for the consistent items were (M = 0.84, SD = 0.18) and for the conflict items (M = 0.51, SD = 0.34). Since this latter study used the same subject pool, it is clear that having much less time significantly decreases performance particularly on the conflict syllogisms, as would indeed be expected.

Fig. 3
figure 3

Proportion of logically correct responses to the consistent and conflict inferences as a function of strategy (counterexample, statistical)

Full size image

As before, participants were classed as either counterexample (N = 43), statistical (N = 56) or other (N = 51). Also, as before, participants classed as other were excluded from further analysis. There was a significant correlation between strategy use and number of correct responses on the Raven’s Matrices, r(99) = .33, p < .001. Counterexample reasoners had higher scores (M = 7.09, SD = 2.34) than do statistical reasoners (M = 5.38, SD = 2.57).

We then examined the relations between the extent to which participants were able to produce the logical correct response to the conflict syllogisms and performance on the Raven’s Matrices and strategy use. We performed a regression analysis, with number of correct responses on the conflict syllogisms as a dependent variable, and number of correct responses on the consistent syllogisms, performance on the Raven’s Matrices, and strategy use as independent variables. This model accounted for 18.8% of the total variance, F(3, 94) = 7.23, p < .0001. Of the three predictors, only strategy use was significant, b = 0.23, t(94) = 4.16, p < .0001. We then conducted a hierarchical regression, with the first two variables entered initially. This showed that strategy use accounted for 14.9% of the variability over and above that of the two other variables, F(1, 94) = 17.27, p < .0001. We also conducted a Bayesian linear regression, then calculated BFinclusion factors. This gave the value of 621 for strategy use and 0.36 for performance on the Raven’s matrices. This indicated extreme evidence for the usefulness of strategy use, but no evidence for the usefulness of performance on Raven’s Matrices.

General discussion

The results of these studies are important on two levels. First, they provide additional strong evidence that people are capable of making logical inferences even when the time allotted is very short, and as indicated by the percentage of nonresponses relatively close to the limits of their ability to process the information (e.g., Morsanyi & Handley, 2012; Newman et al., 2017). Such responses can be explained as being due to the automatization of frequently encountered inferential forms, so that the response is a form of an intuitive procedural rule (see Stanovich, 2018; Thompson et al., 2018). The current studies not only replicate this form of inferential intuition but also extend the notion of logical intuition to a more basic understanding of implication. Previous studies with young children have shown a tendency to accept the conclusion where the major and minor premise do not allow any conclusion at all (e.g., All A are B, and all C are D. Therefore, all A are D), while even young adolescents have no difficulty in rejecting the conclusion in these cases (Markovits et al., 1989). We postulated that severely time-restricted adults would tend to respond in similar ways to children (i.e., by showing a tendency to accept such nonlogical conclusions). The results of these studies clearly show this to be the case—as time is increasingly constrained, people increasingly tend to accept nonlogical conclusions. Nonetheless, even when allotted time is clearly at the limits of people’s processing capacity, people can reject these conclusions at an overall rate that is greater than chance (although this is restricted to counterexample reasoners). Such judgments can be seen as a form of coherence judgment, which has been shown to be amenable to intuitive functioning (Sobkow, Traczyk, Kaufman, & Nosal, 2018; Topolinski & Strack, 2009).

However, the most important contribution of these studies is the extension of the dual strategy model as an indicator of individual differences in the ability to produce very fast logical responses. The dual strategy model has proposed that people reason using one of two differing ways of processing information. The statistical strategy produces rapid likelihood estimations based on associative access to probabilistic information. The counterexample strategy is more focused and requires the generation of a representation of the premises with a corresponding search for potential counterexamples. A series of studies have shown that individual differences in strategy use predict key differences in the way that people reason (Markovits et al., 2015; Markovits, Brisson, & de Chantal, 2017a; Markovits, Brisson, de Chantal, & Singmann, 2018a) and predict differences in the extent to which belief biases unconstrained reasoning (de Chantal et al., 2019; Markovits, Brisson, de Chantal, & Thompson, 2017b). There is evidence that both working memory capacity (de Chantal et al., 2019) and fluid intelligence (Thompson & Markovits, 2020) predict the ability to produce logical conclusions on belief-biased inferences, and that in both cases strategy use is also a predictor of this ability.

Previous results (Thompson et al., 2018) have shown that cognitive capacity alone predicts both slow and fast reasoning. The present results extend and modulate these by showing that strategy use predicts the ability to produce logical responses to three different kinds of reasoning independently of measures of cognitive capacity. Overall, these studies suggest that counterexample reasoners are more able to produce logical responses under strong time constraints than are statistical reasoners. Importantly, these results provide evidence that when strategy use is included, measures of cognitive capacity are much weaker predictors of the ability to produce logical responses under severe time constraints. This suggests that strategy use is a strong predictor of intuitive logical reasoning, while cognitive capacity is not.

There are some alternative explanations of these results that must be considered. One explanation of the consistency between performance on slow and fast forms of reasoning would suggest that people who are more logical reasoners will have acquired the relevant logical schemas in a more solid way and can thus transfer these more easily under time constraint. This is the idea behind Stanovich’s notion of mindware acquisition (Stanovich, 2018), and is supported by the results obtained by Thompson et al. (2018). They found that high IQ predicts logical responding on both slow and fast reasoning, consistent with a mindware acquisition model. This basic model would suggest that cognitive capacity and any shared variance with strategy use would predict the extent to which logical reasoning is found both with unconstrained and time-constrained conditions. However, the present results show that in the latter condition, the predictive value of cognitive capacity is severely reduced, while strategy use becomes a much better predictor of very rapid logical responding. This makes the simpler interpretation of these results very unlikely, and clearly suggests that strategy use captures a specific component of intuitive logic.

What characteristics of strategy use could be responsible for their predictive value in fast reasoning? One possibility is that counterexample reasoners (who are often more logical, but not always; see, for example, Markovits et al., 2017) might generally make inferences more rapidly than statistical reasoners. However, previous results have shown that counterexample reasoners generally take more time to make inferences than statistical reasoners when not under any time constraints (Markovits et al., 2017). A second potential mechanism could be similar to standard dual process explanations that assume that one of the functions of Type 2 reasoning is the ability to inhibit belief-based intuitive responses, something that would be underpinned by general cognitive resources. In fact, the high predictive value of cognitive capacity during unconstrained reasoning of the kind examined here is certainly consistent with this idea. If rapid logical reasoning required strong inhibitory capacities, this relation should continue under time constraint. However, the much stronger effect of strategy use compared with cognitive capacity under time constraint suggests that the inhibitory component diminishes in these circumstances. In fact, the lack of any predictive value of cognitive capacity under time constraint is an extra argument for the existence of some form of logical intuition.

A final mechanism, which underlies our basic hypothesis, is based on the idea that one of the basic differences between counterexample and statistical strategies is attentional. Direct evidence for this was provided by a recent study by de Chantal et al. (2019), which used eye tracking to determine that when reasoning with belief-biased inferences, counterexample reasoners attend more often to problem premises than do statistical reasoners who attend relatively more often to conclusions. Since the role of attention under time constraint would be expected to increase, the attentional differences captured by strategy use would be increasingly important, and thus could well account for the particular predictive value of strategy use in time-constrained reasoning. In this context, it is telling that the relation between strategy use and mental rotation also increases under time constraint (Markovits, 2019), which supports this attentional mechanism, particularly since mental rotation does not involve any form of logical reasoning.

One point that should be discussed is the status of the “other” category. The diagnostic questionnaire that has been developed categorize people who give a consistent probabilistic pattern of responses as statistical reasoners, while those who give a consistent pattern of responses indicating the importance of counterexamples as counterexample reasoners. All other response patterns are classed in the other category and are not analyzed. Previous studies have used this same form of categorization, which has proved to show robust differences between people whose response patterns are consistent across a wide variety of forms of judgment and reasoning. However, the question of just what is included in the other category is an important one. A recent large-scale study of strategy use across a variety of forms of judgment (Thompson & Markovits, 2020) has suggested that this category consists of two subcategories. The first is composed of people who change strategies, while the second is composed of people who do not understand the basic nature of the problems. Subsequent studies are underway to examine how these difference subcategories vary over time.

Finally, these results add to a growing base of empirical results that are consistent with the idea that strategy use is an important individual difference. The fact that such differences have been observed not only with standard deductive reasoning tasks but also with belief-biased reasoning (de Chantal et al., 2019; Markovits et al., 2017), judgements involving processing of negative emotions (Markovits, Trémolière, & Blanchette, 2018b), mental rotation tasks (Markovits, 2019), and social biases (Gagnon-St-Pierre et al., 2020) and now with very fast reasoning, suggests that strategy use is a determinant of differences in the more general processing of information, one that goes well beyond standard deductive reasoning tasks.