7. Conclusions

Approximately 25 years ago, McClelland (1973) questioned the validity of cognitive-skill testing for predicting real-world criteria such as job performance, arguing in favor of competency tests that more closely would reflect job performance itself. Subsequent reviews of the literature on the predictive validity of cognition tests suggest that McClelland may have been pessimistic about the validity of cognition tests: individual differences in cognition-test performance account for, on average, between 4 and 25 percent of the variance in real-world criteria such as job performance (Barrett and Depinet, 1991; Hunter and Hunter, 1984; Schmidt and Hunter, 1998; Wigdor and Garner,1982). Nevertheless, these findings indicate that between 75 and 96 percent of the variance in real-world criteria such as job performance cannot be accounted for by individual differences in cognition-test scores. The emerging literature on practical cognition and similar constructs (e.g., social and emotional cognition) is a belated response to McClelland's call for new methods to assess practical skills. The literature and research reviewed in this volume provide several sources of evidence to support a distinction between academic and practical cognition.

First, the distinction between academic and practical cognition is entrenched in the conception of cognition held by laypeople and researchers alike. In addition to evidence provided by studies of implicit theories of cognition (e.g., Sternberg et al., 1981), analyses of researchers' descriptions of the nature of cognition suggest a prominent role for practical cognition. Seventy years ago, the editors of the Journal of Educational Psychology convened a symposium at which prominent psychological theorists of the day were asked to describe what they imagined cognition to be and what they considered the most crucial "next steps" in research. In a replication, Sternberg and Detterman (1986) posed these same questions to contemporary prominent theorists. An analysis of the responses of both cohorts of cognition theorists revealed concerns about practical aspects of cognition (Sternberg and Berg, 1986). For example, among the 42 crucial next steps that were mentioned by one or more theorists from either cohort, studying real-life manifestations of cognition was among the most frequently mentioned "next steps" of both the contemporary researchers and the original respondents. A distinction between academic and practical aspects of cognition also is supported by older adults' perception of age-related changes in their skill to think and solve problems (Williams, Denney, and Schadler, 1983). Three-fourths of the older adults sampled believed that their skill to solve practical problems increased over the years, despite the fact that performance on academic tasks begins to decline upon completion of formal schooling.

A second source of evidence to support a distinction between academic and practical cognition is the set of results of studies in which participants were assessed on both academic and practical tasks. These studies consistently find little or no correlation between performance on the two kinds of tasks. Overall, cognitive ability tests and similar measures are unrelated to (a) the order-filling performance of milk-processing plant workers (Scribner, 1986); (b) the degree to which racetrack handicappers employ a complex and effective algorithm (Ceci and Liker, 1986, 1988); (c) the complexity of strategies used in computer-simulated roles such as city manager (Dörner and Kreuzig, 1983; Dörner et al., 1983); and (d) the accuracy with which grocery shoppers identified quantities that provided the best value (Lave et al., 1984; Murtaugh, 1985). This research shows that the performance of both children and adults is susceptible to the context in which skills are measured. When problems are presented in a familiar context, whether that context is school or work, individuals appear more intelligent (e.g., Carraher et al., 1985; Roazzi, 1987).