4.2 Tacit knowledge as a measurement construct

What are tacit-knowledge tests, and the items contained within them, intended to measure?

This question can be answered by considering a traditional distinction between achievement testing and cognition testing. In achievement testing, items are presumed to exemplify the measurement construct (e.g., knowledge of world history) but are not commonly viewed as predictors. For example, when an individual correctly answers a factual, multiple-choice question about world history, we assume that she possessed prior knowledge of either the fact in question or related facts that enabled her to rule out incorrect alternatives. We do not commonly view the history question as predictive of performance on other tests or tasks. In cognition testing, by contrast, items are presumed to predict performance but are not commonly viewed as exemplars of the measurement construct. For example, when an individual correctly solves a figural analogy problem, we do not assume that he possessed prior knowledge of the analogical relationship in question. However, we do view such analogy problems as predictive of performance on other tests and tasks of general mental skill.

Is a measure of tacit knowledge a cognition test or an achievement test? Having drawn a distinction between cognition and achievement testing, we must point out that neither type of test exists in a pure form (Sternberg, 1998a). All achievement tests measure underlying skills—if only the skills necessary to acquire and display mastery of the tested content—and so tend to have predictive value. Likewise, all cognition tests measure acculturated knowledge—if only the knowledge necessary to make sense of items and testing conventions—and so tell us something about the knowledge content of individuals rated high and low in general cognition. All of these tests measure a form of developing expertise (Sternberg, 1998a). Tacit-knowledge tests break down the (artificial) boundaries between achievement and skill testing.

Tacit-knowledge tests are everyday situational-judgment tests built on a theory of human cognition (Sternberg, 1995c). They are intended to measure both practical, experience-based knowledge and the underlying dispositions or skills that support the acquisition and use of that knowledge. Thus, scores on tacit-knowledge tests are expected to predict performance on tests or tasks that draw on either tacit knowledge or the mental skills that supported its development and use. These skills are hypothesized to differ from those implicated in the so-called "general factor" in human cognition commonly referred to as g and often approximately measured, in norm-referenced fashion, as IQ. Research by Sternberg and colleagues has produced support for the hypothesis that the skills associated with tacit-knowledge test performance are different than those associated with tests of g (Hedlund et al., 1999; Sternberg et al., 1993; Sternberg et al., 1995).

Because tacit-knowledge items are considered to measure both acquired knowledge and practical skill, we propose that tacit-knowledge tests have the potential to shed light upon (1) the content of tacit knowledge and (2) the events or experiences through which it was acquired. Few would contest that tacit-knowledge items reflect the knowledge of the respondents from whom the items were obtained (in the course of a "story-telling" exercise focusing on personal experiences). The items came from these respondents' memories and so must reflect the content of those memories. What remains to be determined is the degree to which tacit-knowledge items measure the acquisition and use of tacit knowledge by those who did not produce but, rather, endorsed or rated the items. This question is addressed by our numerous research studies in both civilian and military sectors, which we discuss in subsequent sections.