4.5 Summary

The goal of the test-development process outlined in this section is to support the construction of valid tacit-knowledge tests. Our theoretical model of tacit knowledge, described in the previous section, constitutes, we believe, a step in the direction of this goal. By elaborating on what we consider to be tacit knowledge at a theoretical level, we set the stage for a more detailed consideration of item content during the selection process and, in so doing, increase the substantive validity of our tests. The analysis of item ratings and performance data constitutes a second step towards measuring the construct. By identifying those items with the strongest association with performance criteria, we increase the probability that we will select items and construct test questions that embody the construct—given that tacit knowledge has clear benefits for performance. The analysis of the underlying structure by sorting items into categories constitutes a third step toward our goal. By examining the structure of the tacit-knowledge space (based on the data from our sample), we are able to make more informed decisions about the distribution of item content in our tacit-knowledge tests and, in so doing, increase the structural validity and generalizability of score interpretations. Finally, by conducting validation studies we provide support that tacit knowledge is relevant to understanding performance in the domain of interest and that it contributes to that understanding beyond traditional indicators of performance. In the next two sections we discuss the development and validation of tests to measure tacit-knowledge in civilian and military domains.