Responses to the test were scored by correlating ratings on each item with an index variable for group membership (1 = undergraduate, 2 = graduate student, 3 = faculty member). A positive correlation between item and group membership indicated that higher ratings on the item were associated with more expertise, and a negative correlation indicated the opposite. Wagner and Sternberg (1985) validated the test using several criteria. They obtained from faculty members citation rates, the number of publications, number of conferences attended in the last year, number of conference papers presented, distribution of time across teaching and research, academic rank, year Ph.D. was obtained, and level of institutional affiliation (high or low). For undergraduates, they obtained scores on the Verbal Reasoning section of the Differential Aptitude Test (Bennett, Seashore, and Wesman, 1974).

Wagner and Sternberg (1985) found that tacit-knowledge test scores correlated significantly, and positively, with number of publications (.33), number of conferences attended (.34), rated level of institution (.40), and proportion of time spent in research (.39). For the undergraduates, tacit-knowledge tests scores did not correlate significantly with verbal-reasoning scores (r = -.04, ns).

In a follow-up study by Wagner (1987), a revised version of the test was administered to 91 faculty, 61 graduate students, and 60 Yale undergraduates. The revised test contained 12 situations with 9 to 10 response options. Wagner obtained ratings for both conceptions of what the person would do in their actual job and what they would do in their ideal job. Scores were obtained for the overall test, and for six subscales that crossed three kinds of tacit knowledge: tacit knowledge about managing oneself, managing others, and managing tasks, with two orientations of tacit knowledge: local (pertaining to the situation at hand) versus global (pertaining to a bigger picture) tacit knowledge.

A different scoring method was used than in Wagner and Sternberg (1985). An expert profile was created by administering the test to a sample of professors who were nominated as high on practical cognition. A distance score (d²) was computed between the participant's ratings and the mean of the experts' ratings. The mean d² values for the three groups were 339 for faculty, 412 for graduate students, and 429 for undergraduates, indicating that tacit knowledge increased, on average, with level of experience (a smaller value representing greater tacit knowledge). There were exceptions in each group, however, suggesting that what mattered was not merely experience by what one has learned from experience.

Wagner then examined the relationship of tacit knowledge with the same criterion measures that were used in Wagner and Sternberg (1985). Because the tacit-knowledge test was scored using a distance measure, a lower distance, or smaller value, represents better tacit-knowledge score. Therefore, negative correlations reflect a positive association between tacit-knowledge scores and the criterion.

For the actual-job ratings, significant correlations were obtained between tacit-knowledge scores and ratings of department (-.48), number of citations (-.44), number of publications (-.28), proportion of time spent on research (-.41), and number of papers presented. The correlations for ideal-job ratings were slightly lower, but comparable. Again, the tacit-knowledge scores did not correlate with verbal-reasoning skill. Wagner did find significant intercorrelations among the six subscales, ranging from .2 to .4. He interpreted these correlations to indicate a weak general factor for tacit knowledge, a factor that appears to be distinct from the general factor measured by traditional cognition tests.