4.1.2 Simulations

Simulations have been used as both assessment tools and as training methods. Simulations are aimed at assessing job behaviors directly. They involve observing people in situations that have been created to simulate aspects of the actual job situation. Responses to these simulations are considered to represent the actual responses that individuals would exhibit in real situations. Simulations can take the form of in-basket tests, situational interviews, group discussion, assessment centers, and situational-judgement tests. Motowidlo et al. (1990) distinguished between high-fidelity and low-fidelity simulations. In high-fidelity simulations, the stimuli presented to the respondent closely replicate the actual situation and the individual has an opportunity to respond as if they were in the actual situation. In low-fidelity simulations, the stimuli are presented in written or oral form and the individual is asked to describe how he or she would respond to the situation, rather than actually to carry out the behavior.

At the high-fidelity end of the continuum is the assessment center. Assessment centers present small groups of individuals with a variety of tasks, including in-basket tests, simulated interviews, and simulated group discussions (Bray, 1982; Thornton and Byham, 1982). The simulation approach has the advantage of more closely representing actual job performance. However, it is not always clear what aspects of the job should be chosen to simulate or how to evaluate performance.

In-basket tests have a moderate level of fidelity. In an in-basket test, the participant is presented with various materials (e.g., memos, financial reports, letters) and is asked to respond to them (Frederiksen, 1966; Frederiksen, Saunders, and Wand, 1957). The individual, however, has a limited amount of time to deal with the problems presented in the in-basket, giving him or her some of the constraints of actual job situations. Performance is evaluated based on how the items are handled. For example, does the participant respond to a letter from the Director of Finance requesting fourth-quarter financial records with complete and accurate information?

Situational-judgment tests have been considered low-fidelity simulations (see Motowidlo et al., 1990). Situational-judgment tests (SJTs) present descriptions of situations, typically work-related, in which a problem exists (see e.g., Chan and Schmitt, 1998; Legree, 1995; Motowidlo et al., 1990). The descriptions can be of actual situations or written to approximate actual situations in the domain of interest (e.g., a salesperson making a phone solicitation). Situations typically are selected on the bases of a critical-incident analysis. Following each situational description is a set of options (i.e., strategies) for solving the problem. Respondents are asked to indicate their endorsement of the options, either by selecting the best and possibly the worst from among a few strategies, or rating the effectiveness of each alternative. Traditionally, SJTs have been scored by awarding points based on the correct choice of the best and worst options (e.g., Motowidlo et al., 1990), or awarding points based on the percentage of experts who endorse the option (e.g., Chan and Schmitt, 1998).

4.1.3 Tacit-knowledge approach

The tacit-knowledge approach draws on aspects of the above approaches in order to measure the level of domain-specific, procedural knowledge that individuals have acquired from solving everyday problems. It is based on theoretical and empirical claims that the amount and organization of knowledge that experts possess differs from that of novices (see Chi et al., 1988; Ericsson, 1996; Ericsson and Smith, 1991), and that these knowledge differences reflect differences in the developed skills of experts and novices (Sternberg, 1998a; in press-a).