4. The "Project approach"

How can contextualized, real-life problems be defined and transformed into test items? The project approach put forth in this framework uses different problem-solving phases as a dimension along which to generate the actual test items.

Following Pólya (1945, 1980), the process of problem solving has been frequently described in terms of the following stages:

1. Define the goal.
2. Analyze the given situation and construct a mental representation.
3. Devise a strategy and plan the steps to be taken.
4. Execute the plan, including control and — if necessary — modification of the strategy.
5. Evaluate the result.

These stages correspond to the results of research on vocational training and job analyses within educational research and applied psychology that have been described as a part of the so-called "complete action" approach. Extensive analyses of very different jobs (different professions with varying types of work places) indicate that new forms of labor organization require people to perform more complex operations that go "beyond mere routine". Nowadays, even production workers and office clerks are required to master complex tasks requiring integrative skills. Complete actions include different steps such as planning, executing and evaluating. The basic structure of the model of complete action is thus fully compatible with the above-mentioned normative process model for problem solving — action steps are similar to problem-solving steps.

The model of complete action has been successfully applied to curriculum development, assessment, and certification reforms in various professions in both Germany and Luxembourg (Hensgen and Blum, 1998; Hensgen and Klieme, 1998). The main idea is that both training tasks and also test problems should include all or most elements of a complete action.

4.1 ALL problem solving test construction

The ALL project approach uses this complete action model to establish the underlying structure of the problem-solving test. The different action steps define the course of action for an "everyday" project. One or more tasks or items correspond to each of these action steps. The respondents thus work on the individual tasks that have been identified as steps that need to be carried out as a part of their project (e.g., "planning a family reunion" or "renovating a clubhouse").

Embedding the individual tasks in an action context yields a high degree of context authenticity. A project, designed as a complete action, encompasses various tasks that can vary in complexity. Since assessing problem-solving skills is a relatively new endeavor, it seems appropriate to provide a somewhat detailed account of the general construction process. Furthermore, such a description is important for transferring or adapting this concept for other assessment purposes. Figure 1 illustrates the basic construction of a project.