The idea of using planning tasks to measure problem solving was independently invented by other research groups. Funke and Fritz (1995) devised several experimental variants of tests requiring planning skills. Klieme, Ebach et al. (in press) developed a multiple-choice test of problem-solving competencies for a large-scale assessment program in one of the German federal states. This test consists of "projects" such as organizing a party or planning a trip that involve different types of tasks. This "project approach" is currently also used to assess cross-curricular problem-solving competencies in a German extension to PISA (Klieme, Funke et al., 2001; Klieme, in press). On an international scale, similar approaches have been recently used within the OECD's framework for problem-solving assessment (OECD, in press). PISA 2003 will include so-called "decision making" problems as well as "system analysis and design" problems. Both problem-solving types share the main features of "projects" as defined here.

3.4 Conclusions

This overview of the assessment instruments available to measure problem-solving competencies shows that none of these instruments fulfill all of the requirements for the problem-solving assessment within the ALL study. However, the project approach retains the potential to be a powerful instrument for assessing analytical problem-solving skills in real world context, as will be elaborated in greater detail in the next chapter. Solving problems in project-like settings is important and relevant for adults in both their professional and their private life. Furthermore, the project approach has been successfully implemented in other large-scale assessments, and it can be realized as a paper-and-pencil-instrument, which is of crucial importance for contemporary large-scale surveys. The item construction allows for a distinctive operationalization of the various proficiency levels defined in section 2.5. As described in chapter 5, the results of the feasibility study and the pilot study show the success of this approach.

The next chapter gives an in-depth description of the project approach as implemented for the ALL study.