The approach described in this framework for the assessment of problem solving relies on the (moderately) familiar tasks mentioned in the previous section. Within a somewhat familiar context the problems to be solved are "intransparent" enough not to be perceived as pure routine tasks. On the other hand, the domain-specific knowledge prerequisites are sufficiently limited as to make analytical reasoning techniques the main cognitive tool for solving the problems.

As described in section 2.1, given a moderately familiar contextual embedding, the competency level required to solve a problem can be estimated by the scope and complexity of the problem. Based on results from general psychology and developmental psychology (e.g., Post-Piagetian theories of cognitive development), we can describe a hypothetical proficiency scale for analytical problem solving which was tested empirically with the ALL pilot data (see chapter 5). Similar levels have been described within the frameworks of other large-scale assessments of problem-solving competencies such as the project test for Hamburg/Germany (Ebach, Klieme and Hensgen, 2000) and the PISA 2003 assessment of cross-curricular problem solving (OECD, in press).

Four levels of problem-solving proficiency are postulated:

Level 1:
At a very elementary level, concrete, limited tasks can be mastered by applying content-related, practical reasoning. At this level, people will use specific content-related schemata to solve problems.

Level 2:
The second level requires at least rudimentary systematical reasoning. Problems at this level are characterized by well-defined, onedimensional goals; they ask for the evaluation of certain alternatives with regard to transparent, explicitly stated constraints. At this level, people use concrete logical operations.

Level 3:
At the third level of problem-solving proficiency, people will be able to use formal operations (e.g. ordering) to integrate multidimensional or ill-defined goals, and to cope with non-transparent or multiple dependent constraints.

Level 4:
At the final and highest level of competency, people are capable of grasping a system of problem states and possible solutions as a whole. Thus, the consistency of certain criteria, the dependency among multiple sequences of actions and other "meta-features" of a problem situation may be considered systematically. Also, at this stage people are able to explain how and why they arrived at a certain solution. This level of problem-solving competency requires a kind of critical thinking and a certain amount of meta-cognition.

The next chapter gives a brief overview of different tasks and approaches for measuring problem-solving competency. Based on the discussion in chapter 3, a concrete solution for measuring analytical problem solving, as it was just introduced, is presented in chapter 4.