Table I.2

Four levels of difficulty for the problem solving domain
Problem solving
Level 1
(0 to 250)
Tasks in this level typically require the respondent to make simple inferences, based on limited information stemming from a familiar context. Tasks in this level are rather concrete with a limited scope of reasoning. They require the respondent to make simple connections, without having to check systematically any constraints. The respondent has to draw direct consequences, based on the information given and on his/her previous knowledge about a familiar context.
Level 2
(251 to 300)
Tasks in this level often require the respondent to evaluate certain alternatives with regard to well-defined, transparent, explicitly stated criteria. The reasoning however may be done step by step, in a linear process, without loops or backtracking. Successful problem solving may require to combine information from different sources, as e.g. from the question section and the information section of the test booklet.
Level 3
(301 to 350)
Some tasks in this level require the respondent to order several objects according to given criteria. Other tasks require him/her to determine a sequence of actions/events or to construct a solution by taking non-transparent or multiple interdependent constraints into account. The reasoning process goes back and forth in a non-linear manner, requiring a good deal of self-regulation. At this level respondents often have to cope with multi-dimensional or ill-defined goals.
Level 4
(351 to 500)
Items in this level require the respondent to judge the completeness, consistency and/or dependency among multiple criteria. In many cases, he/she has to explain how the solution was reached and why it is correct. The respondent has to reason from a meta-perspective, taking into account an entire system of problem solving states and possible solutions. Often the criteria and the goals have to be inferred from the given information before actually starting the solution process.

Organization of the report

Chapter 1 presents international, provincial and territorial comparisons of performance in the four domains measured (prose literacy, document literacy, numeracy, and problem solving). This chapter also presents analysis exploring changes in literacy performance over time. Chapter 2 examines variations in the population distributions of proficiency by various demographic characteristics, including age, gender and education. Chapter 3 analyses the proficiency distributions of population groups of special interest, including Aboriginal and immigrant populations. Chapter 4 analyses the relationships between proficiency and economic outcomes such as labour force participation and income inequality. Chapter 5 presents analytical results concerning the magnitude of the effects of proficiency on social outcomes, notably health and civic engagement and investigates patterns in the relationships between proficiency and the use of information and communication technologies (ICT) in Canada. The key findings are summarized and discussed in the Conclusion.

Data underlying the graphs and figures in the body of the report can be found, often with more details, in the tables of Annex A. A detailed overview of the IALSS proficiency scales – how they are defined, how they were measured, how proficiency was summarized and how proficiency estimates should be interpreted can be found in Annex B. Annex C documents key methodological aspects of the study and, finally, Annex D identifies the various federal, provincial, territorial and international agencies responsible for the IALSS.