Type of calculation

This variable includes both the type of arithmetic operation (addition, subtraction, multiplication, or division) required and whether that operation must be performed alone or in combination. Tasks involving multiplication and division tend to be more difficult than those requiring addition and subtraction, and tasks requiring two or more operations tend to be more difficult than tasks requiring only a singe operation. Codes for this variable ranged from 1 (easiest) to 5 (most difficult).

Operation specificity

This variable refers to the process of identifying and sometimes entering numbers into an arithmetic expression, including determining the appropriate operation to be performed. Tasks tend to be more difficult when the numbers that must be identified appear in a text and are neither in column format nor adjacent to each other. Tasks also tend to become more difficult when the operation(s) is not specified and when the wording in the question or directive does not contain an explicit semantic relationship statement such as "how many" or "calculate the difference." The codes for operation specificity ranged from 1 (easiest) to 9 (most difficult) based on a set of additive rules reflecting the various facets described here and fully operationalized in Appendix A.

In previous surveys, the goal has been to develop pools of prose, document, and quantitative tasks that represent the range of contexts, texts, and processes outlined here, with no specific requirement for particular numbers of any type of task. The goal was to draw materials from a wide variety of adult contexts that represented a wide range of linguistic structures such as those outlined in this paper. With respect to continuous or prose texts, the focus has been on expository texts since much of what adults read for work and in their community is associated with this type of discourse. However, some surveys did include narratives and poetry in small numbers. In terms of processes/strategies, the goal was to engage adults in the full range of processes that might reasonably be associated with each type of material. That is, the goal was to use the framework to construct questions/directives that were thought to be authentic to the kinds of information someone might want to understand or use from a particular text.

Validating the variables

In a previous section, three task characteristics labeled context, texts, and process/strategy were introduced. It was followed by a section in which each task characteristic was operationalized into a number of variables. This part of the framework describes a procedure for validating the set of variables developed from these characteristics that have been shown to affect task performance and the placement of tasks along each of the reporting scales. This process borrows heavily from work that has been done in the area of adult literacy where several national and international surveys have reported data that followed this approach:

  • The US Department of Labor's Literacy Assessment (Kirsch and Jungeblut, 1992)
  • The International Association for the Evaluation of Educational Achievement (IEA) Reading Literacy Study (Kirsch and Mosenthal 1994)
  • The National Adult Literacy Survey (Kirsch et al., 1993)