Similarly, items had to be sampled from multiple life contexts i.e. home and family, health and safety, community and citizenship, consumer economics, work and leisure and recreation so that individuals of all walks of life were able to find familiar tasks, and no single group would be placed at a disadvantage. Items also had to systematically differ in the type of thinking involved and the type of materials used, elements specified in each framework.

Finally, the assessment methods used had to minimize the amount to which items in a particular domain were dependant on skill in other domains. That is especially true for designing items of low level of difficulty in traits where pure orthogonality between domains is difficult to assume. For example, steps were taken to reduce the amount of reading associated with low level numeracy items so that individuals with low reading skills would have an opportunity to display their numeracy skills.

Development and validation of the ALL frameworks and assessment protocols

Armed with these criteria, a four phase research and development program was launched.

First, an international team of experts was recruited to review the relevant literature with a view to:

• defining the domain
• specifying the variables that underlie relative difficulty
• developing example items to illustrate the recommended approach to assessment

The assessment frameworks and test blueprints produced by these teams were exposed to critical review and revised accordingly.

Second, a variant of each assessment was piloted in at least two countries in order to establish that:

• the items functioned as predicted, and
• respondents reacted in a positive way to the assessment items

At this point the ALL developers faced a decision.

Frameworks and associated approaches to measurement that were judged to be sufficiently robust were referred on to a phase of international item development designed to:

• expose a large number of researchers to the theory and approach to measurement proposed for ALL
• ensure that items selected for inclusion in ALL provided good linguistic, geographic and cultural coverage.