These basic planning assumptions were subsequently ratified at both the first and second meetings of the ALL Project Advisory Group (PAG) which is responsible for providing management oversight and advice to the Project within the OECD programme of work.

A meeting of experts, convened in May 1997, reviewed the proposed design of the study and the frameworks for each of the skill domains. Held at the OECD in Paris, the meeting concluded that additional work needed to be done on several of the skill frameworks if the study was to have a reasonable chance of generating valid, reliable and comparable skill profiles at the international level. As a result, the teams responsible for numeracy, problem-solving, teamwork and practical cognition were funded to refine their frameworks and to collect sufficient empirical data to demonstrate the measurement properties of the proposed measures trans-nationally.

Two subsequent meetings of the international study team were hosted by the US National Center for Education Statistics in Washington, DC to review progress and plan for further development. Held in April 1998 and September 1998, these meetings concluded that while the proposed frameworks for problem-solving and attitudes towards teamwork were judged to be adequate, the approach to measurement had failed to yield data of sufficient quality. In addition, the proposed instrumentation for measuring computer literacy was judged inadequate. As a result, new development teams were recruited and funded by Statistics Canada, NCES and the Governments of Sweden and Luxembourg.

Two additional meetings were held.

  • First, a meeting of all development team members was held in Washington, DC January 20-23, 1999 to help integrate the different assessments and to provide expert feedback.
  • A second meeting of development team leaders was held in Princeton, NJ on August 23-24, 1999 to review the frameworks and the results of small-scale, multi-country piloting with a view to selecting domains for inclusion in the ALL pilot.

International teams were subsequently recruited to draft final assessment frameworks and test specifications and to develop items in seven cognitive domains:

  • prose literacy
  • document literacy
  • numeracy
  • problem solving
  • practical cognition
  • teamwork and,
  • information and communication technology