Phase 4
Examining and Revising the Project
The projects and tasks are then revised and modified by experts according to the following criteria:
- General logic of action steps
- Coverage of relevant cognitive processes
- Completeness of the information provided
- Clarity in the statement of the item
- Unequivocal solutions
- Balance among the item formats
- Independence of the questions from each other
- Life relevance/face validity
- Degree of difficulty appropriate to the target group
- Amount and type of previous knowledge required
- The appropriateness in other cultures
- Political correctness
- Lack of group-related bias
Phase 5
Feasibility Study
A feasibility study with small samples should provide first results on:
- Consistency, reliability, difficulty, and discriminative power of the individual items
- Internal structure of the test
- Adequacy with respect to gender, education level, socio-economic status, motivation, previous experience, and language/culture
- Acceptance of instrument
- Operational concerns
Based on these results, the projects should then be revised and modified.
The thus constructed projects are structured in the following way:
- A description of the problem situation is given, i.e. the "project" is introduced and the respondent's role is specified, followed by a list of steps that need to be worked through as a part of this "project". This list of things that need to be done corresponds exactly with the items the respondent is then asked to solve.
- The different tasks: The information section supplies an introductory description and provides all the information the respondent needs to solve the task. The question section specifies the concrete question or questions (there can also be multiple, combined questions) that the respondent is requested to answer, and specifies how these are to be answered.