Appendix A
Coding rules for the process variables
Type of information
Type of information requested refers to the nature of information that readers
must identify to complete a question or directive. Types of information form
a continuum of
concreteness, which was operationalized as follows for purposes of this analysis:
- When the requested information is a person, animal, place, or thing,
score 1.
- When the requested information is an amount(s), time(s), attribute(s),
action(s), or location(s), score 2.
- When the requested information is a manner, goal, purpose, condition,
or predicate adjective, score 3.
- When the requested information is a cause, result, reason, evidence,
similarity, or pattern, score 4.
- When the requested information is an equivalent, difference, or theme,
score 5.
Plausibility of distracting information
Plausibility of distracting information refers to whether or not an identifiable
match exists between information in the question and the text, or between the
text and the
distractors in a multiple-choice question, which makes it difficult for readers
to identify the correct answer. The scoring rules for plausibility of distracting
information are as
follows:
- When there is no distracting information in the text, score 1.
- When distractors contain information that corresponds literally or
synonymous to information in the text but not in the same paragraph
as the answer, score 2.
- When distractors contain information that represent plausible invited
inferences not based on information related to the paragraph in which
the answer occurs, score 3.
- When one distractor in the choices contains information that is related
to the information in the same paragraph as the answer, score 4.
- When two or more distractors in the choices contain information that
is related to the information in the same paragraph as the answer, score
5.
- When one or more distractors represent plausible inferences based on
information outside the text, score 5.
|