8. Future directions
There still are some issues that need to be addressed in further work on the scale, and
we plan to address these issues in the next version.
- Face validity. The use of the term boss proved
to be a mistake, as it carries a somewhat negative connotation in European
and other countries were
hierarchical
arrangements
of workers are less socially acceptable than in the United States. The
inventory also relied too much on office-type settings and needs to be expanded
to include
not only a
broader range of occupational settings, but settings outside of the workplace.
Our goal in the next version of our questionnaire, therefore, is to do
extensive revision for face
validity to ensure that the questionnaire we use will be viewed as face
valid by all test-takers. Following a procedure we have used in other research,
our
plan is to ask test-takers
to evaluate the face validity of the questionnaire by asking them how realistic
the scenarios and solutions are.
- Length. The ESJI was longer than would be ideal, given realistic constraints
upon administration time. It could be and should be shortened both with
respect to number
of items (from 30 to, perhaps, 20) and number of items (from 8 to, perhaps,
5).
- Description of test. The test originally was described
as a test of practical ability or of
everyday cognition. We have changed the name, effective as of this
article, to reflect better both what the test measures and the constraints
of the
sociopolitical context in
which the test will be administered. We thus will refer to the test
as an everyday-situation- judgment test.
- Number of scale points. The number of scale points were item will be
seven in order to ensure that each point supplies useful information.
Each scale
point will have a
verbal label describing its meaning.
- Need for anchors in scoring. Scoring in the future will be converted
to a 0-500 scale in order to match other assessments in the study of
adult competencies
in OECD
countries. The mean will be 250 and the standard deviation, 50.
We will anchor score points to specific levels of competency verbally described.
Specific levels of competency
will be anchored to various indicators of job performance, career
satisfaction,
and life satisfaction (with relevant data to be obtained from a
new validity
study).
- Objective scoring. In addition to the type of prototype scoring we
have used, we will experiment with right-wrong scoring based on sets
of values that our previous research
indicates seem to be universally accepted as indicating preferred
behavior (e.g., honesty, sincerity, hard work, compassion, etc.)
- Uninformative options. Options that elicit means very near the middle
of the scale together with high standard deviations will be eliminated,
as
such
options tend to be
ones on which there is substantial disagreement among respondents.
- Skills taxonomy. We already have begun development of a skills taxonomy
and this development will be formalized for the next version of the
ESJI. We have discovered
that certain behaviors tend to be valued cross-situationally and
other behaviors
to be devalued. Examples of valued behaviors are productivity, honesty,
politeness, serving
as a good example to others, and doing what needs to be done even
if one does not want to do it. Examples of devalued behaviors are lying,
not working when one should,
passing the buck, blaming others for one's own mistakes, failing to meet one's
responsibilities, and doing things grudgingly.
- Test-retest reliabilities. The present design did not
allow for computation of test-retest reliabilities to measure scale stability,
but future
studies will do so.
|