6.3 Internal validation

6.3.1 Internal-consistency reliabilities

For distance scores, coefficient a internal-consistency reliabilities were .96 for the U.S. sample and .92 for the Spanish sample. For rank-order correlation scores, internal-consistency reliabilities were .94 for the U.S. sample and .73 for the Spanish sample. For 1/0 scores, KR-20 internal-consistency reliabilities were .89 for the U.S. sample and .62 for the Spanish sample. The overall 1/0 internal-consistency reliability for the combined samples was .82.

6.3.2 Comparison of responses between U.S. and Spanish samples

The correlation for mean profiles of responses to item options was r = .91, and the correlation for standard deviations of responses to item options was r = .66 (with 8 options for 30 items, or 240 observations, in each set). These correlations indicate that the responses across countries were about as similar as one could hope for, given the reliabilities of the data.

6.3.3 Item characteristics

We analyzed item characteristics for the combined samples using 1/0 scoring. The range of facilities (p-values) was from .42 to .84, with a mean p-value of .62. The range of difficulties (Δ values) was from 9.1 to 13.4 with a mean of 11.8. The range of discriminating power for items was computed with both biserial and point-biserial correlations. The range of rbis was from .38 to .72, with a mean of .53. The range of rpbis was from .29 to .53, with a mean of .41.

Specifics for the 1-Parameter Logistic (Rasch) and 2-Parameter Logistic (Birnbaum) Models. One- and two-parameter-logistic (PL) models, as derived from item-response theory (IRT), were fit to the data. The difference in the maximal marginal likelihoods was 84.7, with 30 degrees of freedom, indicating the superior fit of the two-parameter model. For the one-parameter model, 7 items had statistically significant chi-squares (p < .05), whereas for the two-parameter model, none of the items did.

6.3.4 Item-information statistics

Table 1 shows the point of maximum information for each of the 30 items in the ESJI. The results suggest that the items tended to be rather easy, on the whole, and that subsequent versions of the ESJI probably need some more difficult items.