Abstract
In this paper, the efficiency of conditional maximum likelihood (CML) and marginal maximum likelihood (MML) estimation of the item parameters of the Rasch model in incomplete designs is investigated. The use of the concept of F-information (Eggen, 2000) is generalized to incomplete testing designs. The scaled determinant of the F-information matrix is used as a scalar measure of information contained in a set of item parameters. In this paper, the relation between the normalization of the Rasch model and this determinant is clarified. It is shown that comparing estimation methods with the defined information efficiency is independent of the chosen normalization. The generalization of the method to other models than the Rasch model is discussed.
In examples, information comparisons are conducted. It is found that for both CML and MML some information is lost in all incomplete designs compared to complete designs. A general result is that with increasing test booklet length the efficiency of an incomplete design, compared to a complete design, is increasing, as is the efficiency of CML compared to MML. The main difference between CML and MML is seen in the effect of the length of the test booklet. It will be demonstrated that with very small booklets, there is a substantial loss in information (about 35%) with CML estimation, while this loss is only about 10% in MML estimation. However, with increasing test length, the differences between CML and MML quickly disappear.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
1 Andrich, D. (1978). Relationships between the Thurstone and Rasch approaches to item scaling. Applied Psychological Measurement 2, 451–462.
2 Andrich, D. (1988). The application of an unfolding model of the PIRT type for the measurement of attitude. Applied Psychological Measurement, 12, 33–51.
3 Andrich, D. (1989). A Probabilistic IRT model for unfolding preference data. Applied Psychological Measurement, 13(2), 193–216.
4 Andrich, D., & Luo, G. (1993). A hyperbolic cosine latent trait model for unfolding dichotomous single-stimulus responses. Applied Psychological Measurement, 17, 253-276.
6 Bickel, P.J., & Lehmann, E.L. (1979). Descriptive statistics for nonparametric models IV: Spread. In Jurecková, (Ed.), Contributions to Statistcs, Hájek Memorial Volume, (pp. 33–40). Prague Academia. casellanberger
5 Casella, G., & Berger, R.L., (1990). Statistical Inference, Belmont, CA, Duxbury. InCollectionbickel lehmann
7 Coombs, C.H. (1964). A theory of data, New York, Wiley.
8 Davison, M., (1977). On a metric, unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523–548.
9 DeSarbo, W.S., & Hoffman, D.L. (1986). Simple and weighted unfolding threshold models for the spatial representation of binary choice data. Applied Psychological Measurement, 10, 247–264.
10 Douglas, J. (1997). Joint consistency of nonparametric item characteristic curve and ability estimates. Psychometrika, 47, 7–28.
11 Formann, A.K. (1988). Latent class models for nonmonotone dichotomous items. Psychometrika, 53, 45–62.
11 Hemker, B.T., Sijtsma, K., Molenaar, I. W., & Junker, B.W. (1997). Stochastic ordering using the latent trait and the sum score in polytomous IRT models. Psychometrika, 62, 331–347.
12 Hoijtink, H.,(1990). A latent trait model for dichotomous choice data. Psychometrika, 55, 641–656.
13 Hoijtink, H. (1991). PARELLA: Measurement of latent traits by proximity items. Leiden, The Netherlands: DSWO Press.
14 Hoijtink, H., & Molenaar, I.W. (1992). Testing for diff in a model with single peaked item characteristic curves: The parella model. Psychometrika, 57, 383–397.
15 Johnson, M.S., (2001). Parametric and non-parametric extensions to unfolding response models.PhD thesis, Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213, USA.
16 Johnson, M.S. (2004). Nonparametric estimation of item and respondent locations from unfolding-type items. Technical report. Baruch College, Department of Statistics & Computer Information Systems, New York: Available for downloand at stat.baruch.cuny.edu/johnson/NPUF.pdf.
17 Johnson, M.S., & Junker, B.W. (2003). Using data augmentation and Markov chain Monte Carlo for the estimation of unfolding response models. Journal of Educational and Behavioral Statistics, 28(3), 195–230.
18 Junker, B.W. (1991). Essential independence and likelihood-based ability estimation for polytomous items. Psychometrika, 56, 255–278.
19 Junker, B.W. (1993). Conditional association, essential independence and monotone unidimensional item response models. Annals of Statistics, 21, 1359–1378.
20 Junker, B.W., & Sijtsma, K. (2001). Nonparametric item response theory in action: An overview of the special issue. Applied Psychological Measurement, 25, 211–220.
21 Karlin, S. (1968). Total positivity Vol. 1. Stanford, CA: Stanford University Press.
22 Luo, G. (1998). A general formulation for unidimensional latent trait unfolding models: Making explicit the latitude of acceptance. Journal of Mathematical Psychology, 42, 400–417.
23 Maris, E. (1995). Psychometric latent response models. Psychometrika, 60, 523–547.
24 Maris, G., & Maris, E. (2002). Are attitude items monotone or single-peaked? An analysis using Bayesian methods. Technical Report 2002-02, Arnhem: Citgogroep Measurement and Research Department.
25 Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
26 Mokken, R.J. (1971). A theory and procedure of scale analysis. New York: De Gruyter.
27 Muhlberger, P. (1999).A general unfolding, non-folding scaling model and algorithm. Presented at the 1999 American Political Science Association Annual Meeting, Atlanta, GA.
28 Noël, Y. (1999). Recovering unimodal latent patterns of change by unfolding analysis: Application to smoking cessation. Psychological Methods, 4(2), 173–191.
29 Post, W.J. (1992). Nonparametric unfolding models: a latent structure approach. M&T Series, Leiden, The Netherlands: DSWO Press.
30 Ramsay, J.O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630.
Ripley, B.D. (1987). Stochastic simulaton. New York: Wiley.
31 Roberts, J.S., Donoghue, J.R., & Laughlin, J.E. (1999). A general model for unfolding unidimensional polytomous responses using item response theory. Applied Psychological Measurement.
32 Roberts, J.S., Donoghue, J.R., & Laughlin, J.E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3–32.
33 Rosenbaum, P. (1987). Comparing item characteristic curves. Psychometrika, 52, 217–233.
34 Sijtsma, K. (1998). Methodology review: nonparametric IRT approaches to the analysis of dichotomous item scores. Applied Psychological Measurement, 22(1), 3–31.
35 Sijtsma, K., & Junker, B.W. (1996). A survey of theory and methods of invariant item ordering, with results for parametric models. British Journal of Mathematical and Statistical Psychology, 49, 79–105.
36 Stout, W.F. (1990). A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika, 55, 293–325.
37 Thurstone, L.L. (1927). A law of comparative judgment. Psychological Review, 34, 278–286.
38 Thurstone, L.L. (1928). Attitudes can be measured. American Journal of Sociology, 33, 529–554.
39 van Schuur, W.H. (1988). Stochastic unfolding. In W.E. Saris, & I.N. Gallhofer, (Eds.), Sociometric research, volume I: Data collection and scaling(vol. 1,chap. 9, p. 137–157). London: Macmillan.
40 Verhelst, N.D., & Verstralen, H.H.F.M. (1993). A stochastic unfolding model derived from the partial credit model, Kwantitative Methoden, 42, 73–92.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Eggen, T.J.H.M., Verhelst, N.D. Loss of Information in Estimating Item Parameters in Incomplete Designs. Psychometrika 71, 303–322 (2006). https://doi.org/10.1007/s11336-004-1205-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-004-1205-6