Risk of Metropolis-Hastings Robbins-Monro algorithm in multidimensional models of item-response theory in binary data considering test length
Subject Areas : Researchmehdi molaei yasavoli 1 , ali delavar 2 , Mohammad Asgari 3 , Jalil Younesi 4 , vahid rezaei tabar 5
1 - PhD Student in Measurement and Assesment, Faculty of Psychology and Educational Sciences, Allameh Tabatabai University, Tehran, Iran
2 - Professor, Department of Measurement and assesment, Faculty of Psychology and Educational Sciences, Allameh Tabatabai University, Tehran, Iran
3 - Mohammad Asgari is Associate Professor of --- in the Department of Deliberation and Measuring , Faculty of psychology and educational sceinces .
4 - Department of Deliberation and Measuring, Faculty of Psychology and Educational Sciences, Allameh Tabataba’i University, Tehran, Iran.
5 - Associate Professor, Department of Statistics, Faculty of Statistics, Mathematics and Computer, Allameh Tabatabai University, Tehran, Iran.
Keywords: Risk, MHRM algorithm, multidimensional models of item-response theory, binary data, test length,
Abstract :
The present study was conducted with the aim of investigating the risk of MHRM algorithm in multi-dimensional models of item-response theory in binary data, taking into account different test dimensions and lengths. The research method used was a real experiment using a multi-group post-test design. The studied sample was created based on simulation studies under different conditions of independent variables in 27 modes with 100 repetitions for each. The model used was the two-parameter multidimensional model of logistics and the investigated parameters were the slope and difficulty of the items. In order to check the risk of each of the parameters in different experimental conditions, the average squared error index was used. R statistical software packages mirt, interactions, car and psych were used for data generation and analysis. The results of the research showed that the MHRM algorithm has less risk compared to the EM and MCEM algorithms. This issue was especially evident under the conditions of high dimensional data (5 dimensions) and short test length (15 questions). Also, the results of the research showed that when the dimensions of the test increase and the length of the test decreases, the risk of parameter estimation increases significantly. As a result, it can be said that the application of the MHRM algorithm in data with a high number of dimensions and a short test length is necessary, and researchers are advised to use it in the analysis of data with a complex structure such as a high number of dimensions
دلاور، علی (۱۴۰۱). کتاب احتمالات و آمار کاربردی در روان شناسی و علوم تربیتی. تهران: نشر رشد.
Asparouhov, T., & Muthén, B. (2012). General random effect latent variable modeling: Random subjects, items, contexts, and parameters. In annual meeting of the National Council on Measurement in Education, Vancouver, British Columbia.
Bartolucci, F., Bacci, S., & Gnaldi, M. (2015). Statistical analysis of questionnaires: A unified approach based on R and Stata (Vol. 34). CRC Press.
Bashkov, B. M., & DeMars, C. E. (2017). Examining the performance of the Metropolis–Hastings Robbins–Monro algorithm in the estimation of multilevel multidimensional IRT models. Applied psychological measurement, 41(5): 323-337.
Bulut, O., & SÜNBÜL, Ö. (2017). Monte Carlo Simulation Studies in Item Response Theory with the R Programming Language R Programlama Dili ile Madde Tepki Kuramında Monte Carlo Simülasyon Çalışmaları. Journal of Measurement and Evaluation in Education and Psychology, 8(3): 266-287.
Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis–Hastings Robbins–Monro algorithm. Psychometrika, 75(1): 33-57.
Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahway.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1): 1-22.
Delavar, A. (2021). Applied probability and statistics in psychology and educational sciences. Tehran: Roshd Published [In Persian].
Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.
Gibbons, R. D., Weiss, D. J., Frank, E., & Kupfer, D. (2016). Computerized adaptive diagnosis and testing of mental health disorders. Annual Review of Clinical Psychology, 12, 83-104.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1985). Principles and applications of item response theory.
Han, K. T., & Hambleton, R. K. (2014). User's Manual for WinGen 3: Windows Software that Generates IRT Model Parameters and Item Responses (Center for Educational Assessment Report No. 642). Amherst, MA: University of Massachusetts.
Liu, Q., & Pierce, D. A. (1994). A note on Gauss—Hermite quadrature. Biometrika, 81(3), 624-629.
Kuo, F. Y., & Sloan, I. H. (2005). Lifting the curse of dimensionality. Notices of the AMS, 52(11), 1320-1328.
Kasim, M. F., Bott, A. F. A., Tzeferacos, P., Lamb, D. Q., Gregori, G., & Vinko, S. M. (2019). Retrieving fields from proton radiography without source profiles. Physical Review E, 100(3): 033208.
Kuo, T. C., & Sheng, Y. (2016). A comparison of estimation methods for a multi-unidimensional graded response IRT model. Frontiers in psychology, 7, 880.), 1-29.
Lesaffre, E., & Spiessens, B. (2001). On the effect of the number of quadrature points in a logistic random effects model: an example. Journal of the Royal Statistical Society: Series C (Applied Statistics), 50(3): 325-335.
Linden, W. J., & van der, & Hambleton, RK (1997). Handbook of modern item response theory, 9-39.
Meng, X. L., & Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. Journal of the American Statistical Association, 91(435): 1254-1267.
Naylor, J. C., & Smith, A. F. (1982). Applications of a method for the efficient computation of posterior distributions. Journal of the Royal Statistical Society: Series C (Applied Statistics), 31(3): 214-225.
Patz, R. J., & Junker, B. W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of educational and behavioral statistics, 24(4): 342-366.
Robbins, H., & Monro, S. (1951). A stochastic approximation method. The annals of mathematical statistics, 400-407.
Yang, J. S., & Cai, L. (2014). Estimation of contextual effects through nonlinear multilevel latent variable modeling with a Metropolis–Hastings Robbins–Monro algorithm. Journal of Educational and Behavioral Statistics, 39(6): 550-582.
Sahin, A., & Anil, D. (2017). The effects of test length and sample size on item parameters in item response theory.
Patsula, L. (1995). A comparison of item parameter estimates and ICCs produced with TESTGRAF and BILOG under different test lengths and sample sizes. University of Ottawa (Canada).
_||_Asparouhov, T., & Muthén, B. (2012). General random effect latent variable modeling: Random subjects, items, contexts, and parameters. In annual meeting of the National Council on Measurement in Education, Vancouver, British Columbia.
Bartolucci, F., Bacci, S., & Gnaldi, M. (2015). Statistical analysis of questionnaires: A unified approach based on R and Stata (Vol. 34). CRC Press.
Bashkov, B. M., & DeMars, C. E. (2017). Examining the performance of the Metropolis–Hastings Robbins–Monro algorithm in the estimation of multilevel multidimensional IRT models. Applied psychological measurement, 41(5): 323-337.
Bulut, O., & SÜNBÜL, Ö. (2017). Monte Carlo Simulation Studies in Item Response Theory with the R Programming Language R Programlama Dili ile Madde Tepki Kuramında Monte Carlo Simülasyon Çalışmaları. Journal of Measurement and Evaluation in Education and Psychology, 8(3): 266-287.
Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis–Hastings Robbins–Monro algorithm. Psychometrika, 75(1): 33-57.
Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahway.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1): 1-22.
Delavar, A. (2021). Applied probability and statistics in psychology and educational sciences. Tehran: Roshd Published [In Persian].
Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.
Gibbons, R. D., Weiss, D. J., Frank, E., & Kupfer, D. (2016). Computerized adaptive diagnosis and testing of mental health disorders. Annual Review of Clinical Psychology, 12, 83-104.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1985). Principles and applications of item response theory.
Han, K. T., & Hambleton, R. K. (2014). User's Manual for WinGen 3: Windows Software that Generates IRT Model Parameters and Item Responses (Center for Educational Assessment Report No. 642). Amherst, MA: University of Massachusetts.
Liu, Q., & Pierce, D. A. (1994). A note on Gauss—Hermite quadrature. Biometrika, 81(3), 624-629.
Kuo, F. Y., & Sloan, I. H. (2005). Lifting the curse of dimensionality. Notices of the AMS, 52(11), 1320-1328.
Kasim, M. F., Bott, A. F. A., Tzeferacos, P., Lamb, D. Q., Gregori, G., & Vinko, S. M. (2019). Retrieving fields from proton radiography without source profiles. Physical Review E, 100(3): 033208.
Kuo, T. C., & Sheng, Y. (2016). A comparison of estimation methods for a multi-unidimensional graded response IRT model. Frontiers in psychology, 7, 880.), 1-29.
Lesaffre, E., & Spiessens, B. (2001). On the effect of the number of quadrature points in a logistic random effects model: an example. Journal of the Royal Statistical Society: Series C (Applied Statistics), 50(3): 325-335.
Linden, W. J., & van der, & Hambleton, RK (1997). Handbook of modern item response theory, 9-39.
Meng, X. L., & Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. Journal of the American Statistical Association, 91(435): 1254-1267.
Naylor, J. C., & Smith, A. F. (1982). Applications of a method for the efficient computation of posterior distributions. Journal of the Royal Statistical Society: Series C (Applied Statistics), 31(3): 214-225.
Patz, R. J., & Junker, B. W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of educational and behavioral statistics, 24(4): 342-366.
Robbins, H., & Monro, S. (1951). A stochastic approximation method. The annals of mathematical statistics, 400-407.
Yang, J. S., & Cai, L. (2014). Estimation of contextual effects through nonlinear multilevel latent variable modeling with a Metropolis–Hastings Robbins–Monro algorithm. Journal of Educational and Behavioral Statistics, 39(6): 550-582.
Sahin, A., & Anil, D. (2017). The effects of test length and sample size on item parameters in item response theory.
Patsula, L. (1995). A comparison of item parameter estimates and ICCs produced with TESTGRAF and BILOG under different test lengths and sample sizes. University of Ottawa (Canada).
