Differential Item Functioning of Psychasthenia and Psychopathic Deviate Items in Entrance Employment Tests: A Comparison between the Mantel-Haenszel and Rasch Model

Subject Areas : Psychology

Mehdi Rahmani malek abad ¹ , Mohammad Reza Falsafinejad ² , Asghar Minaei ³ , Nourali Farrokhi ⁴

1 - Assessment and Measurement, Faculty of Psychology and Educational Sciences, Allameh Tabataba’i University, Tehran, Iran
2 - Associate Professor, Faculty of Psychology and Educational Sciences, Allameh Tabataba’i University, Tehran, Iran
3 - Associate Professor, Faculty of Psychology and Educational Sciences, Allameh Tabataba’i University, Tehran, Iran
4 - Associate Professor, Faculty of Psychology and Educational Sciences, Allameh Tabataba’i University, Tehran, Iran

Received: 2023-09-17 Accepted : 2023-10-11 Published : 2024-12-11

Keywords: differential item functioning, Rasch model, Mantel-Haenszel, psychopathic deviate, psychasthenia, recruitment candidates.,

Abstract :

Choosing healthy human resources is one of the main tasks of recruitment tests, provided that the measurement tools are fairness in test. Methods: This research is survey research and the statistical population was all the Recruitment candidates of Isfahan city. The sample size was 5997 people who were selected by one-step cluster method. The tools used was the MMPI-2. The fit of the Items with the model was confirmed using outfit MnSq and infit MnSq, and DIF according to gender was checked and compared with Rasch Model and Mantel-Haenszel by Winsteps version 3.64. Results: After fitting the data with Rasch model, DIF analysis was performed. The results showed that 38% of psychopathic deviate items and 22.9% of psychasthenia items were biased towards men. Mantel-Haenszel had a weaker performance than Rasch Model in some items. Conclusion: Psychopathic deviate and psychasthenia and items have been to the bias of men and they have naturally obtained higher scores in these two scales. Also, due to the strength of the Rasch model compared to the CTT, it is better to use this model to detect biased items

References:

Antikchi. E, Bigdeli. I. A. & Sabahi. P. (2017). The comparision of neuropsychological index related to executive functions in antisocial personality disorder, obsessive-compulsive personality disorder and normal people. Advances in Cognitive Science, Vol. 19, No. 1. [Persian]
Baker. C. A, Baum. L. J & Francis. J. C. (2023). Assessment of test bias on the MMPI-2-RF higher order and restructured clinical scales as a function of gender and race. Professional Psychology Research and Practice. 54 (4) Follow journal. DOI: 10.1037/pro0000517.
Bethune M. M. (2011). Predictors of Performance in a Professional Counselor Masters Program, Doctoral Thesis http://gradworks.umi.com/34/93/3493607.html.
Björkqvist, K. (2018). Gender differences in aggression. Current Opinion in Psychology. Volume 19, February, Pages 39-42.
Buribayev. Y. A and Khamzina Z. A. (2019). Gender equality in employment: The experience of Kazakhstan. International Journal of Discrimination and the Law, Vol. 19(2) 110–124.
Camilli, G. (2006). Test fairness. In R. Brennan (Ed.), Educational measurement (4th ed.) (pp. 221-256). New York: American Council on Education & Praeger series on higher education.
Camilli, G., & Congdon, P. (1999). Application of a method of estimating DIF for polytomous test items. Journal of Educational and Behavioral Statistics, 24(4), 323-341.
Camilli, G., & Shepard, L. (1994). Methods for identifying biased test items. Newbury Park, CA Sage.
Cao. W, Li. P. Vander Wal. R.C & Taris. T.W. (2022). Leadership and Workplace Aggression: A Meta-analysis. Journal of Business Ethics. Published: 15 July. volume 186, pages347–36
Card N. A, Stucky. B. D, Sawalani. G. M & Little. T. D. (2008). Direct and indirect aggression during childhood and adolescence: a meta-analytic review of gender differences, intercorrelations, and relations to maladjustment. Sep-Oct;79(5): 1185-229. doi: 10.1111/j.1467-8624.2008.01184.x.
Chapelle, C. A. (2020). Validity in language assessment. The Routledge Handbook of Second Language Acquisition and Language Testing, 11.
Chen, M. Y., Liu, Y., & Zumbo, B. D. (2020). A propensity score method for investigating differential item functioning in performance assessment. Educational and Psychological Measurement, 80(3), 476-498.
Cherry Kendra. (2021) The Minnesota Multiphasic Personality Inventory (MMPI). Updated on September 02 Medically reviewed by Amy Morin, LCSW.
Embretson, S. E., and Reise, S. P. (2000). Item Response Theory for Psychologists. Mahwah, NJ: Lawrence Erlbaum Associates. 11, 57- 74.
Gori. E & Marin. R. F. (2015). Rasch Analysis of some MMPI-2 scales in a sample of university freshmen. International Journal of Arts & Sciences. 1944-6934 :: 08(03):107–150.
Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods. Applied Measurement in Education, 2(4), 313-334.
Hambleton, Ronald K.; Saminathan, H.; Rogers, H.. Jane (1991). Fundamentals of Question-Answer Theory. Translated by Mohammad Reza Filsafinejad (2010). Tehran: Allameh Tabatabai University. [Persian]
Karami, H. (2013) The quest for fairness in language testing. Educational Research and Evaluation, 19(2&3), 158-169. [Persian]
Karami, Hossein & Khodi, Ali (2021). Differential Item Functioning and Test Performance: a Comparison Between the Rasch Model, Logistic Regression and Mantel-Haenszel. Journal of Foreign Language Research, 10 (4), 842-853. [Persian]
Karami. H. R, Gramipour. M & Minaei. A. (2021). Differential Item Functioning (DIF) Detection Rate Using Rasch Trees Model: A Simulated and Real Data Study of the NAJA High Stakes Tests. Educational Measurement. 11(44), 1-30. [Persian]
Lee JC, Zhang Z, Yin H. (2010). Using multidimensional Rasch analysis to validate the Chinese version of the Motivated Strategies for Learning Questionnaire (MSLQ-CV). Eur J Psychol Educ; 25.
Linacre, J. M. (2010). A User's Guide to WINSTEPS®. Retrieved May 2, from http://www.winsteps.com/ .
Lord, F. M.y. & Novick, MR. (1968). Statistical Theory of Mental Test Scores. Addison-Wesley Publishing Company.
Marnat, G. G. & Wright, A. J. (2016). Handbook of Psychological Assessment. 6th ed.
McNulty J. L., Forbey J. D., Graham J. R., Ben-Porath Y. S., Black M. S., Anderson S. V. & Burlew A. K. (2015). MMPI-2 Validity Scale Characteristics in a Correctional Sample. Sage Journals, Volume 10, No. 3, September. 288-298.
Minaei. A. (2015). Aapplication of Rasch measurement model to evaluate measurement properties of the Test of Visual- Motor Skills-Revised. Journal of Educational Measurement. Volume 5, Issue 18 - Serial Number 18. January. 77-114. [Persian]
Penfield, R. D. & Algina, J. (2003). Applying the Liu–Agresti estimator of the cumulative common odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40: 353–370.
Queirolo L, Bacci C, Roccon A, Zanette G and Mucignat C. (2023). Anxiety in a regular day of work: A 24 hour psychophysiological investigation in young dentists with gender comparison. Front Psychol. 14:1045974. doi: 10.3389/fpsyg.1045974
Siegert, R. J., Tennant, A., & Turner-Stokes, L. (2010). Rasch analysis of the Beck Depression Inventory-II in a neurological rehabilitation sample. Disability and Rehabilitation, 32(1), 8–17.
Skrondal, A., Rabe-Hesketh, S., and Boca Raton, F. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models: Chapman & Hall/ CRC Press.
Soltani Shal R, Saadatbin Javaheri F, Zebardast F. (2020). Survey the level of well-being and Psychometric characteristics of hospital nurses’ well-being at work scale. Occupational Medicine Quarterly Journal:12(1): 55-68. [Persian]
Su. Y & Wang N. (2005). Use of the Rasch IRT model in standard setting: An item‐ mapping method. Journal of Educational Measurement. Sep; 40 (3): 231- 53.
Talerico G. M, McCallum. J. J, Whitman M. R, Tarescavage. M, Corey D. M & Ben-Porath. Y. B. (2023): Comparing the Validity of MMPI-3 Scores in Prehire Psychological Screenings of Male and Female Police Officer Candidates, Journal of Personality Assessment, DOI: 10.1080/00223891.2023.2191278
Tennant, A., & Conaghan, P.G. (2007). The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis & Rheumatism journal. 57, 1358– 1362.
Valianpour. Z, Modarres Gharavi. M & Mahram. B. (2020). Validation of the Minnesota Multiphasic Personality Inventory (MMPI 2) in psychiatric patients and non-patient individuals in Mashhad city, Iran. Journal of Fundamentals of Mental Health, Nov-Dec. [Persian]
Wiberg, M. (2007). Measuring and detecting differential item functioning in criterion-referenced licensing test: A theoretic comparison of methods. Educational Measurement, technical report No. 2.
Zahid. Z, nasrollahi. Z & Mahinizadeh. M. (2020). From Gender Discrimination to Equality and Economic Growth (Study of Developing Countries). Journal of Woman in Development and Politics. 662- 643. [Persian]
Zumbo, B. D. (2003). Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests. Language testing, 20(2), 136-147.