ارزشیابی مقالات علمی علوم تربیتی به منظور شناسایی اشتباهات رایج آماری
محورهای موضوعی : روانشناسی تربیتیغلامرضا یادگارزاده 1 , عادل فاطمی 2
1 - استادیار سازمان سنجش آموزش کشور
2 - استادیار گروه آمار، واحد سنندج، دانشگاه آزاد اسلامی، سنندج، ایران
کلید واژه: ارزشیابی, پژوهش, اشتباه (خطای) آماری, مقالة علمی- پژوهشی, مجلة علمی-پژوهشی, آمار پارامتری و ناپارامتری,
چکیده مقاله :
هدف از این پژوهش شناسایی اشتباهات آماری در مقالات منتشر شده در حوزة علوم تربیتی است. برای این منظور از جامعة مجلات علوم تربیتی تعداد 92 مقاله از 12 شماره به صورت خوشه‌ای دو مرحله‌ای انتخاب و مورد بررسی قرار گرفت. طرح پژوهش از نوع تحلیل مستندات و روش تحلیل داده و اطلاعات به روش مقایسه‌های چندگانه به کمک آزمون فرض مبتنی بر فاصله اطمینان نمودارهای خطا بود. نتایج نشان داد از 92 مقاله مورد بررسی 39 مقاله (42%) دارای حداقل یک اشتباه آماری بوده‌اند و در بین این مقالات 49 اشتباه رایج مشاهده گردید که شامل 9 نوع اشتباه، برازش مدل رگرسیون بدون بررسی فرضیات باقی مانده‌های مدل، استفاده از آزمون T مستقل بدون بررسی همگنی واریانس‌ها (لِون)، برازش غلط مدل معادلات ساختاری، روش نمونه گیری اُریب، استفاده از تحلیل واریانس به جای آزمون T، استفاده از روش ناپارامتریک فریدمن بدون بررسی وضعیت توزیع داده ها، عدم همخوانی نتایج تحلیلهای آماری، استفاده از آزمون T مستقل به جای آزمون T وابسته، تفسیر نتایج آزمون لِون به جای نتایج آزمون T، انجام آزمون تعقیبی بدون انجام تحلیل واریانس و نهایتا تحلیل معنی‌داری آماری تنها براساس نتایج آمار توصیفی به ترتیب فراوانی بود. نتایج همچنین نشان داد بیشترین فراوانی اشتباهات آماری مربوط به مقالات مستخرج از پایان‌نامه‌های کارشناسی ارشد، وابستگی سازمانی مربوط به دانشگاه آزاد اسلامی، مرتبة علمی مربی، افراد غیر متخصص علوم تربیتی، مجله‌های دارای سابقة انتشار زیر 20 شماره و دارای 3 نفر نویسنده بوده است.
The goal of this research is to identify the statistical errors in the articles published in the field of Educational Sciences. For this purpose, by using a two-stage clustering sampling method, 92 articles were selected out of 12 issues of the journals in Educational Sciences studied. The research design was based on document analysis and the method for analyzing the data was according to multiple comparisons by the use of error bars. The results showed that 39 articles out of 92 ones had at least one statistical error, and among this article, there were 49 common error including, error types listed based on frequency of occurrence: "Fitting the regression model without examining the assumptions of the residuals, Use of independent T-test without examining the homogeneity of variances (levene's test), false fitting of the structural equations model, Biased sampling method, Using the analysis of variance instead of T-test, Using Friedman's nonparametric test without considering the distribution of data, Mismatch results of the statistical analysis, Using the independent T-test instead of paired one, Interpreting the results of Levene's test instead of the results of T-test, Performing the follow-up test without performing the analysis of variance, and finally Analysis of statistical significance just based on the results of descriptive statistics". The results also showed that the highest frequency of errors is related to the extracted articles out of master's theses, organizational affliction related to Islamic Azad University, academic degree of the instructor, non-specialists in Educational Sciences, journals with a history of publication below 20 years and article with 3 authors.
Borg, W. R., & Gall, M. D. (2003). Educational research: An introduction. Longman Publishing.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27-40.
Casadevall, A., Steen, R. G., & Fang, F. C. (2014). Sources of error in the retracted scientific literature. The FASEB Journal, 28(9), 3847-3855.
Ercan, I., Karadeniz, P. G., Cangur, S., Ozkaya, G., & Demirtas, H. (2015). Examining of Published Articles with Respect to Statistical Errors in Medical Sciences. International Journal of Hematology and Oncology, 27(1), 130-138.
Erdogan, M., Kaplan, H., Kayir, C. G., Asik, U. O., & Akbunar, S. (2012). Common method-related problems in selected research studies on educational sciences in Turkey. Procedia-Social and Behavioral Sciences, 46, 3230-3234.
Fraser B.J., & Wong A. F. L. (2003). Training of Educational Research Workers. In: Keeves J.P. et al. (eds), International Handbook of Educational Research in the Asia-Pacific Region. Springer International Handbooks of Education, vol 11.
Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and mixed approaches. Sage.
Harnqvist, K. (1997). Training of Research Workers in Education. In: Keeves J.P. (eds) Educational research, methodology and measurement: an international handbook, Second edition. Oxford, UK; Tarrytown, N. Y.: Pergamon
Keeves, J. P., & Watanabe, R. (Eds.). (2013). The international handbook of educational research in the Asia-Pacific region (Vol. 11). Springer Science & Business Media.
Lapanachokdee, W., Lawthong, N., & Piyapimonsit, C. (2016). Critiques on Thai Educational Research Methodology. Asian Social Science, 12(3), 93.
Levin, J. R. (1994). Crafting educational intervention research that's both credible and creditable. Educational Psychology Review, 6(3), 231-243.
Miller, D. W. (1999). The black hole of education research. Chronicle of Higher Education, 45(48), A17-A18.
Nuijten, M. B., Hartgerink, C. H., van Assen, M. A., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior research methods, 48(4), 1205-1226.
McMillan, J. H. (1996). Educational research: Fundamentals for the consumer. HarperCollins College Publishers, 10 East 53rd Street, New York, NY 10022; World Wide Web: http://www. harpercollins. com/college.
Picho, K., & Artino Jr, A. R. (2016). 7 deadly sins in educational research.
Ross, K. N. (1979). An empirical investigation of sampling errors in educational survey research. Journal of Educational Statistics, 4(1), 24-40.
Sotos, A. E. C., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2(2), 98-113.
Strasser, N. (2011). Avoiding statistical mistakes. Journal of College Teaching & Learning (TLC), 4(7).
Strasak, A. M., Zaman, Q., Pfeiffer, K. R., Gobel, G., & Ulmer, H. (2007). Statistical errors in medical research--a review of common pitfalls. Swiss medical weekly, 137(3-4), 44-49.
Sayer, A. (2010). Method in social science: revised 2nd edition. Routledge.
Scheaffer, R., Aliaga, M., Diener-West, M., Garfield, J., Higgins, T., Hilton, S., ... & Lester, F. K. (2007). Using statistics effectively in mathematics education research. Alexandria, VA: American Statistical Association.
Thompson, B. (1995). Inappropriate Statistical Practices in Counseling Research: Three Pointers for Readers of Research Literature. ERIC Digest.
Thompson, B. (1998). Five Methodology Errors in Educational Research: The Pantheon of Statistical Significance and Other Faux Pas.
Thompson, B. (1999). Common Methodology Mistakes in Educational Research, Revisited, along with a Primer on Both Effect Sizes and the Bootstrap.
Veldkamp, C. L., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A., & Wicherts, J. M. (2014). Statistical reporting errors and collaboration on statistical analyses in psychological science. PloS one, 9(12), e114876.
Walliman, N. (2011). Your research project: Designing and planning your work. Sage
_||_Borg, W. R., & Gall, M. D. (2003). Educational research: An introduction. Longman Publishing.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27-40.
Casadevall, A., Steen, R. G., & Fang, F. C. (2014). Sources of error in the retracted scientific literature. The FASEB Journal, 28(9), 3847-3855.
Ercan, I., Karadeniz, P. G., Cangur, S., Ozkaya, G., & Demirtas, H. (2015). Examining of Published Articles with Respect to Statistical Errors in Medical Sciences. International Journal of Hematology and Oncology, 27(1), 130-138.
Erdogan, M., Kaplan, H., Kayir, C. G., Asik, U. O., & Akbunar, S. (2012). Common method-related problems in selected research studies on educational sciences in Turkey. Procedia-Social and Behavioral Sciences, 46, 3230-3234.
Fraser B.J., & Wong A. F. L. (2003). Training of Educational Research Workers. In: Keeves J.P. et al. (eds), International Handbook of Educational Research in the Asia-Pacific Region. Springer International Handbooks of Education, vol 11.
Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and mixed approaches. Sage.
Harnqvist, K. (1997). Training of Research Workers in Education. In: Keeves J.P. (eds) Educational research, methodology and measurement: an international handbook, Second edition. Oxford, UK; Tarrytown, N. Y.: Pergamon
Keeves, J. P., & Watanabe, R. (Eds.). (2013). The international handbook of educational research in the Asia-Pacific region (Vol. 11). Springer Science & Business Media.
Lapanachokdee, W., Lawthong, N., & Piyapimonsit, C. (2016). Critiques on Thai Educational Research Methodology. Asian Social Science, 12(3), 93.
Levin, J. R. (1994). Crafting educational intervention research that's both credible and creditable. Educational Psychology Review, 6(3), 231-243.
Miller, D. W. (1999). The black hole of education research. Chronicle of Higher Education, 45(48), A17-A18.
Nuijten, M. B., Hartgerink, C. H., van Assen, M. A., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior research methods, 48(4), 1205-1226.
McMillan, J. H. (1996). Educational research: Fundamentals for the consumer. HarperCollins College Publishers, 10 East 53rd Street, New York, NY 10022; World Wide Web: http://www. harpercollins. com/college.
Picho, K., & Artino Jr, A. R. (2016). 7 deadly sins in educational research.
Ross, K. N. (1979). An empirical investigation of sampling errors in educational survey research. Journal of Educational Statistics, 4(1), 24-40.
Sotos, A. E. C., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2(2), 98-113.
Strasser, N. (2011). Avoiding statistical mistakes. Journal of College Teaching & Learning (TLC), 4(7).
Strasak, A. M., Zaman, Q., Pfeiffer, K. R., Gobel, G., & Ulmer, H. (2007). Statistical errors in medical research--a review of common pitfalls. Swiss medical weekly, 137(3-4), 44-49.
Sayer, A. (2010). Method in social science: revised 2nd edition. Routledge.
Scheaffer, R., Aliaga, M., Diener-West, M., Garfield, J., Higgins, T., Hilton, S., ... & Lester, F. K. (2007). Using statistics effectively in mathematics education research. Alexandria, VA: American Statistical Association.
Thompson, B. (1995). Inappropriate Statistical Practices in Counseling Research: Three Pointers for Readers of Research Literature. ERIC Digest.
Thompson, B. (1998). Five Methodology Errors in Educational Research: The Pantheon of Statistical Significance and Other Faux Pas.
Thompson, B. (1999). Common Methodology Mistakes in Educational Research, Revisited, along with a Primer on Both Effect Sizes and the Bootstrap.
Veldkamp, C. L., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A., & Wicherts, J. M. (2014). Statistical reporting errors and collaboration on statistical analyses in psychological science. PloS one, 9(12), e114876.
Walliman, N. (2011). Your research project: Designing and planning your work. Sage