Evaluation of Educational Scientific Articles with the Purpose of Identification Common Statistical Errors
Subject Areas : Educational Psychology
Golamreza
Yadegarzadeh
1
(Tehran)
Adel
Fatemi
2
(Sanandaj)
Keywords: Evaluation, Research Paper, Research, Statistical errors, Peer review Journals, Parametric and nonparametric statistics,
Abstract :
The goal of this research is to identify the statistical errors in the articles published in the field of Educational Sciences. For this purpose, by using a two-stage clustering sampling method, 92 articles were selected out of 12 issues of the journals in Educational Sciences studied. The research design was based on document analysis and the method for analyzing the data was according to multiple comparisons by the use of error bars. The results showed that 39 articles out of 92 ones had at least one statistical error, and among this article, there were 49 common error including, error types listed based on frequency of occurrence: "Fitting the regression model without examining the assumptions of the residuals, Use of independent T-test without examining the homogeneity of variances (levene's test), false fitting of the structural equations model, Biased sampling method, Using the analysis of variance instead of T-test, Using Friedman's nonparametric test without considering the distribution of data, Mismatch results of the statistical analysis, Using the independent T-test instead of paired one, Interpreting the results of Levene's test instead of the results of T-test, Performing the follow-up test without performing the analysis of variance, and finally Analysis of statistical significance just based on the results of descriptive statistics". The results also showed that the highest frequency of errors is related to the extracted articles out of master's theses, organizational affliction related to Islamic Azad University, academic degree of the instructor, non-specialists in Educational Sciences, journals with a history of publication below 20 years and article with 3 authors.
Borg, W. R., & Gall, M. D. (2003). Educational research: An introduction. Longman Publishing.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27-40.
Casadevall, A., Steen, R. G., & Fang, F. C. (2014). Sources of error in the retracted scientific literature. The FASEB Journal, 28(9), 3847-3855.
Ercan, I., Karadeniz, P. G., Cangur, S., Ozkaya, G., & Demirtas, H. (2015). Examining of Published Articles with Respect to Statistical Errors in Medical Sciences. International Journal of Hematology and Oncology, 27(1), 130-138.
Erdogan, M., Kaplan, H., Kayir, C. G., Asik, U. O., & Akbunar, S. (2012). Common method-related problems in selected research studies on educational sciences in Turkey. Procedia-Social and Behavioral Sciences, 46, 3230-3234.
Fraser B.J., & Wong A. F. L. (2003). Training of Educational Research Workers. In: Keeves J.P. et al. (eds), International Handbook of Educational Research in the Asia-Pacific Region. Springer International Handbooks of Education, vol 11.
Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and mixed approaches. Sage.
Harnqvist, K. (1997). Training of Research Workers in Education. In: Keeves J.P. (eds) Educational research, methodology and measurement: an international handbook, Second edition. Oxford, UK; Tarrytown, N. Y.: Pergamon
Keeves, J. P., & Watanabe, R. (Eds.). (2013). The international handbook of educational research in the Asia-Pacific region (Vol. 11). Springer Science & Business Media.
Lapanachokdee, W., Lawthong, N., & Piyapimonsit, C. (2016). Critiques on Thai Educational Research Methodology. Asian Social Science, 12(3), 93.
Levin, J. R. (1994). Crafting educational intervention research that's both credible and creditable. Educational Psychology Review, 6(3), 231-243.
Miller, D. W. (1999). The black hole of education research. Chronicle of Higher Education, 45(48), A17-A18.
Nuijten, M. B., Hartgerink, C. H., van Assen, M. A., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior research methods, 48(4), 1205-1226.
McMillan, J. H. (1996). Educational research: Fundamentals for the consumer. HarperCollins College Publishers, 10 East 53rd Street, New York, NY 10022; World Wide Web: http://www. harpercollins. com/college.
Picho, K., & Artino Jr, A. R. (2016). 7 deadly sins in educational research.
Ross, K. N. (1979). An empirical investigation of sampling errors in educational survey research. Journal of Educational Statistics, 4(1), 24-40.
Sotos, A. E. C., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2(2), 98-113.
Strasser, N. (2011). Avoiding statistical mistakes. Journal of College Teaching & Learning (TLC), 4(7).
Strasak, A. M., Zaman, Q., Pfeiffer, K. R., Gobel, G., & Ulmer, H. (2007). Statistical errors in medical research--a review of common pitfalls. Swiss medical weekly, 137(3-4), 44-49.
Sayer, A. (2010). Method in social science: revised 2nd edition. Routledge.
Scheaffer, R., Aliaga, M., Diener-West, M., Garfield, J., Higgins, T., Hilton, S., ... & Lester, F. K. (2007). Using statistics effectively in mathematics education research. Alexandria, VA: American Statistical Association.
Thompson, B. (1995). Inappropriate Statistical Practices in Counseling Research: Three Pointers for Readers of Research Literature. ERIC Digest.
Thompson, B. (1998). Five Methodology Errors in Educational Research: The Pantheon of Statistical Significance and Other Faux Pas.
Thompson, B. (1999). Common Methodology Mistakes in Educational Research, Revisited, along with a Primer on Both Effect Sizes and the Bootstrap.
Veldkamp, C. L., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A., & Wicherts, J. M. (2014). Statistical reporting errors and collaboration on statistical analyses in psychological science. PloS one, 9(12), e114876.
Walliman, N. (2011). Your research project: Designing and planning your work. Sage
_||_Borg, W. R., & Gall, M. D. (2003). Educational research: An introduction. Longman Publishing.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27-40.
Casadevall, A., Steen, R. G., & Fang, F. C. (2014). Sources of error in the retracted scientific literature. The FASEB Journal, 28(9), 3847-3855.
Ercan, I., Karadeniz, P. G., Cangur, S., Ozkaya, G., & Demirtas, H. (2015). Examining of Published Articles with Respect to Statistical Errors in Medical Sciences. International Journal of Hematology and Oncology, 27(1), 130-138.
Erdogan, M., Kaplan, H., Kayir, C. G., Asik, U. O., & Akbunar, S. (2012). Common method-related problems in selected research studies on educational sciences in Turkey. Procedia-Social and Behavioral Sciences, 46, 3230-3234.
Fraser B.J., & Wong A. F. L. (2003). Training of Educational Research Workers. In: Keeves J.P. et al. (eds), International Handbook of Educational Research in the Asia-Pacific Region. Springer International Handbooks of Education, vol 11.
Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and mixed approaches. Sage.
Harnqvist, K. (1997). Training of Research Workers in Education. In: Keeves J.P. (eds) Educational research, methodology and measurement: an international handbook, Second edition. Oxford, UK; Tarrytown, N. Y.: Pergamon
Keeves, J. P., & Watanabe, R. (Eds.). (2013). The international handbook of educational research in the Asia-Pacific region (Vol. 11). Springer Science & Business Media.
Lapanachokdee, W., Lawthong, N., & Piyapimonsit, C. (2016). Critiques on Thai Educational Research Methodology. Asian Social Science, 12(3), 93.
Levin, J. R. (1994). Crafting educational intervention research that's both credible and creditable. Educational Psychology Review, 6(3), 231-243.
Miller, D. W. (1999). The black hole of education research. Chronicle of Higher Education, 45(48), A17-A18.
Nuijten, M. B., Hartgerink, C. H., van Assen, M. A., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior research methods, 48(4), 1205-1226.
McMillan, J. H. (1996). Educational research: Fundamentals for the consumer. HarperCollins College Publishers, 10 East 53rd Street, New York, NY 10022; World Wide Web: http://www. harpercollins. com/college.
Picho, K., & Artino Jr, A. R. (2016). 7 deadly sins in educational research.
Ross, K. N. (1979). An empirical investigation of sampling errors in educational survey research. Journal of Educational Statistics, 4(1), 24-40.
Sotos, A. E. C., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2(2), 98-113.
Strasser, N. (2011). Avoiding statistical mistakes. Journal of College Teaching & Learning (TLC), 4(7).
Strasak, A. M., Zaman, Q., Pfeiffer, K. R., Gobel, G., & Ulmer, H. (2007). Statistical errors in medical research--a review of common pitfalls. Swiss medical weekly, 137(3-4), 44-49.
Sayer, A. (2010). Method in social science: revised 2nd edition. Routledge.
Scheaffer, R., Aliaga, M., Diener-West, M., Garfield, J., Higgins, T., Hilton, S., ... & Lester, F. K. (2007). Using statistics effectively in mathematics education research. Alexandria, VA: American Statistical Association.
Thompson, B. (1995). Inappropriate Statistical Practices in Counseling Research: Three Pointers for Readers of Research Literature. ERIC Digest.
Thompson, B. (1998). Five Methodology Errors in Educational Research: The Pantheon of Statistical Significance and Other Faux Pas.
Thompson, B. (1999). Common Methodology Mistakes in Educational Research, Revisited, along with a Primer on Both Effect Sizes and the Bootstrap.
Veldkamp, C. L., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A., & Wicherts, J. M. (2014). Statistical reporting errors and collaboration on statistical analyses in psychological science. PloS one, 9(12), e114876.
Walliman, N. (2011). Your research project: Designing and planning your work. Sage
