Designing a Comprehensive Model for Evaluating the Effectiveness of Gamification Projects in Online Environment

Subject Areas : Infomartion Technology

mohsen rahbari ¹ , neda soleymani ² , mohammad soltanifar ³ , ataollah abtahi ⁴ , afsane zamanie moghaddam ⁵

1 - دانشجوی دکتری مدیریت رسانه‌ای، گروه مدیریت رسانه‌ای، دانشکده مدیریت و اقتصاد، واحد علوم و تحقیقات، دانشگاه آزاد اسلامی، تهران، ایران
2 - Assistant Professor, Department of Social Communication Sciences, Faculty of Humanities and Education, Tabriz Branch, Islamic Azad University, Tabriz, Iran
3 - Associate Professor, Department of Social Communication Sciences, Faculty of Literature, Humanities and Social Sciences, Science and Research Unit, Islamic Azad University, Tehran, Iran
4 - Assistant Professor, Department of Media Management, Faculty of Management and Economics, Science and Research Branch, Islamic Azad University, Tehran, Iran
5 - Associate Professor, Department of Educational Management, Faculty of Management and Economics, Science and Research Branch, Islamic Azad University, Tehran, Iran

Received: 2020-11-27 Accepted : 2021-12-13 Published : 2021-12-22

Keywords: effectiveness, motivation, Gamification, User Experience, Behavioral changes,

Abstract :

The increasing use of communication technologies and the impact that it has on the culture, way of thinking and practice and style of activity. has also faced important changes in the educational system. Due to the increasing growth of gaming or gamification with different functions and in online environments, especially in educational and online education, as well as the importance and necessity of effectiveness evaluation processes as one of the most important and necessary control activities, it is necessary to developers measure the success or effectiveness of their gamification plans. Since the dimensions, components and indicators of gamification effectiveness have not been identified in general and there is no comprehensive model in this field, this research while identifying the desired dimensions and variables through the Delphi method in four rounds and Targeted sampling provides a comprehensive model for evaluating the effectiveness of gamification. The results show that the effectiveness of gamification is affected by three main dimensions called user experience (with eight variables), behavioral changes and user achievements (with five variables) and motivation (with one variable). In addition to providing a comprehensive and new model, the present study examines and evaluates the role of new variables for the first time.

References:

Ab. Rahman. R., Ahmad, S.,& Hashim, U.R. (2018). "The effectiveness of gamification technique for higher education students engagement in polytechnic Muadzam Shah Pahang, Malaysia", International Journal of Educational Technology in Higher Education, Vol. 15 No. 41, pp. 1-16.

Ahmadi, Fazlollah., Nasiriani, Khadijeh and Abazari, Parvaneh (2008), "Delphi technique as a tool in research", Journal of Education in Medical Sciences, 8 (1), pp. 175-185. (in Persian).

Amir. B. and Ralph. P. (2014). "Proposing a Theory of Gamification Effectiveness", ICSE Companion: Companion Proceedings of the 36th International Conference on Software Engineering, Pages 626–627.

Berkling, K. & Thomas, C. (2013). "Gamification of a software engineering course and a detailed analysis of the factors that lead to it’s failure". International Conference on Interactive Collaborative Learning, ICL, (September), pp.525–530. Brito. R.D.S., Pinochet. L.H.C., Lopes. E.L. and Oliveira. M.A.de. (2018). "Development of a gamification characteristics measurement scale for mobile application users", Review of International Business, 13(1), p. 01-16.

Csikszentmihalyi, M., & Csikszentmihalyi, I.S. (1988). Optimal experience: Psychological studies of flow in consciousness. New York: Cambridge University Press. Davis, F. D. (1996). "A model of the antecedents of perceived ease of use: Development and test." Decision Sciences, 27(3), pp. 451– 481.

Ducheneaut, N.Y, Nickell, E. & Moore R.J. (2006). "Alone together? exploring the social dynamics of massively multiplayer online games." Proceedings of the SIGCHI conference on Human Factors in computing systems, pp. 407–416.

Fishbein, M. and Ajzen, I. (1975). "Belief, attitude, intention and behavior: An introduction to theory and research". Addison-wesley. Publishing company, USA.

Fitz-Walter, Z., T, D. and Wyeth., p. (2015). "Investigating the Design of Effective Gamification Experiences for Mobile Applications and Devices", Ph.D Thesis Written under the supervision of Dian Tjondronegoro & Peta Wyeth. Science & Engineering Faculty Queensland University of Technology.

Hamari. J., (2013), “Transforming Homo Economicus into Homo Ludens: A Field Experiment on Gamification in a Utilitarian Peer-To-Peer Trading Service”, Electronic Commerce Research and Applications, 12(4), pp. 236-245.

Hamari, J., Koivisto, J., & Harri, S. (2014). "Does Gamification Work? - A Literature Review of Empirical Studies on Gamification.", 47th Hawaii International Conference on System Sciences, IEEE. pp. 3025-3034. Hamed, H. m., (2017), "Investigating the Effectiveness of Applying Gamification on Tourism Websites from the Customers’ Perspective: An Exploratory Study", European Journal of Hospitality and Tourism Research, Vol.5, No.4, pp.1-27.

Hanus, M. D., & Fox, J. (2014). "Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance." Computers & Education, Vol. 80, pp. 151-162.

Hofacker, Ch. F., Ruyter, Ko de, Lurie N. H., Manchanda, P. & Donaldson, J. (2016). "Gamification and Mobile Marketing Effectiveness", Journal of Interactive Marketing, 34, pp. 25–36.

Javadi Bora, M., Ebrahimzadeh, I., Farajollahi, M., Sarmadi, M. (2011). 'Designing an Effectiveness Assessment Model for Distance Education System in Payame Noor University', Information and Communication Technology in Educational Sciences, 2(1), pp. 79-98. (in Persian).

Kim, B. (2015). "Understanding gamification". Library Technology Reports, Vol. 51, No. 2. Pp 1-37. Koufaris, M. (2002). "Applying the technology acceptance model and flow theory to online consumer behavior." Information Systems Research, 13(2), pp. 205–223.

Lallemand, C., Gronier, G., & Koenig, V. (2015). "User experience: A concept without consensus? Exploring practitioners’ perspectives through an international survey", Computers in Human Behaviour , Volume 43, pp. 35-48

Landsell., J. and Hägglund, E. (2016). "Towards a Gamification Framework: Limitations and opportunities when gamifying business processes", Institutionen för informatik, SPB 2016.01.

Miranisargazi, N., asgari, M., Malekhosseyni, S., Yazdi, A. (2020). "Evaluation and ranking of factors affecting the success of gamification implementation from the perspective of student teachers and teachers using reasoned activity model (TRA)", Information and Communication Technology in Educational Sciences, 10(40), pp. 25-44. (in Persian).

Oghuma, A. P., Libaque-Saenz, C. F., Wong, S. F., & Chang, Y. (2016). "Na expectation-confirmation model of continuance intention to use mobile instant messaging". Telematics and Informatics, Vol.33 No. 1, pp. 34-47

Osipov, I., Volinsky, A., Nikulchev, E. and Prasikova, A Y. (2015). "Study of Gamification Effectiveness in Online e-Learning Systems", International Journal of Advanced Computer Science and Applications, Vol. 6, No. 2. Pp 10-24.

Seaborn, K. & Fels, D.I. (2015). "Gamification in theory and action: A survey". International Journal of Human-Computer Studies, Vol.74, pp. 14-31.

Sigala. M. (2015). "The application and impact of gamification funware on trip planning and experiences: the case of TripAdvisor’s funware", The International Journal on Networked Business, Vol. 25, No 3., pp. 189-209.

Venkatesh, V. (1996), "A model of the antecedents of perceived ease of use: Development and test." Decision Sciences, 27(3), pp. 451– 481. Winn, B. M. (2009). "The Design, Play, and Experience Framework. In R. E. Ferdig", Handbook of Research on Effective Electronic Gaming in Education, 3, 1010-1024.

Yee. N. (2002). "Facets: 5 motivation factors for why people play mmorpgs". www.nickyee.com Zicherman, G., & Linder, J. (2013). "The Gamification Revolution". Mc Graw Hill Education.