An Empirical Investigation on the Effects of Gamifying an Online Judge in an Introductory Programming Course
DOI:
https://doi.org/10.5753/rbie.2020.28.0.461Keywords:
Gamification, Introductory Programming, Gamified Online JudgeAbstract
CONTEXT. This paper presents a gamified online judge developed to motivate non-major freshmen in programming classes (CS1). The system was applied in 11 classes of different engineering and science programs of Federal University of Amazonas throughout the first academic term of 2017. OBJECTIVE. The main objective of this study is to assess the impact of this gamified online judge in a CS1 course on student performance, motivation, experience and learning perception. METHOD. A quasi-experiment was carried out with the aim of measuring the influence of the gamified system on student performance, by statistically comparing the data of 2017/1 (with gamification) to 2016/1 (without gamification). In addition, in 2017/1 classes, a qualitative questionnaire was applied to measure students' perceptions on three components: motivation, user experience and learning perception. RESULTS. There was a significant increase in the final grade of CS1 among non-dropout students which did not depend on the reported previous experience in programming, but focused more on first-time students. In addition, there was an increase in the number of logins, number of code tests and IDE usage time. There was no significant difference in the number of correct exercises, but there was a reduction in variance, especially among students with lower performance. Also, in the experimental classes, there was a decrease in the association between the Enem Mathematics test score and the performance in programming. Regarding students’ perceptions of the three components of qualitative analysis, most responses were positive. CONCLUSIONS. Although the dropout rate of CS1 has increased during the academic term in which the gamified online judge was used, the results reinforce the evidence about the potential of gamification in motivating programming learning.
Downloads
References
Alves, F. (2015). Gamification: como criar experiências de aprendizagem engajadoras (2nd ed). DVS Editora. [GS Search]
Bez, J. L., Tonin, N. A., & Rodegheri, P. R. (2014). URI Online Judge Academic: A tool for algorithms and programming classes. In Proceedings of the International Conference on Computer Science & Education, 149-152. Vancouver, BC. [DOI:10.1109/ICCSE.2014.6926445] [GS Search]
Bosse, Y., & Gerosa, M. A. (2015). Reprovações e Trancamentos nas Disciplinas de Introdução à Programação da Universidade de São Paulo: Um Estudo Preliminar. In XXIII Workshop sobre Educação em Computação. Recife. [GS Search]
Branch, R. M. (2009). Instructional design: The ADDIE approach (Vol. 722). Springer Science & Business Media. [DOI:10.1007/978-0-387-09506-6] [GS Search]
Campos, A., Gardiman, R., & Madeira, C. (2015). Aplicação da Gamificação na Disciplina de Empreendedorismo. In XXIII Workshop sobre Educação em Computação. Recife. [GS Search]
Chaves, J. O. M., Castro, A. F., Lima, R. W., Lima, M. V. A., & Ferreira, K. H. (2013). Integrando Moodle e Juízes Online no Apoio a Atividades de Programação. In Anais do Simpósio Brasileiro de Informática na Educação (Vol. 24, No. 1, p. 244) .[DOI:10.5753/cbie.sbie.2013.244] [GS Search]
Dick, W., Carey, L., & Carey, J. O. (2005). The Systematic Design of Instruction (6th ed). Pearson. [GS Search]
Fardo, M. L. (2013). A gamificação como estratégia pedagógica: estudos de elementos dos "games" aplicados em processos de ensino e aprendizagem. Dissertação de mestrado em educação, Universidade de Caxias do Sul, Caxias do Sul, RS. [GS Search]
Francisco, R. E., Ambrósio, A. P. L., Pereira Júnior, C. X., & Fernandes, M. A. (2018). Juiz online no ensino de CS1 - lições aprendidas e proposta de uma ferramenta. Revista Brasileira de Informática na Educação; v. 26, n. 03. [DOI:10.5753/rbie.2018.26.03.163] [GS Search]
Galvão, L., Fernandes, D., & Gadelha, B. (2016). Juiz online como ferramenta de apoio a uma metodologia de ensino híbrido em programação. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE) (Vol. 27, No. 1, p. 140). [DOI:10.5753/cbie.sbie.2016.140] [GS Search]
INEP. (2017) Nota Técnica Nº 33/2017/CGCQES/DAES - Cálculo do IDD. Disponível em: [Link]. Acesso em 30 de setembro de 2019.
Kapp, K. (2012). The gamification of learning and instruction: game-based methods and strategies for training and education. John Wiley & Sons. [GS Search]
Kasahara, R., Sakamoto, K., Washizaki, H., & Fukazawa, Y. (2019). Applying Gamification to Motivate Students to Write High-Quality Code in Programming Assignments. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '19), 92–98. ACM, New York. [DOI:10.1145/3304221.3319792] [GS Search]
Keller, J. M. (1983). Development and Use of the ARCS Model of Motivational Design. Classic writings on instructional technology, 10(3), 2–10. [DOI:10.1007/BF02905780] [GS Search]
Klock, A. C. T., da Cunha, L. F., de Carvalho, M. F., Rosa, B. E., Anton, A. J., & Gasparini, I. (2015). Gamification in e-learning systems: A conceptual model to engage students and its application in an adaptive e-learning system. Learning and collaboration technologies, 595–607. Springer, Cham. [DOI:10.1007/978-3-319-20609-7_56] [GS Search]
Lowry, P. B., Gaskin, J., Twyman, N., Hammer B., & Roberts, T. (2012). Taking ‘Fun and Games’ Seriously: Proposing the Hedonic Motivation System Adoption Model (HMSAM). Journal of the Association for Information Systems, vol. 14, no. 11, pp. 617–671. [DOI:10.17705/1jais.00347] [GS Search]
McCabe, T. J. (1976). A Complexity Measure. IEEE Transactions on Software Engineering SE-2, 4, 308–320. [DOI:10.1109/TSE.1976.233837] [GS Search]
Nicolaci-da-Costa, A. M. (2007). O campo da pesquisa qualitativa e o Método de Explicitação do Discurso Subjacente (MEDS). Psicologia: reflexão e crítica, 20(1), 65–73. [DOI:10.1590/S0102-79722007000100009] [GS Search]
Oliveira Júnior, J. G., Noronha, R. V., & Kaestner, C. A. A. (2017). Método de Seleção de Atributos Aplicados na Previsão da Evasão de Cursos de Graduação. Revista de Informática Aplicada, 13(2). [GS Search]
Peixoto, M. M., Silva, C., Gonçalves, E., & Vilena, J. (2015). Um Mapeamento Sistemático de Gamificação em Software Educativo no Contexto da Comunidade Brasileira de Informática na Educação. In Anais do XXI Workshop de Informática na Escola. [DOI:10.5753/cbie.wie.2015.584] [GS Search]
Petit, J., Roura, S., Carmona, J., Cortadella, J., Duch, A., Gimenez, O., Mani, A., Mas, J., Rodriguez-Carbonella, E., Rubio, A., Pedro, J., & Venkataramani, D. (2018). Jutge.org: Characteristics and Experiences. In IEEE Transactions on Learning Technologies, vol. 11, no. 3, 321–333. [DOI:10.1109/TLT.2017.2723389] [GS Search]
Petri, G., Wangenheim, C. G., & Borgatto, A. F. (2018). MEEGA+, Systematic Model to Evaluate Educational Games. In N. Lee (Ed.) Encyclopedia of Computer Graphics and Games. Springer. [DOI:10.1007/978-3-319-08234-9_214-1]
Prather, J., Pettit, R., McMurry, K., Peters, A., Homer, J., & Cohen, M. (2018). Metacognitive Difficulties Faced by Novice Programmers in Automated Assessment Tools. In Proceedings of the 2018 ACM Conference on International Computing Education Research, 41–50. [DOI:10.1145/3230977.3230981] [GS Search]
Savi, R., Wangenheim, C., & Borgatto, A. (2011). Um modelo de avaliação de jogos educacionais na engenharia de software. In Anais do XXV Simpósio Brasileiro de Engenharia de Software (SBES 2011), São Paulo. [DOI:10.1109/SBES.2011.27] [GS Search]
Silva, A. C., Correa, C. S., Coelho, D. A., Silva Neto, D. T., Ferraz, L., Xavier, M. M., Reis, R. S., Rocha, F. A., & Santos, P. A. (2016). Análise dos Índices de Reprovação nas Disciplinas de Cálculo I e AVGA do Curso de Engenharia Elétrica do Instituto Federal da Bahia de Vitória da Conquista. In Proceedings of the XIV International Conference on Engineering and Technology Education. [DOI:10.13140/2.1.3348.0487] [GS Search]
Silva, T. S. C., Melo, J. C. B., & Tedesco, P. C. D. A. R. (2018). A Model to Promote Student Engagement in Programming Learning Using Gamification. Brazilian Journal of Computers in Education, 26(03), 120. [DOI:10.5753/rbie.2018.26.03.120] [GS Search]
Sinly, C., Rusli, A., & Winarno, P. M. (2018). Utilizing Gamification to Improve User Participation in Online Judge. In 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS)(pp. 543-547). Toyama, Japan. [DOI:10.1109/SCIS-ISIS.2018.00096] [GS Search]
Tondello, G. F., Mora, A., & Nacke, L. E. (2017). Elements of Gameful Design Emerging from User Preferences. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, New York. [DOI:10.1145/3116595.3116627] [GS Search]
Wangenheim, C. G., & Wangenheim, A. (2012). Ensinando computação com jogos. Bookess Editora, Florianópolis, SC, Brasil. [GS Search]
Wasik, S., Antczak, M., Badura, J., Laskowski, A., & Sternal, T. (2018). A Survey on Online Judge Systems and Their Applications. ACM Computing Surveys, 51, 1, Article 3.[DOI:10.1145/3143560] [GS Search]
Werbach, K., & Hunter, D. (2012). For the win: How game thinking can revolutionize your business. Wharton Digital Press.[GS Search]
Wöhlin, C., Höst, M., & Henningsson, K. (2003). Empirical research methods in software engineering. Empirical methods and studies in software engineering, 7–23. Springer, Berlin, Heidelberg. [DOI:10.1007/978-3-540-45143-3_2] [GS Search]
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Ralph B. S. Ribeiro, Leandro S. G. Carvalho, David B. F. Oliveira, Elaine H. T. de Oliveira, Marcela Pessoa
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
