Sensation in Gamification: A Qualitative Investigation of Background Music in Gamified Learning

José Alexandre de Freitas; Mateus Oliveira; Carlos Martinelli; Fernando Amorim; Armando M. Toda; Paula Palomino; Ana C. T. Klock; Guilherme Guerino; Anderson P. Avila-Santos; Luiz Rodrigues

doi:10.5753/jis.2024.4501

Authors

José Alexandre de Freitas UniSENAI PR - Londrina https://orcid.org/0009-0006-1393-8454
Mateus Oliveira UniSENAI PR - Londrina https://orcid.org/0009-0008-1756-9072
Carlos Martinelli UniSENAI PR - Londrina https://orcid.org/0009-0004-1507-9513
Fernando Amorim UniSENAI PR - Londrina https://orcid.org/0009-0004-5184-6951
Armando M. Toda Independent Researcher https://orcid.org/0000-0003-2681-8698
Paula Palomino Sao Paulo State College of Technology - Matão https://orcid.org/0000-0002-9730-2253
Ana C. T. Klock Tampere University https://orcid.org/0000-0003-3774-6511
Guilherme Guerino State University of Paraná https://orcid.org/0000-0002-4979-5831
Anderson P. Avila-Santos University of São Paulo https://orcid.org/0000-0002-6791-0768
Luiz Rodrigues Federal University of Alagoas https://orcid.org/0000-0003-0343-3701

DOI:

https://doi.org/10.5753/jis.2024.4501

Keywords:

Gamification, Education, Sensation, Music, ChatGPT

Abstract

Gamification in education has received significant attention for its potential to enhance student engagement and motivation. However, challenges arise from the excessive use of reward-oriented game elements, which are assumed to have negative effects on learning outcomes in many cases. In contrast, while the literature argues immersion-based gamification holds great potential, there is little research on how such an approach affects learning experiences. The Sensation game element, for instance, might contribute to students' experiences by providing sensory queues, such as auditory feedback based on background music, to foster concentration, engagement, and immersive learning experiences. Nevertheless, past studies have not sufficiently investigated how the Sensation game element affects gamified learning experiences. Therefore, this paper implements the Sensation game element in two studies: Study 1 introduced background music during a reading activity, while Study 2 implemented multiple background music tracks aligned with different learning stages to drive students' experiences. Accordingly, we evaluated this game element through usability tests, based on high-fidelity prototypes of online learning environments, followed by semi-structured interviews that were analyzed through thematic analysis with the help of ChatGPT. Overall, we found that Sensation, particularly instrumental music, positively influenced concentration but requires careful design to maintain engagement. The findings highlight the importance of tailoring Sensation's implementation to consider individual preferences and contextual factors and the need for thoughtful selection and management of sensory queues to optimize learning environments effectively. Additionally, we provide evidence emphasizing the value of using tools like ChatGPT to optimize qualitative data analysis, although human oversight remains prominent to ensure robust research outcomes. Overall, this study contributes insights for designing personalized and effective gamified learning experiences based on the Sensation game element in enhancing learning experiences.

Downloads

Download data is not yet available.

References

Altmeyer, M., Hnatovskiy, V., Rogers, K., Lessel, P., and Nacke, L. E. (2022). Here comes no boom! the lack of sound feedback effects on performance and user experience in a gamified image classification task. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–14. DOI: https://doi.org/10.1145/3491102.3517581.

Bai, S., Hew, K. F., and Huang, B. (2020). Is gamification “bullshit”? evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, page 100322. DOI: https://doi.org/10.1016/j.edurev.2020.100322.

Barbosa, S. D. J., Silva, B. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuario. Auto-publicação.

Blandford, A., Furniss, D., and Makri, S. (2016). Qualitative hci research: Going behind the scenes. Synthesis lectures on human-centered informatics, 9(1):1–115. DOI: https://doi.org/10.1007/978-3-031-02217-3.

Buil, I., Catalán, S., and Martínez, E. (2019). The influence of flow on learning outcomes: An empirical study on the use of clickers. British Journal of Educational Technology, 50(1):428–439. DOI: https://doi.org/10.1111/bjet.12561.

de Freitas, J. A., Oliveira, M., Martinelli, C., Toda, A. M., Palomino, P. T., Klock, A. C., Avila-Santos, A. P., and Rodrigues, L. (2023). Finding harmony in gamification: Examining the role of music in gamified learning. In Anais do II Workshop sobre Interação e Pesquisa de Usuários no Desenvolvimento de Jogos, pages 1–12. SBC. DOI: https://doi.org/10.5753/wiplay.2023.236059.

de la Mora Velasco, E. and Hirumi, A. (2020). The effects of background music on learning: A systematic review of literature to guide future research and practice. Educational Technology Research and Development, 68:2817–2837. DOI: https://doi.org/10.1007/s11423-020-09783-4.

De Paoli, S. (2023). Performing an inductive thematic analysis of semi-structured interviews with a large language model: An exploration and provocation on the limits of the approach. Social Science Computer Review, page 08944393231220483. DOI: https://doi.org/10.1177/08944393231220483.

de Sousa Santos, C., Garcia, B. C. C., de Souza, L. A. M., Gomes, M. F., Fernandes, B. S., Esteves, E. A., et al. (2023). Development, usability, formative assessment, and story immersion of nutrigame, a mhealth nutrition education app. Journal on Interactive Systems, 14(1):470–480. DOI: https://doi.org/10.5753/jis.2023.3274.

Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, pages 9–15. ACM. DOI: https://doi.org/10.1145/2181037.2181040.

Durmanova, K. (2022). The effects of juicy game design on exergames. Master’s thesis, University of Waterloo.

Dyson, M. C. and Haselgrove, M. (2001). The influence of reading speed and line length on the effectiveness of reading from screen. International Journal of Human-Computer Studies, 54(4):585–612. DOI: https://doi.org/10.1006/ijhc.2001.0458.

Ferreri, L. and Verga, L. (2016). Benefits of music on verbal learning and memory: How and when does it work? Music Perception: An Interdisciplinary Journal, 34(2):167–182.

Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. (2023). Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764–10799. PMLR.

Garro, R. S. et al. (2019). A música como auxílio para o transe e o êxtase.

Giannakos, M. and Cukurova, M. (2023). The role of learning theory in multimodal learning analytics. British Journal of Educational Technology, 54(5):1246–1267. DOI: https://doi.org/10.1111/bjet.13320.

Giray, L. (2023). Prompt engineering with chatgpt: A guide for academic writers. Annals of Biomedical Engineering, pages 1–5. DOI: https://doi.org/10.1007/s10439-023-03272-4.

Goldstein, E. B. (2014). Cognitive psychology: Connecting mind, research and everyday experience. Nelson Education.

Guillen, G., Jylhä, H., and Hassan, L. (2021). The role sound plays in games: a thematic literature study on immersion, inclusivity and accessibility in game sound research. In Proceedings of the 24th International Academic Mindtrek Conference, pages 12–20. DOI: https://doi.org/10.1145/3464327.3464365.

Hallam, S. and Himonides, E. (2022). The power of music: An exploration of the evidence. Open Book Publishers.

Hanus, M. D. and Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80:152 – 161. DOI: https://doi.org/10.1016/j.compedu.2014.08.019.

Hicks, K., Gerling, K., Dickinson, P., and Vanden Abeele, V. (2019). Juicy game design: Understanding the impact of visual embellishments on player experience. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play, pages 185–197. DOI: https://doi.org/10.1145/3311350.3347171.

Huang, R., Ritzhaupt, A. D., Sommer, M., Zhu, J., Stephen, A., Valle, N., Hampton, J., and Li, J. (2020). The impact of gamification in educational settings on student learning outcomes: a meta-analysis. Educational Technology Research and Development, pages 1–27. DOI: https://doi.org/10.1007/s11423-020-09807-z.

Husain, G., Thompson, W. F., and Schellenberg, E. G. (2002). Effects of musical tempo and mode on arousal, mood, and spatial abilities. Music perception, 20(2):151–171.

Kämpfe, J., Sedlmeier, P., and Renkewitz, F. (2011). The impact of background music on adult listeners: A meta-analysis. Psychology of music, 39(4):424–448. DOI: https://doi.org/10.1177/0305735610376261.

Klock, A. C. T., Gasparini, I., Pimenta, M. S., and Hamari, J. (2020). Tailored gamification: A review of literature. International Journal of Human-Computer Studies, page 102495. DOI: https://doi.org/10.1016/j.ijhcs.2020.102495.

Koivisto, J. and Hamari, J. (2019). The rise of motivational information systems: A review of gamification research. International Journal of Information Management, 45:191–210. DOI: https://doi.org/10.1016/10.1016/j.ijinfomgt.2018.10.013.

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.

Landers, R. N., Auer, E. M., Collmus, A. B., and Armstrong, M. B. (2018). Gamification science, its history and future: Definitions and a research agenda. Simulation & Gaming, 49(3):315–337. DOI: https://doi.org/10.1177/1046878118774385.

Lehmann, J. A. and Seufert, T. (2017). The influence of background music on learning in the light of different theoretical perspectives and the role of working memory capacity. Frontiers in psychology, 8:1902. DOI: https://doi.org/10.3389/fpsyg.2017.01902.

Lipscomb, S. D. and Zehnder, S. M. (2004). Immersion in the virtual environment: The effect of a musical score on the video gaming experience. Journal of Physiological Anthropology and Applied Human Science, 23(6):337–343. DOI: https://doi.org/10.2114/jpa.23.337.

LIXANDRU, D. (2024). The use of artificial intelligence for qualitative data analysis: Chatgpt. Informatica Economica, 28(1).

Lopes, W., Augusto, P., Fernandes, I., and Madeira, C. (2024). Proposal for a gamification strategy applied to remote learning. Journal on Interactive Systems, 15(1):92–103. DOI: https://doi.org/10.5753/jis.2024.2700.

Loughrey, K. and Broin, D. (2018). Are we having fun yet? misapplying motivation to gamification. In 2018 IEEE Games, Entertainment, Media Conference (GEM), pages 1–9. IEEE. DOI: https://doi.org/10.1109/GEM.2018.8516535.

Mohan, A. and Thomas, E. (2020). Effect of background music and the cultural preference to music on adolescents’ task performance. International Journal of Adolescence and Youth, 25(1):562–573. DOI: https://doi.org/10.1080/02673843.2019.1689368.

Munday, R. (2007). Music in video games. Music, sound and multimedia: From the live to the virtual, pages 51–67.

Owoahene Acheampong, K. and Nyaaba, M. (2024). Review of qualitative research in the era of generative artificial intelligence. Matthew, Review of Qualitative Research in the Era of Generative Artificial Intelligence (January 7, 2024). DOI: http://dx.doi.org/10.2139/ssrn.4686920.

Petersen, G. B., Petkakis, G., and Makransky, G. (2022). A study of how immersion and interactivity drive vr learning. Computers & Education, 179:104429. DOI: https://doi.org/10.1016/j.compedu.2021.104429.

Rodrigues, L., Toda, A. M., Oliveira, W., Palomino, P. T., Avila-Santos, A. P., and Isotani, S. (2021). Gamification works, but how and to whom? an experimental study in the context of programming lessons. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, pages 184–190. DOI: https://doi.org/10.1145/3408877.3432419.

Rodrigues, L., Toda, A. M., Oliveira, W., Palomino, P. T., and Isotani, S. (2020a). Just beat it: Exploring the influences of competition and task-related factors in gamified learning environments. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 461–470. SBC. DOI: https://doi.org/10.5753/cbie.sbie.2020.461.

Rodrigues, L., Toda, A. M., Palomino, P. T., Oliveira, W., and Isotani, S. (2020b). Personalized gamification: A literature review of outcomes, experiments, and approaches. In Eighth International Conference on Technological Ecosystems for Enhancing Multiculturality, pages 699–706. DOI: https://doi.org/10.1145/3434780.3436665.

Rodrigues, M. V. L. (2015). Explorando o impacto dos efeitos sonoros na aprendizagem com jogos. Master’s thesis, Pós-Graduação Em Jogos Digitais Polo: Jundiaí.

Rogers, K., Milo, M., Weber, M., and Nacke, L. E. (2020). The potential disconnect between time perception and immersion: Effects of music on vr player experience. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play, pages 414–426. DOI: https://doi.org/10.1145/3410404.3414246.

Ryan, R. M. and Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications.

Sailer, M. and Homner, L. (2020). The gamification of learning: a meta-analysis. Educ Psychol Rev, 32:77–112. DOI: https://doi.org/10.1007/s10648-019-09498-w.

Toda, A. M., Klock, A. C., Oliveira, W., Palomino, P. T., Rodrigues, L., Shi, L., Bittencourt, I., Gasparini, I., Isotani, S., and Cristea, A. I. (2019). Analysing gamification elements in educational environments using an existing gamification taxonomy. Smart Learning Environments, 6(1):16. DOI: https://doi.org/10.1186/s40561-019-0106-1.

Toda, A. M., Valle, P. H. D., and Isotani, S. (2018). The dark side of gamification: An overview of negative effects of gamification in education. In Cristea, A. I., Bittencourt, I. I., and Lima, F., editors, Higher Education for All. From Challenges to Novel Technology-Enhanced Solutions, pages 143–156, Cham. Springer International Publishing. DOI: https://doi.org/10.1007/978-3-319-97934-2_9.

Toledo Palomino, P., Nacke, L., and Isotani, S. (2023). Gamification of virtual learning environments: A narrative and user experience approach. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, pages 1–10. DOI: https://doi.org/10.1145/3638067.3638103.

Van Roy, R. and Zaman, B. (2018). Need-supporting gamification in education: An assessment of motivational effects over time. Computers & Education, 127:283–297. DOI: https://doi.org/10.1016/j.compedu.2018.08.018.

Vansteenkiste, M., Sierens, E., Soenens, B., Luyckx, K., and Lens, W. (2009). Motivational profiles from a self-determination perspective: The quality of motivation matters. Journal of educational psychology, 101(3):671. DOI: https://doi.org/10.1037/a0015083.

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D. C. (2023). A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382.

Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., and Wessln, A. (2012). Experimentation in Software Engineering. Springer Publishing Company, Incorporated.

Zambrano, A. F., Liu, X., Barany, A., Baker, R. S., Kim, J., and Nasiar, N. (2023). From ncoder to chatgpt: From automated coding to refining human coding. Advances in Quantitative Ethnography. ICQE 2023. DOI: https://doi.org/10.1007/978-3-031-47014-1_32.