Roll Like Thunder: Expert Validation and Refinement of a Design Process to Convey Emotions in Music Visualizations

Caio Nunes; Ticianne Darin

doi:10.5753/jbcs.2026.5747

Authors

Caio Nunes Federal University of Ceará https://orcid.org/0000-0001-5997-3518
Ticianne Darin Federal University of Ceará https://orcid.org/0000-0003-3617-5462

DOI:

https://doi.org/10.5753/jbcs.2026.5747

Keywords:

Music Visualization Design, Emotional Communication, User Experience, UX, Player Experience, PX, Design Process, Expert Review, Audio Design, Audio Assessment, Well-being, Hearing Accessibility, Human-Computer Interaction (HCI), Affective Computing, Iterative Design, Evaluation Metrics, Immersive Audio

Abstract

Music evokes complex emotions and enhances engagement across interactive media. Music visualizations — graphical representations of sound — have gained relevance in games, virtual environments, and digital art. Yet, many existing approaches fail to convey emotional nuance, focusing instead on structural or reactive aspects of sound. To support the creation of emotionally engaging visualizations, we previously introduced Thunder, a design process grounded in a Research Through Design (RtD) approach. While its initial application showed creative potential, it also highlighted opportunities to improve structural clarity, enhance guidance for designers, and increase adaptability across varied contexts. In this paper, we present a refined version of Thunder, developed through an expert review with specialists in Human-Computer Interaction, Music, and Computing. The updated process is organized into three core phases -- Conceptualization, Prototyping, and Evaluation -- each offering clearer support for emotional, aesthetic, and musical decision-making. Implementation is now framed as an external and adaptable step, allowing flexibility across technical scenarios. The new refined Thunder offers an improved foundation and design guidance for creating emotionally resonant music visualizations in diverse contexts.

Downloads

Download data is not yet available.

References

Abreu, R., dos Santos, J., and Bezerra, E. (2018). A bimodal learning approach to assist multi-sensory effects synchronization. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE. DOI: 10.1109/IJCNN.2018.8489357.

Almeida, H., Cabral, G., and Moura, R. (2021). Design process and rapid prototyping of animated music visualizations. In Anais do XVIII Simpósio Brasileiro de Computação Musical, pages 114-120. SBC. DOI: 10.5753/sbcm.2021.19435.

Armstrong, J. S. (2001). Principles of Forecasting A Handbook for Researchers and Practitioners. International Series in Operations Research & Management Science. Springer. DOI: 10.1007/978-0-306-47630-3.

Barbosa, S. D. J., Silva, B. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuario. Auto publicação. Book.

Beat Games (2019). Beat saber. Available at:[link].

Bergstrom, T., Karahalios, K., and Hart, J. C. (2007). Isochords: visualizing structure in music. In Proceedings of Graphics Interface 2007, pages 297-304. DOI: 10.1145/1268517.1268565.

Brown, R. J. (1976). Atari video music. digital music visualizer.

Brown, R. J. (1978). Audio activated video display. Avaialble at:[link].

Chanda, M. L. and Levitin, D. J. (2013). The neurochemistry of music. Trends in cognitive sciences, 17(4):179-193. DOI: 10.1016/j.tics.2013.02.007.

Chasin, M. (2003). Music and hearing aids. The Hearing Journal, 56(7):36-38. DOI: 10.1097/01.HJ.0000292553.60032.c2.

Chen, C., Cao, N., Hou, J., Guo, Y., Zhang, Y., and Shi, Y. (2023). Musicjam: Visualizing music insights via generated narrative illustrations. arXiv preprint arXiv:2308.11329. DOI: 10.48550/arXiv.2308.11329.

Ciuha, P., Klemenc, B., and Solina, F. (2010). Visualization of concurrent tones in music with colours. In Proceedings of the 18th ACM international conference on Multimedia, pages 1677-1680. DOI: 10.1145/1873951.1874320.

Collopy, F. (2021). Color, form, and motion: Dimensions of a musical art of light. Visual Communication, 20(2):167-188. DOI: 10.1162/002409400552829.

Cowen, A. S., Fang, X., Sauter, D., and Keltner, D. (2020). What music makes us feel: At least 13 dimensions organize subjective experiences associated with music across different cultures. Proceedings of the National Academy of Sciences, 117(4):1924-1934. DOI: 10.1073/pnas.1910704117.

Cuny, C., Fornerino, M., and Helme-Guizon, A. (2015). Can music improve e-behavioral intentions by enhancing consumers' immersion and experience? Inf. Manag., 52:1025-1034. DOI: 10.1016/j.im.2015.07.009.

De Prisco, R., Malandrino, D., Pirozzi, D., Zaccagnino, G., and Zaccagnino, R. (2018). Evaluation study of visualisations for harmonic analysis of 4-part music. In 2018 22nd International Conference Information Visualisation (IV), pages 484-489. IEEE. DOI: 10.1109/iV.2018.00090.

Desmet, P. (2002). Designing emotions. Available at:[link].

Diener, E. and Chan, M. Y. (2011). Happy people live longer: Subjective well-being contributes to health and longevity. Applied Psychology: Health and Well-Being, 3(1):1-43. DOI: 10.1111/j.1758-0854.2010.01045.x.

Disney, W. (1940). Fantasia.

Eitan, Z. and Granot, R. Y. (2006). How music moves. Music perception, 23(3):221-248. DOI: 10.1525/mp.2006.23.3.221.

Erdmann, M., von Berg, M., and Steffens, J. (2025). Development and evaluation of a mixed reality music visualization for a live performance based on music information retrieval. Frontiers in Virtual Reality, 6:1552321. DOI: 10.3389/frvir.2025.1552321.

Fourney, D. W. and Fels, D. I. (2009). Creating access to music through visualization. In 2009 ieee toronto international conference science and technology for humanity (tic-sth), pages 939-944. IEEE. DOI: 10.1109/TIC-STH.2009.5444364.

Gabrielsson, A. (2001). Emotion perceived and emotion felt: Same or different? Musicae Scientiae, 5(1 suppl):123-147. DOI: 10.1177/10298649020050s105.

Graf, M., Opara, H. C., and Barthet, M. (2021). An audio-driven system for real-time music visualisation. arXiv preprint arXiv:2106.10134. DOI: 10.48550/arXiv.2106.10134.

Hallam, S. and MacDonald, R. (2013). Introduction: Perspectives on the power of music. Research Studies in Music Education, 35(1):83-86. DOI: 10.1177/1321103X13488485.

Hassenzahl, M. (2008). User experience (ux): towards an experiential perspective on product quality. pages 11-15. DOI: 10.1145/1512714.1512717.

Hiraga, R., Watanabe, F., and Fujishiro, I. (2002). Music learning through visualization. In Second International Conference on Web Delivering of Music, 2002. WEDELMUSIC 2002. Proceedings., pages 101-108. IEEE. DOI: 10.1109/WDM.2002.1176199.

Holm, J. and Siirtola, H. (2012). A comparison of methods for visualizing musical genres. In 2012 16th International Conference on Information Visualisation, pages 636-645. DOI: 10.1109/IV.2012.107.

Huppert, F. A. (2009). Psychological well-being: Evidence regarding its causes and consequences. Applied psychology: health and well-being, 1(2):137-164. DOI: 10.1111/j.1758-0854.2009.01008.x.

Hwang, A. and Oh, J. (2020). Interacting with background music engages e-customers more: The impact of interactive music on consumer perception and behavioral intention. Journal of Retailing and Consumer Services, 54:101928. DOI: 10.1016/j.jretconser.2019.101928.

Juslin, P. N. and Laukka, P. (2004). Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. Journal of new music research, 33(3):217-238. DOI: 10.1080/0929821042000317813.

Juslin, P. N. and Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. Behavioral and brain sciences, 31(5):559-575. DOI: 10.1017/S0140525X08005293.

Kamioka, H., Tsutani, K., Yamada, M., Park, H., Okuizumi, H., Tsuruoka, K., Honda, T., Okada, S., Park, S.-J., Kitayuguchi, J., et al. (2014). Effectiveness of music therapy: a summary of systematic reviews based on randomized controlled trials of music interventions. Patient preference and adherence, pages 727-754. DOI: 10.2147/PPA.S61340.

Keyes, C. L. (2007). Promoting and protecting mental health as flourishing: a complementary strategy for improving national mental health. American psychologist, 62(2):95. DOI: 10.1037/0003-066X.62.2.95.

Khulusi, R., Kusnick, J., Meinecke, C., Gillmann, C., Focht, J., and Jänicke, S. (2020). A survey on visualizations for musical data. In Computer Graphics Forum, volume 39, pages 82-110. Wiley Online Library. DOI: 10.1111/cgf.13905.

Kim, J. and André, E. (2010). Emotion recognition system using short-term monitoring of physiological signals. Medical & biological engineering & computing, 48:1087-1098. DOI: 10.1007/bf02344719.

Koelsch, S. (2010). Towards a neural basis of music-evoked emotions. Trends in cognitive sciences, 14(3):131-137. DOI: 10.1016/j.tics.2010.01.002.

Koelsch, S. (2014). Brain correlate of music-evoked emotions. Nature reviews. Neuroscience, 15:170-180. DOI: 10.1038/nrn3666.

Koelsch, S. (2015). Music-evoked emotions: principles, brain correlates, and implications for therapy. Annals of the New York Academy of Sciences, 1337(1):193-201. DOI: 10.1111/nyas.12684.

Lazar, J., Feng, J. H., and Hochheiser, H. (2017). Research methods in human-computer interaction. Morgan Kaufmann. DOI: 10.1016/b978-0-12-805390-4.09991-x.

Lee, C. and Hong, J.-H. (2025). musicolors: Bridging sound and visuals for synesthetic creative musical experience. DOI: 10.48550/arXiv.2503.14220.

Lee, Y. and Fathia, R. N. (2016). Interactive music visualization for music player using processing. In 2016 22nd International Conference on Virtual System & Multimedia (VSMM), pages 1-4. IEEE. DOI: 10.1109/VSMM.2016.7863205.

Lima, H. B., Santos, C. G. D., and Meiguins, B. S. (2021). A survey of music visualization techniques. ACM Computing Surveys (CSUR), 54(7):1-29. DOI: 10.1145/3461835.

Liu, V., Long, T., Raw, N., and Chilton, L. (2023). Generative disco: Text-to-video generation for music visualization. arXiv preprint arXiv:2304.08551. DOI: 10.48550/arXiv.2304.08551.

Miller, M., Bonnici, A., and El-Assady, M. (2019). Augmenting music sheets with harmonic fingerprints. In Proceedings of the ACM Symposium on Document Engineering 2019, pages 1-10. DOI: 10.1145/3342558.3345395.

Mori, J. and Fels, D. I. (2009). Seeing the music can animated lyrics provide access to the emotional content in music for people who are deaf or hard of hearing? In 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH), pages 951-956. IEEE. DOI: 10.1109/TIC-STH.2009.5444362.

Munteanu, L. H. (2012). Musical culture, a finality of musical education. Procedia-Social and Behavioral Sciences, 46:4195-4199. DOI: 10.1016/j.sbspro.2012.06.225.

Norman, D. (2007). Emotional design: Why we love (or hate) everyday things. Basic books. DOI: 10.5860/choice.41-6846.

Nunes, C. and Darin, T. (2024). Echoes of player experience: A literature review on audio assessment and player experience in games. Proc. ACM Hum.-Comput. Interact., 8(CHI PLAY). DOI: 10.1145/3677069.

Nunes, C., Reinbold, I., Castro, M., and Darin, T. (2024). Thunder: A design process to build emotionally engaging music visualizations. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, pages 1-15. DOI: 10.1145/3702038.3702077.

Palmer, S. E., Schloss, K. B., Xu, Z., and Prado-León, L. R. (2013). Music-color associations are mediated by emotion. Proceedings of the National Academy of Sciences, 110(22):8836-8841. DOI: 10.1073/pnas.1212562110.

Pereira, D. and Chambel, T. (2023). Enhancing emotional awareness and regulation in movies and music based on personality. Proceedings of the 2023 ACM International Conference on Interactive Media Experiences. DOI: 10.1145/3573381.3596462.

Pokharel, R. (2020). Exploring the different roles and aspects of music. Molung Educational Frontier, pages 125-133. DOI: 10.3126/mef.v10i1.34035.

Pouris, M. and Fels, D. I. (2012). Creating an entertaining and informative music visualization. In International Conference on Computers for Handicapped Persons, pages 451-458. Springer. DOI: 10.1007/978-3-642-31522-0_68.

Prisco, R. D., Malandrino, D., Pirozzi, D., Zaccagnino, G., and Zaccagnino, R. (2017). Understanding the structure of musical compositions: Is visualization an effective approach? Information Visualization, 16(2):139-152. DOI: 10.1177/1473871616655468.

Reybrouck, M. and Eerola, T. (2017). Music and its inductive power: a psychobiological and evolutionary approach to musical emotions. Frontiers in Psychology, 8:242694. DOI: 10.3126/mef.v10i1.34035.

Rogers, Y., Sharp, H., and Preece, J. (2023). Interaction Design: Beyond Human-Computer Interaction. Wiley, 6th edition. Book.

Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., and Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social work research, 27(2):94-104. DOI: 10.1093/swr/27.2.94.

Ryan, R. M. and Deci, E. L. (2001). On happiness and human potentials: A review of research on hedonic and eudaimonic well-being. Annual review of psychology, 52(1):141-166. DOI: 10.1146/annurev.psych.52.1.141.

Ryan, R. M. and Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford publications. DOI: 10.1521/978.14625/28806.

Salakka, I., Pitkäniemi, A., Pentikäinen, E., Mikkonen, K., Saari, P., Toiviainen, P., and Särkämö, T. (2021). What makes music memorable? relationships between acoustic musical features and music-evoked emotions and memories in older adults. PloS one, 16(5):e0251692. DOI: 10.1371/journal.pone.0251692.

Schloss, K. B. and Palmer, S. E. (2018). Color aesthetics: A deep relationship between color and emotion. In Elliot, A. J., Fairchild, M. D., and Franklin, A., editors, Handbook of Color Psychology, pages 177-194. Cambridge University Press. Book.

Schmidt, K. L. (2018). Meaningful Music Visualizations. PhD thesis, Purdue University. Available at:[link].

Siu, A., S-H Kim, G., O'Modhrain, S., and Follmer, S. (2022). Supporting accessible data visualization through audio data narratives. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI '22, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3491102.3517678.

Stewart, R., Kudumakis, P., and Sandler, M. (2010). Interactive music applications and standards. pages 20-30. DOI: 10.1007/978-3-642-23126-1_2.

Zhang, Y., Pan, Y., and Zhou, J. (2018). Study on application of audio visualization in new media art. In Journal of Physics: Conference Series, volume 1098, page 012003. IOP Publishing. DOI: 10.1088/1742-6596/1098/1/012003.

Zimmerman, J. and Forlizzi, J. (2014). Research through design in hci. In Ways of Knowing in HCI, pages 167-189. Springer. DOI: 10.1007/978-1-4939-0378-8_8.