The PGIRec Software to Support the Evaluation of User-System Interaction with a Synchronous and Multi-perspective Recording System

Gabriel Alves Mendes Vasiljevic; Leonardo Cunha de Miranda; Murilo de Araújo Bento

doi:10.5753/jis.2023.3240

Authors

Gabriel Alves Mendes Vasiljevic Edmond and Lily Safra International Institute of Neuroscience - Santos Dumont Institute, Federal University of Rio Grande do Norte https://orcid.org/0000-0001-5322-601X
Leonardo Cunha de Miranda Federal University of Rio Grande do Norte https://orcid.org/0000-0003-1929-9391
Murilo de Araújo Bento Federal University of Rio Grande do Norte https://orcid.org/0009-0004-4816-4511

DOI:

https://doi.org/10.5753/jis.2023.3240

Keywords:

HCI, user experience, audiovisual recording, case study, experiment, computer games

Abstract

Audiovisual recording is a highly relevant activity when investigating and analyzing qualitative data, both for scientific researches and for the industry. Games are a particularly challenging application for user experience and interaction evaluation, given the multitude of platforms, modes of operation, number of players and game environments that may occur through different games genres. This diversity of interaction possibilities makes it difficult for employing current audiovisual recording tools in a number of scenarios, especially considering multi-user interaction in the same or in separate environments. In this context, this paper presents an open-source recording tool, called Player-Game Interaction Recorder (PGIRec), specialized in synchronous recording of multiple sources of audio and video. The software was evaluated using brain-computer interface games in two different studies, with a total of 22 participants and over 16 hours of recorded activities using different experimental setups. The evaluation showed that the solution was stable and adaptable, since it generated audiovisual media without occurrences in different situations, with multiple audio and video sources concurrently.

Downloads

Download data is not yet available.

References

Bennett, C., Weiss, B., Suh, J., Yoon, E., Jeong, J., and Chae, Y. (2022). Exploring data-driven components of socially intelligent AI through cooperative game paradigms. Multimodal Technologies and Interaction, 6(2):16.

Bento, M. A. and Miranda, L. C. (2015). Steering all: Infinite racing game controlled by haar cascade through OpenCV. In 2015 XVII Symposium on Virtual and Augmented Reality. IEEE.

Bento, M. A., Vasiljevic, G. A. M., and Miranda, L. C. (2022). Player-game interaction recorder: Um software de gravação para apoiar a avaliação da interação de jogadores com jogos digitais. In Proceedings of the XXI Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames’21), pages 1–10. SBC.

Bottorff, J. L. (1994). Critical Issues in Qualitative Research Methods, chapter Using videotaped recordings in qualitative research, pages 244–261. SAGE Publishing.

Brizolara, P. L. S., Miranda, L. C., and Menezes, B. C. (2022). Esfera interativa: Design de um novo controle para jogos digitais. In Extended Proceedings of the XXI Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames’22), pages 1–10. SBC.

Chuang, T.-Y., Yeh, M. K.-C., and Lin, Y. L. (2021). The impact of game playing on students’ reasoning ability, varying according to their cognitive style. Educational Technology & Society, 24(3):29–43.

Correia, A. C. C., Miranda, L. C., and Hornung, H. H. (2013). Gesture-based interaction in domotic environments: State of the art and hci framework inspired by the diversity. In Proceedings of the 14th IFIP TC13 Conference on Human-Computer Interaction (Lecture Notes in Computer Science), volume 8118, pages 300–317, Heidelberg, Germany. Springer.

Ferreira, A. L. S., Marciano, J. N., Miranda, L. C., and Miranda, E. E. C. (2014). Understanding and proposing a design rationale of digital games based on brain-computer interface: Results of the Admiralmind Battleship study. SBC Journal on Interactive Systems, 5(1):3–15.

Ferreira, A. L. S., Miranda, L. C., and Miranda, E. E. C. (2012). Interfaces cérebro-computador de sistemas interativos: estado da arte e desafios de IHC. In Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems (IHC’12), pages 239–248. SBC.

Ferreira, A. L. S., Miranda, L. C., Miranda, E. E. C., and Sakamoto, S. G. (2013). A survey of interactive systems based on brain-computer interfaces. SBC Journal on 3D Interactive Systems, 4(1):3–13.

Furtado, E., Militão, G., Andrade, R., Miranda, L., and Oliveira, K. (2014). Ubiquidade, múltiplos dispositivos e tangibilidade. In Baranauskas, S. and (org.), P., editors, I GranDIHC-BR – Grandes Desafios de Pesquisa em Interação Humano-Computador no Brasil. Relatório Técnico, pages 23–26. Comissão Especial de Interação Humano-Computador (CEIHC) da Sociedade Brasileira de Computação (SBC).

Furtado, E. S., Nova, G. V., Gasparini, I., de Miranda, L. C., Barbosa, M., de Alencar, T. S., and de Almeida Neris, V. P. (2017). Re-evaluating the first five years of the GrandIHC-br-challenge 3. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems. ACM.

Garcez, A., Duarte, R., and Eisenberg, Z. (2011). Production and analysis of video recordings in qualitative research. Educação e Pesquisa, 37(2):249–260.

Guo, J.-L., Hsu, H.-P., Lin, M.-H., Lin, C.-Y., and Huang, C.-M. (2020). Testing the usability of digital educational games for encouraging smoking cessation. International Journal of Environmental Research and Public Health, 17(8):2695.

Ishii, H. and Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, pages 234–241. ACM.

Lima, F. M. C., Miranda, L. C., and Baranauskas, M. C. C. (2018). Two HCI communities in perspective: Revealing and contrasting trends. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems, pages 1–10, New York, USA. ACM.

Lima, F. M. C., Miranda, L. C., Vasiljevic, G. A. M., and Baranauskas, M. C. C. (2019). An intra and interconferences correlation analysis of the field of HCI. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems. ACM.

Lima, F. M. C., Vasiljevic, G. A. M., Miranda, L. C., and Baranauskas, M. C. C. (2021). An analysis of IHC and HCII publication titles: Revealing and comparing the topics of interest of their communities. Journal on Interactive Systems, 12(1):1–20.

Marciano, J. N., Ferreira, A. L. S., Correia, A. C. C., Miranda, L. C., and Miranda, E. E. C. (2013). Karuchā ships invaders: Cultural issues on the design/development of a japanese call game made by/to brazilians. In Proceedings of the XII Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames’13), pages 172–180. SBC.

Marciano, J. N., Miranda, E. E. C., and Miranda, L. C. (2014). Applying the method for evaluation of motivational aspects on karuchā ships invaders educational game with brazilian students of japanese. In Proceedings of the XX Workshop de Informática na Escola (WIE 2014), pages 66–75. SBC.

Marciano, J. N., Miranda, L. C., Miranda, E. E. C., and Oliveira, P. T. R. (2016). Karuchā ships invaders em sala de aula: Relato de experiência do uso do jogo educacional por estudantes de japonês do IFRN. In Anais do XXII Workshop de Informática na Escola, pages 555–564, Porto Alegre, RS. SBC.

Marciano, J. N., Oliveira, J. B. C., Menezes, B. C., Miranda, L. C., and Miranda, E. E. C. (2015). Katakana star samurai: A mobile tool to support learning of a basic japanese alphabet. In Proceedings of the XLI Latin American Computing Conference, pages 517–524, Los Alamitos, USA. IEEE.

Marques, F., Menezes, B., Miranda, L., Miranda, E., and Marciano, J. (2015a). Sumo sensei: Design, implementação e teste com usuários de uma ferramenta móvel para apoiar o estudo de kanjis básicos. In Proceedings of the XLI Latin American Computing Conference, pages 492–503, Los Alamitos, USA. IEEE.

Marques, F., Miranda, L., Menezes, B., Miranda, E., and Marciano, J. (2015b). Karuta kanji: Jogo educacional para estudar e praticar vocabulário com kanjis da língua japonesa. In Proceedings of the XLI Latin American Computing Conference, pages 504–516, Los Alamitos, USA. IEEE.

Martens, A. L., Grover, C. A., Saucier, D. A., and Morrison, B. A. (2018). An examination of gender differences versus similarities in a virtual world. Computers in Human Behavior, 84:404–409.

Miranda, L. C., Hornung, H., Pereira, R., and Baranauskas, M. C. C. (2013). Exploring adjustable interactive rings in game playing: Preliminary results. In Design, User Experience, and Usability. Health, Learning, Playing, Cultural, and Cross-Cultural User Experience, pages 518–527. Springer Berlin Heidelberg.

Miranda, L. C., Hornung, H. H., and Baranauskas, M. C. C. (2010). Adjustable interactive rings for iDTV. IEEE Transactions on Consumer Electronics, 56(3):1988–1996.

Miranda, L. C., Hornung, H. H., and Baranauskas, M. C. C. (2011a). Adjustable interactive rings for iDTV: First results of an experiment with end-users. In Human Centered Design, pages 262–271. Springer Berlin Heidelberg.

Miranda, L. C., Hornung, H. H., and Baranauskas, M. C. C. (2011b). Qualitative results of an experiment with adjustable interactive rings for idtv. In Proceedings of the IADIS International Conference on Interfaces and Human Computer Interaction, pages 233–240. IADIS Press.

Ouherrou, N., Benmarrakchi, F., Elhammoumi, O., and Kafi, J. E. (2019a). Evaluation of an educational game for children with dyslexia: Funlexia - a case study. International Journal of Information Science & Technology, 3(6):4–14.

Ouherrou, N., Elhammoumi, O., Benmarrakchi, F., and Kafi, J. E. (2019b). Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment. Education and Information Technologies, 24(2):1777–1792.

Pasquali, M. (2007). Video in science. protocol videos: the implications for research and society. EMBO reports, 8(8):712–716.

Penn-Edwards, S. (2012). Human factors affecting the use of video recording methodology in qualitative research. International Journal of Multiple Research Approaches, 6(2):150–159.

Quail, N., Holmes, K., Linn, A., Rea, P., Livingstone, D., and Boyle, J. (2018). Serious digital games using virtual patients and digital chalk-talk videos: a novel approach to teaching acute diabetes care in the undergraduate medical curriculum. Diabetic Medicine, 35:112–135.

Vasiljevic, G. A. M. and Miranda, L. C. (2019a). The effect of auditory stimuli on user’s meditation and workload in a brain–computer interface game. Interacting with Computers, 31(3):250–262.

Vasiljevic, G. A. M. and Miranda, L. C. (2019b). The influence of graphical elements on user’s attention and control on a neurofeedback-based game. Entertainment Computing, 29:10–19.

Vasiljevic, G. A. M. and Miranda, L. C. (2020a). Brain-computer interface games based on consumer-grade EEG devices: A systematic literature review. International Journal of Human-Computer Interaction, 36(2):1–38.

Vasiljevic, G. A. M. and Miranda, L. C. (2020b). A general model for electroencephalography-controlled brain-computer interface games. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 12250 LNCS, pages 174–189.

Vasiljevic, G. A. M. and Miranda, L. C. (2022). Comparing users' performance and game experience between a competitive and collaborative brain-computer interface. Behaviour & Information Technology, pages 1–20.

Vasiljevic, G. A. M., Miranda, L. C., and Menezes, B. C. (2018a). Mental War: An Attention-based Single/Multi-player Brain-Computer Interface Game. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 10960 LNCS, pages 450–465. Springer.

Vasiljevic, G. A. M., Miranda, L. C., and Menezes, B. C. (2018b). Zen Cat: A Meditation-Based Brain-Computer Interface Game. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 10960 LNCS, pages 294–309. Springer.

Vasiljevic, G. A. M., Miranda, L. C., and Miranda, E. E. C. (2014). MasterMind Chess: Design and implementation of Classic, Capablanca and Fischer modes with real time match observation. In Proceedings of the XIII Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames’14), pages 1–9, Porto Alegre. SBC.

Vasiljevic, G. A. M., Miranda, L. C., and Miranda, E. E. C. (2016). A case study of MasterMind Chess: Comparing mouse/keyboard interaction with Kinect-based gestural interface. Advances in Human-Computer Interaction, 2016(1):1–10.

Vasiljevic, G. A. M., Miranda, L. C., Miranda, E. E. C., and Silva, L. F. (2012). Prototipação de interfaces tangíveis de produtos interativos: Estado da arte e desafios da plataforma arduino. In Anais do XI Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (ACM International Conference Proceeding Series), pages 249–258, Porto Alegre, RS. SBC.

Vidal, J. J. (1973). Toward direct brain-computer communication. Annual Review of Biophysics and Bioengineering, 2(1):157–180.