Exploring the potential of NUI interaction in a computational thinking game for children

Authors

  • Elisa M. de Castro UFSCar - campus Sorocaba
  • Tiemi C. Sakata UFSCar - campus Sorocaba
  • Luciana A. M. Zaina UFSCar - campus Sorocaba

DOI:

https://doi.org/10.5753/rbie.2019.27.02.140

Keywords:

Computational Thinking, Game, NUI Interaction, Children

Abstract

The new paradigms of interactions have arisen over the years and motivated the investigation of their potential for use within teaching-learning context. In a natural user interface (NUI), the interaction comes naturally while users interact with an application by using gestures, voice, touch, and so on. These ways of interaction are called natural, so they are based on body actions and movements commonly adopted by users in their daily life. Recently, the teaching of computational thinking in the early stages of children education has been highly discussed by education professionals and researchers in informatics and education in Brazil. And most of them recognize that games are a means to encourage children to develop the computational thinking. Taking into account that NUI can be adopted by applications with learning purposes, this article explores the potential of using such ways of interaction in a game for teaching computational thinking to children. The study was conducted in three stages. First, 10 children were co-participants in the selection of gestures and images of the game. In the second stage, considering the results of the first stage, two versions of the game were developed; one allowing the interactions by gestures and another by touch. Finally, an investigation was carried out on the acceptance and use of the two versions of the game, with 49 children. All children that took part in the study belonged to primary schools in Brazil. The results showed that by using the gestures to interact the children become more focused and more conscious about the movements they were performing in contrast to when they interacted by touch. In addition, a high level of acceptance for both ways of interaction was reported by the participants by which we could conclude that they had no difficulties in understanding the gestures and images.

Downloads

Download data is not yet available.

References

Baranauskas, M. C. C., Martins, M. C., & Valente, A. (2013). Codesign de redes digitais: Tecnologia e educação a serviço da inclusão social. Porto Alegre: Penso.[GS Search]

Bastos, N., Adamatti, D., & de Carvalho, F. (2016). Development of logic skills in high school students: a proposal based on neuroscience. Brazilian Journal of Computers in Education (RBIE), 24(1).[DOI:10.5753/rbie.2016.24.1.53].[GS Search]

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry (25), 49–59.[DOI:10.1016/0005-7916(94)90063-9]. [GS Search]

Cox, K., & Bittencourt, R. (2017). Bibliographical study on digital games building process: The need for synergy between educating and fun. Brazilian Journal of Computers in Education (RBIE), 25(01), 16. [DOI:10.5753/rbie.2017.25.01.16]. [GS Search]

CSTA, & ISTE. (2011). Computational thinking toolkit. CSTA (Computer Science Teacher Association) and ISTE (International Society for Technology In Education). Disponível em: [Link]. [GS Search]

Falcão, T. P., & Barbosa, R. (2015). Aperta o Play! Análise da interação exploratória em um jogo baseado em pensamento computacional. In Proceedings of XXVI Simpósio Brasileiro de Informática na Educação (SBIE’15), 419–428. [DOI:10.5753/cbie.sbie.2015.419]. [GS Search]

Falcão, T. P., Gomes, T. C. S., & Albuquerque, I. R. (2015). O Pensamento Computacional Através de Jogos Infantis: uma Análise de Elementos de Interação. In Proceedings of XIV Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC’15), Salvador, BA. [DOI:10.1145/3148456.3148485]. [GS Search]

Falcão, T. P., Santos, R., Rodrigues, A., & Silva, P. (2014). Corpo e mente: uma análise educacional e de usabilidade da interação corporal em videogames. In Proceedings XIII of Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais (IHC’14), Foz do Iguaçu, PR. [GS Search]

Gomes, T., & Tedesco, P. (2017). Gamificando a sala de aula: desafios e possibilidades em uma disciplina experimental de pensamento computacional no ensino fundamental. In Proceedings of XXIII Workshop de Informática na Escola (WIE’17), 1–10.[DOI:10.5753/cbie.wie.2017.1]. [GS Search]

Gomes, T. C. S., Barreto, P., Lima, I. R. A., & Falcão, T. P. (2015). Avaliação de um jogo educativo para o desenvolvimento do pensamento computacional na educação infantil. In Proceedings of IV Congresso Brasileiro de Informática na Educação (CBIE’15).[DOI:10.5753/cbie.wcbie.2015.1349]. [GS Search]

Goth, G. (2011). Brave NUI World. Communications of the ACM, 54(12), 14–16.[DOI:10.1016/C2009-0-64091-5]. [GS Search]

Goyal, S., Chopra, S., & Mohanan, D. (2017). Codefruits: Teaching computational thinking skills through hand gestures. In Proceedings Annual Symposium on Computer-Human Interaction (extended abstracts), 291–298, New York, NY.[DOI:10.1145/3130859.3131335]. [GS Search]

Hachaj, T., & Ogiela, M. R. (2014). Recognition of body movements patterns for immersive virtual reality system interface. In Proceedings of 9th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC’14), 290–294, Guangdong.[DOI:10.1109/3PGCIC.2014.79]. [GS Search]

Hayashi, E. C. S., & Baranauskas, M. C. C. (2013). Affectibility in educational technologies: A socio-technical perspective for design. Educational Technology Society,16(1), 57–68. Disponível em: https://www.j-ets.net/ets/journals/16_1/6.pdf. [GS Search]

Hayashi, E. C. S., Posada, J. E. G., Maike, V. R. M. L., & Baranauskas, M. C. C. (2016). Exploring new formats of the self-assessment manikin in the design with children. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computer Systems (IHC ’16). New York, NY, USA: ACM.[DOI:10.1145/3033701.3033728]. [GS Search]

Hiniker, A., Sobel, K., Hong, S. R., Suh, H., Irish, I., Kim, D., & Kientz, J. A. (2015). Touchscreen prompts for preschoolers: designing developmentally appropriate techniques for teaching young children to perform gestures. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC’15), 109–118. New York, NY.[DOI:10.1145/2771839.2771851]. [GS Search]

Kauppinen, S., Luojus, S., Tuomisto, J., & Ahlgren, A. (2013). Utilizing gesture recognition technology in children’s interactive storybook. In Proceedings of International Conference on Making Sense of Converging Media (AcademicMindTrek ’13), 76–79. New York, NY, USA: ACM.[DOI:10.1145/2523429.2523474]. [GS Search]

Lazar, J., Feng, J., & Hochheiser, H. (2017). Research methods in human-computer interaction (2nd ed.). Morgan Kaufmann.[GS Search]

Maike, V. R. M. L., de Sousa Britto Neto, L., Goldenstein, S. K., & Baranauskas, M. C. C. (2015). Heuristics for NUI revisited and put into practice. In M. Kurosu (Ed.), Human-Computer Interaction: Interaction Technologies, 317–328, Springer International Publishing.[DOI:10.1145/2523429.2523474]. [GS Search]

Martin-SanJose, J.-F., Juan, M.-C., Mollá, R., & Vivó, R. (2017). Advanced displays and natural user interfaces to support learning. Interactive Learning Environments, 25(1), 17–34.[DOI:10.1080/10494820.2015.1090455]. [GS Search]

Mendes, D., Lopes, P., & Ferreira, A. (2011). Hands-on interactive tabletop LEGO application. In Proceedings of 8th International Conference on Advances in Computer Entertainment Technology (ACE’11). New York, NY.[DOI:10.1145/2071423.2071447]. [GS Search]

Morais, D. C. S., Falcão, T. P., Oliveira, G. S., & Peres, F. M. d. A. (2017). Conteúdos curriculares em jogos digitais educacionais: Desafios de um processo participativo. In Proceedings of Workshop de Informática na Escola (WIE’17) (pp. 343–352). Recife, PE. [DOI:10.5753/cbie.wie.2017.343].[GS Search]

Muller, M. J. (2003). The human-computer interaction handbook. Chapter: Participatory design: The third space in HCI. In J. A. Jacko & A. Sears (Eds.), 1051–1068. Hillsdale, NJ, USA: L. Erlbaum Associates Inc.[GS Search]

Norman, D. A. (2010). Natural user interfaces are not natural interactions,17(3), 6–10.[DOI:10.1145/1744161.1744163].[GS Search]

Papadakis, S., Kalogiannakis, M., & Zaranis, N. (2016). Developing fundamental programming concepts and computational thinking with ScratchJr in preschool education: A case study. International Journal of Mobile Learning and Organization, 10(3), 187–202.[DOI:10.1504/IJMLO.2016.077867].[GS Search]

Plouffe, G., & Cretu, A. (2016). Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Transactions on Instrumentation and Measurement (2), 305–316.[DOI:10.1109/TIM.2015.2498560].[GS Search]

Raabe, A. L. A., Frango, I., Ribeiro, L., Granville, L. Z., Salgado, L., da Cruz, M. J. K., Fortes, S. (2017). Referenciais de Formação em Computação: Educação Básica. Sociedade Brasileira de Computação - SBC. Disponível em: [Link].

Rahman, M. S. A., Ali, N. M., & Masnizah, M. (2013). A study on the naturalness of gesture-based interaction for children. In Proceedings of 3rd International Visual Informatics Conference, Advances in Visual Informatics (IVIC’13) (pp. 718–728). Springer International Publishing.[DOI:10.1007/978-3-319-02958-0_65].[GS Search]

Reis, H. M., Wiechmann, L., Borges, S. S., Gasparini, I., Tsutsumi, M., Brandão, A. F., Isotani, S. (2015). Rehabilitation using kinect and an outlook on its educational applications: A review of the state of the art. In Proceedings of XXVI Simpósio Brasileiro de Informática na Educação (SBIE’15), 802–811. Maceió.[DOI:10.5753/cbie.sbie.2015.802].[GS Search]

Renzi, M., Vassos, S., Catarci, T., & Kimani, S. (2015). Touching notes: A gesture-based game for teaching music to children. In Proceedings of 9th International Conference on Tangible, Embedded, and Embodied Interaction (TEI’15), 603–606. New York, NY.[DOI:10.1145/2677199.2688810].[GS Search]

Ringland, K. E., Zalapa, R., Neal, M., Escobedo, L., & Tentori, M. E. (2014). Sensory paint: A natural user interface supporting sensory integration in children with neuro developmental disorders. In Proceedings of Conference on Human Factors in Computing Systems (CHI’14), 1681–1686. New York, NY. [DOI:10.1145/2559206.2581249].[GS Search]

Rodriguez, C., Zem-Lopes, A. M., Marques, L., & Isotani, S. (2015). Pensamento computacional: transformando ideias em jogos digitais usando o scratch. In Proceedings of XXI Workshop de Informática na Escola (WIE’15), 62–71.[DOI:10.5753/cbie.wie.2015.62].[GS Search]

Santos, P., Araujo, L., & Bittencourt, R. (2018). A mapping study of computational thinking and programming in brazilian k-12 education. In Proceedings of 48th Annual Frontiers in Education Conference (FIE’18).[DOI:10.1109/FIE.2018.8658828].[GS Search]

Shiratuddin, M. F., & Wong, K. W. (2012). Game design considerations when using non-touch based natural user interface. Transactions on Edutainment, 35–45.[DOI:10.1007/978-3-642-23456-9_43].[GS Search]

Soares, F., Esteves, J. S., Carvalho, V., Lopes, G., Barbosa, F., & Ribeiro, P. (2015). Development of a serious game for Portuguese sign language. In Proceedings of 7th International congress on Ultra Modern Telecommunications and Control Systems (ICUMT’15), 226–230.[DOI:10.1109/ICUMT.2015.7382432].[GS Search]

Subramanian, A. (2015). Integration of natural user interface in a real-world environment. In Proceedings of IEEE International Conference on computational Intelligence & Communication Technology (CICT’15), 714-718.[DOI:10.1109/CICT.2015.73].[GS Search]

Valente, J. A. (2002). A tecnologia no ensino: implicações para a aprendizagem. São Paulo: Casa do Psicólogo Editora.[GS Search]

Wigdor, D., & Wixon, D. (2011). Brave NUI World: Designing Natural User Interfaces for Touchand Gesture. San Francisco: Morgan Kaufmann Publishers Inc. [GS Search]

Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. Disponível em: [Link]. [DOI:10.1145/1118178.1118215].[GS Search]

Wing, J. M. (2011). Computational thinking – what and why? The magazine of Carnegie Mellon University’s School of Computer Science. Disponível em: [Link]. [GS Search]

Published

2019-05-31

How to Cite

DE CASTRO, E. M.; SAKATA, T. C.; ZAINA, L. A. M. Exploring the potential of NUI interaction in a computational thinking game for children. Brazilian Journal of Computers in Education, [S. l.], v. 27, n. 2, p. 140–166, 2019. DOI: 10.5753/rbie.2019.27.02.140. Disponível em: https://journals-sol.sbc.org.br/index.php/rbie/article/view/4747. Acesso em: 19 sep. 2024.

Issue

Section

Awarded Papers