Heuristics to Support the Evaluation of Optimal Experience in Educational Games for Learning Japanese as a Second Language

Fábio Phillip Rocha Marques; Leonardo Cunha de Miranda

doi:10.5753/jis.2023.3239

Authors

Fábio Phillip Rocha Marques Federal University of Rio Grande do Norte https://orcid.org/0000-0002-3858-2056
Leonardo Cunha de Miranda Federal University of Rio Grande do Norte https://orcid.org/0000-0003-1929-9391

DOI:

https://doi.org/10.5753/jis.2023.3239

Keywords:

CALL, L2, JCALL, flow theory, flow dimensions, flow experience

Abstract

The evaluation of Computer-Assisted Language Learning (CALL) tools can be carried out from the perspective of different aspects, and there’s still no proposal from the literature for the evaluation of CALL games from the perspective of an “optimal experience”. Therefore, this paper proposes a set of 32 heuristics for the evaluation of educational games for Japanese Language Learning (JCALL). Furthermore, the heuristics were applied in the evaluation of JCALL L2 (i.e., as a second language) games in order to verify their usefulness. Findings show the heuristics provide good support in the evaluation of language learning educational games, and can collaborate in the redesign of these games in order to improve the optimal player experience. Finally, results also indicate the feasibility of using these heuristics in the evaluation of other types of CALL tools other than games.

Downloads

Download data is not yet available.

References

Ang, C. S. and Zaphiris, P. (2008). Computer games and language learning. In Handbook of research on instructional systems and technology, pages 449–462. IGI Global.

Bull, D. (1996). Karuta: Sport or culture? Japan Quarterly, 43(1):67.

Butler, Y. G., Someya, Y., and Fukuhara, E. (2014). Online games for young learners’ foreign language learning. Elt Journal, 68(3):265–275.

Carson, J. G. (1992). Becoming biliterate: First language influences. Journal of Second Language Writing, 1(1):37–60.

Chinnery, G. M. (2006). Going to the mall: Mobile assisted language learning.

Ciampa, K. (2014). Learning in a mobile age: an investigation of student motivation. Journal of Computer Assisted Learning, 30(1):82–96.

Csikszentmihalhi, M. (2020). Finding flow: The psychology of engagement with everyday life. Hachette UK.

Csikszentmihalyi, M. (1988). The flow experience and its significance for human psychology. Cambridge University Press.

Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience. Harper & Row.

Csikszentmihalyi, M. (1993). The evolving self: A psychology for the third millennium, volume 5. HarperCollins Publishers New York.

Csikszentmihalyi, M. (2000). Beyond boredom and anxiety. Jossey-bass.

Csikszentmihalyi, M. (2013). Flow: The psychology of happiness. Random House.

Csikszentmihalyi, M. and Csikszentmihalyi, I. S. (2006). A life worth living: Contributions to positive psychology. Oxford University Press.

Desurvire, H., Caplan, M., and Toth, J. A. (2004). Using heuristics to evaluate the playability of games. In CHI’04 extended abstracts on Human factors in computing systems, pages 1509–1512.

Fernandes, K. T., da Trindade, G. O., Rêgo, A. H. G., Miranda, L. C., Lucena, M. J. N. R., and Gomes, A. V. (2012). E-learning via dispositivos móveis no brasil: Estado da arte e desafios à luz do acesso participativo e universal do cidadão brasileiro ao conhecimento. In Anais do Workshop de Desafios da Computação Aplicada à Educação, pages 128–137.

Godwin-Jones, R. (2014). Games in language learning: Opportunities and challenges.

Govender, T. and Arnedo-Moreno, J. (2021). An analysis of game design elements used in digital game-based language learning. Sustainability, 13(12):6679.

Haristiani, N. and Firmansyah, D. B. (2016). Android application for enhancing japanese jlpt n5 kanji ability. Journal of Engineering Science and Technology, 12:106–114.

Hubbard, P. (1988). An integrated framework for call courseware evaluation. CALICO journal, pages 51–72.

Hubbard, P. (2006). Evaluating call software. Calling on CALL: From theory and research to new directions in foreign language teaching, pages 313–338.

Hubbard, P. (2011). Evaluation of courseware and websites. Present and future promises of CALL: From theory and research to new directions in language teaching, pages 407–440.

Ibrahim, R., Yusoff, R. C. M., Mohamed-Omar, H., and Jaafar, A. (2011). Students perceptions of using educational games to learn introductory programming. Computer and Information Science, 4(1):205.

Ishaq, K., Rosdi, F., Zin, N. A. M., and Abid, A. (2021). Heuristics and think-aloud method for evaluating the usability of game-based language learning. International Journal of Advanced Computer Science and Applications, 12(11).

Kacetl, J. and Klímová, B. (2019). Use of smartphone applications in english language learning—a challenge for foreign language education. Education Sciences, 9(3):179.

Kato, T. (2005). Learning strategies employed by chinese-background learners in learning japanese vocabulary. Japanese Studies, 25(3):271–286.

Kickmeier-Rust, M. D., Mattheiss, E., Steiner, C., and Albert, D. (2011). A psycho-pedagogical framework for multi-adaptive educational games. International Journal of Game-Based Learning (IJGBL), 1(1):45–58.

Kim, H. and Kwon, Y. (2012). Exploring smartphone applications for effective mobile-assisted language learning. Multimedia-Assisted Language Learning, 15(1):31–57.

Kuhara-Kojima, K., Hatano, G., Saito, H., and Haebara, T. (1996). Vocalization latencies of skilled and less skilled comprehenders for words written in hiragana and kanji. Reading Research Quarterly, 31(2):158–171.

Kukulska-Hulme, A. and Traxler, J. (2013). Design principles for mobile learning. In Rethinking pedagogy for a digital age, pages 268–281. Routledge.

Macedonia, M. (2005). Games and foreign language teaching. Support for learning, 20(3):135–140.

Marciano, J., Miranda, L. C., de Miranda, E. E. C., and de Oliveira, P. d. T. R. (2016). Karucha ships invaders em sala de aula: Relato de experiência do uso do jogo educacional por estudantes de japonês do ifrn. In Anais do XXII Workshop de Informática na Escola, pages 555–564. SBC.

Marciano, J., Miranda, L. C., Miranda, E., and Pereira, R. (2013a). Android mall apps for japanese language: identifying and eliciting interface/interaction requirements through the semiotic framework. In XXXIX Conferencia Latinoamericana en Informática, pages 220–230.

Marciano, J. N., de Miranda, E. E. C., and Miranda, L. C. (2014). Applying the method for evaluation of motivational aspects on karuchā ships invaders educational game with brazilian students of japanese. In Anais do XX Workshop de Informática na Escola, pages 66–75. SBC.

Marciano, J. N., de Oliveira, J. B. C., de Menezes, B. C., Miranda, L. C., and de Miranda, E. E. C. (2015). Katakana star samurai: A mobile tool to support learning of a basic japanese alphabet. In 2015 Latin American Computing Conference (CLEI), pages 1–8. IEEE.

Marciano, J. N., Ferreira, A. L. S., Correia, A., Miranda, L. C., and Miranda, E. (2013b). Karuchā ships invaders: cultural issues on the design/development of a japanese call game made by/to brazilians. Anais do XII Simpósio Brasileiro de Jogos e Entretenimento Digital, pages 172–180.

Marciano, J. N., Miranda, L. C., and de Miranda, E. E. (2013c). Japanese call web tools: identifying and evaluating their multimedia behaviour in android os. In EdMedia+ Innovate Learning, pages 1087–1096. Association for the Advancement of Computing in Education (AACE).

Marciano, J. N., Miranda, L. C., and de Miranda, E. E. C. (2012). Japanese language learning supported by computational tools: State of the art and challenges for the latin america community. In 2012 XXXVIII Conferencia Latinoamericana En Informatica (CLEI), pages 1–10. IEEE.

Marques, F. and Miranda, L. C. (2022a). Avaliação de jogos educacionais para aprendizado da língua japonesa: Uma proposta baseada em heurísticas. In Anais Estendidos do XXI Simpósio Brasileiro de Jogos e Entretenimento Digital, pages 1009–1018. SBC.

Marques, F. and Miranda, L. C. (2022b). Design de jogo e experiência de fluxo: Reflexão e desafios na perspectiva da teoria do fluxo. In Anais Estendidos do XXI Simpósio Brasileiro de Jogos e Entretenimento Digital, pages 41–50. SBC.

Marques, F. A. R. and Miranda, L. C. (2018a). Assessment of lexical acquisition of a student with high japanese language proficiency: An analysis based on nihongo kotoba shiken. In 2018 XIII Latin American Conference on Learning Technologies (LACLO), pages 216–223. IEEE.

Marques, F. A. R. and Miranda, L. C. (2018b). Nihongo kotoba shiken: a computerized exam of japanese lexical proficiency. In 2018 XLIV Latin American Computer Conference (CLEI), pages 398–407. IEEE.

Marques, F. A. R., Miranda, L. C., de Menezes, B. C., et al. (2015a). Karuta kanji: Jogo educacional para estudar e praticar vocabulário com kanjis da lingua japonesa. In 2015 Latin American Computing Conference (CLEI), pages 1–13. IEEE.

Marques, F. P. R., de Menezes, B. C., Miranda, L. C., de Miranda, E. E. C., and Marciano, J. N. (2015b). Sumo sensei: Design, implementação e teste com usuários de uma ferramenta móvel para apoiar o estudo de kanjis básicos. In 2015 Latin American Computing Conference (CLEI), pages 1–12. IEEE.

Miangah, T. M. and Nezarat, A. (2012). Mobile-assisted language learning. International Journal of Distributed and Parallel Systems, 3(1):309.

Mohamed, H., Jaafar, A., et al. (2010). Heuristics evaluation in computer games. In 2010 International Conference on Information Retrieval & Knowledge Management (CAMP), pages 188–193. IEEE.

Mohammad, N. M. N., Mamat, M. N., and Isa, P. M. (2012). M-learning in malaysia: Challenges and strategies. Procedia-Social and Behavioral Sciences, 67:393–401.

Moreno Fuentes, E., Risueno Martinez, J. J., et al. (2018). Design of a checklist for evaluating language learning websites.

Nakamura, J., Csikszentmihalyi, M., et al. (2002). The concept of flow. Handbook of positive psychology, 89:105.

Nielsen, J. (1992). Reliability of severity estimates for usability problems found by heuristic evaluation. In Posters and short talks of the 1992 SIGCHI conference on Human factors in computing systems, pages 129–130.

Okuyama, Y. (2007). Call vocabulary learning in japanese: Does romaji help beginners learn more words? CALICO journal, pages 355–379.

Petri, G. and von Wangenheim, C. G. (2016). How to evaluate educational games: a systematic. Journal of Universal Computer Science, 22(7):992–1021.

Rêgo, M. and Medeiros, I. (2015). Heeg: Heuristic evaluation for educational games. Proceedings of SBGames.

Rowley, M. (1992). Kanji pict-o-graphix: over 1,000 Japanese kanji and kana mnemonics. Stone Bridge Press Berkeley, CA.

Sakamoto, T. (1976). Writing systems in japan. New horizons in reading, pages 244–249.

Samimy, K. K. (1994). Teaching japanese: Consideration of learners’ affective variables. Theory into Practice, 33(1):29–33.

Sung, Y.-T., Chang, K.-E., and Yang, J.-M. (2015). How effective are mobile devices for language learning? a meta-analysis. Educational research review, 16:68–84.

Sweetser, P. and Wyeth, P. (2005). Gameflow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE), 3(3):3–3.

Sykes, J. M. (2018). Digital games and language teaching and learning. Foreign Language Annals, 51(1):219–224.

Torat, B. (2000). Computer-assisted language learning: An overview. Silpakorn University International Journal, 1(1):131–153.

Traxler, J. (2009). Current state of mobile learning. mobile learning: Transforming the delivery of education and training. URL: [link] (04.02. 2020).

Traxler, J., Barcena, E., and Laborda, J. G. (2015). Mobile technology for foreign language teaching: building bridges between non-formal and formal scenarios j. ucs special issue. Journal of Universal Computer Science, 21(10):1234–1247.

Whitton, N. (2009). Learning and teaching with computer games in higher education. In Games-Based learning advancements for multi-sensory human computer interfaces: Techniques and effective practices, pages 18–33. IGI Global.

Xu, Z., Chen, Z., Eutsler, L., Geng, Z., and Kogut, A. (2020). A scoping review of digital game-based technology on english language learning. Educational Technology Research and Development, 68:877–904.

Zaibon, S. B. and Shiratuddin, N. (2010). Mobile game-based learning (mgbl): Application development and heuristics evaluation strategy. Malaysian Journal of Learning and Instruction (MJLI), 7(2010):37–73.