Creating a guide for selecting evaluation methods for M-learning software technologies
DOI:
https://doi.org/10.5753/rbie.2021.29.0.73Keywords:
M-learning, Evaluation methods, Software quality in educational technologiesAbstract
M-learning is one of the main learning tools in the educational context that refers to mobility in learning, anywhere, anytime and by anyone. Considering that M-learning is, relatively, new, it is important to search for suitable quality evaluation methods. Thus, in this research, we aim to create a guide to select a suitable method for evaluating the quality of M-learning technologies. In this paper, we present the development of an initial version of a guide developed and a feasibility study. By conducting a literature review, we identified evaluation methods, quality characteristics, criteria and subcriteria to be considered in a M-learning application. Thus, we characterized each method found and quality criteria evaluated by each method. We created a guide as a dynamic table in excel. In order to evaluate the feasibility of the guide, we conducted a case study in a Software Quality class. The results show that more than 80% of evaluators agree that the guide is useful, easy to use and that they intend to use the guide in the future. However, some difficulties in using the guide were observed, due to lack or little experience with the tool used for creating the guide, excel. As future work, we intend to develop a software tool in order to improve the guide’s ease of use and its efficacy in suggesting evaluation methods.
Downloads
References
Andrade, M. V. M., Araújo Jr, C. F., & Silveira, I. F. (2015). Critérios de qualidade para aplicativos educacionais no contexto dos dispositivos móveis (M-Learning). In Nuevas Ideas em Informática Educativa. Congresso Internacional sobre Informática na Educação (TISE). [GS Search]
Ataíde, A. P. R., Teixeira, A. B. M., & da Silva, C. I. P. (2003). Maqsei-uma metodologia de avaliação de qualidade de software educacional infantil. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), (Vol. 1, No. 1, pp. 356-365). doi: 10.5753/cbie.sbie.2003.356-365 [GS Search]
Baloh, M., Zupanc, K., Košir, D., Bosnić, Z., & Scepanovic, S. (2015). A quality evaluation framework for mobile learning applications. In 4th Mediterranean Conference on Embedded Computing (MECO) (pp. 280-283). IEEE. doi: 10.1109/MECO.2015.7181923 [GS Search]
Barbosa, G., Oliveira, E., & D'Carlo, D. (2016). Usabilidade em aplicativos móveis educacionais: Um conjunto de heurísticas para avaliação. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE) (Vol. 27, No. 1, p. 777). doi: 10.5753/cbie.sbie.2016.777 [GS Search]
Bednarik, R., Gerdt, P., Miraftabi, R., & Tukiainen, M. (2004). Development of the TUP model-Evaluating educational software. In IEEE International Conference on Advanced Learning Technologies (pp. 699-701). IEEE. doi: 10.1109/ICALT.2004.1357627 [GS Search]
Cáceres, S. V. & Pow-Sang, J. A. (2018). A systematic mapping review of usability evaluation methods for educational applications on mobile devices. In 7th International Conference On Software Process Improvement (CIMPS), Guadalajara, Jalisco, Mexico, pp. 59-68, IEEE. doi: 10.1109/CIMPS.2018.8625629 [GS Search]
CAMPOS, G. H. B. D. (1989). Construção e validação de ficha de avaliação de produtos educacionais para microcomputadores. Rio de Janeiro. [GS Search]
Coomans, S., & Lacerda, G. S. (2015). Petese, a pedagogical ergonomic tool for educational software evaluation. Procedia Manufacturing, 3, 5881-5888. doi: 10.1016/j.promfg.2015.07.895 [GS Search]
Cota, C. X. N., Díaz, A. I. M., & Duque, M. Á. R. (2014). Developing a framework to evaluate usability in m-learning systems: Mapping study and proposal. In Second international conference on technological ecosystems for enhancing multiculturality (pp. 357-364). doi: 10.1145/2669711.2669924 [GS Search]
Da Silva, M. G., & Batista, S. C. F. (2015). Metodologia de avaliação: análise da qualidade de aplicativos educacionais para matemática do ensino médio. RENOTE-Revista Novas Tecnologias na Educação, 13(1). doi: 10.22456/1679-1916.57641 [GS Search]
Da Silva, R. S., Silva, W. R., Cardoso Filho, R., Pereira, W., Aguiar, Y., & Dantas, V. (2016). Avaliação de Software Educativo: a complexidade de escolher uma abordagem adequada. In Congresso sobre Tecnologias na Educação (Ctrl+E 2016), (pp. 116-126). [GS Search]
Gomes, A. S., & Wanderley, E. G. (2003). Elicitando requisitos em projetos de Software Educativo. In Anais do Workshop de Informática na Escola (Vol. 1, No. 1, pp. 119-130). doi: 10.5753/cbie.wie.2003.119-130 [GS Search]
Gomes, A. S., Castro Filho, J. A., Gitirana, V., Spinillo, A., Alves, M., Melo, M., & Ximenes, J. (2002). Avaliação de software educativo para o ensino de matemática. In WIE 2002 Workshop Brasileiro de Informática Educativa. Florianópolis: SBC. [GS Search]
ISO/IEC 25010 (2011). ISO/IEC 25010:2011, Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. [GS Search]
Kitchenham, B. & Charters, S. (2007) Guidelines for performing Systematic Literature Reviews in Software Engineering, Version 2.3, EBSE Technical Report EBSE-2007-01, Keele University and University of Durham. [GS Search]
Nunes, R., & Santos, I. (2018). A importância da avaliação ergonômico-pedagógica de aplicativos educacionais e os desafios encontrados no campo da aprendizagem móvel. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE) (Vol. 29, No. 1, p. 41). doi: 10.5753/cbie.sbie.2018.41 [GS Search]
Pereira, W. S., Cardoso Filho, R. J., Silva, W. R. D. A., Da Silva, R. S. T., Dantas, V. F., & Aguiar, Y. P. (2016). Validação de uma abordagem combinada para avaliação de software educativo: Avanços e Desafios. Revista Tecnologias na Educação, 16(8). [GS Search]
Petri, G., & von Wangenheim, C. G. (2017). How games for computing education are evaluated? A systematic literature review. Computers & education, 107, 68-90. doi: 10.1016/j.compedu.2017.01.004 [GS Search]
Petri, G., von Wangenheim, C. G., & Borgatto, A. F. (2016). MEEGA+: an evolution of a model for the evaluation of educational games. INCoD/GQS, 3. [GS Search]
Petri, G., von Wangenheim, C. G., & Borgatto, A. F. (2020). MEEGA+: Um Modelo para a Avaliação de Jogos Educacionais para o ensino de Computação. Revista Brasileira de Informática na Educação, 27(3), 52. doi: 10.5753/rbie.2019.27.03.52 [GS Search]
Porto, J. V., Barbosa, H., & von Wangenheim, C. G. (2018). Proposta de um Checklist de Avaliação de Usabilidade de Aplicativos Android no Contexto Educacional. In Computer on the Beach, (pp. 870-879). [GS Search]
Runeson, P., Host, M., Rainer, A., & Regnell, B. (2012). Case study research in software engineering: Guidelines and examples. John Wiley & Sons. [GS Search]
Soad, G. W., & Barbosa, E. F. (2017). MoLEva: um método de avaliação de qualidade para aplicativos educacionais móveis. In Simpósio Brasileiro de Qualidade de Software, (pp. 91-105). [GS Search]
Soad, G. W., Duarte Filho, N. F., & Barbosa, E. F. (2015). Uma contribuição ao estabelecimento de características de qualidade para aplicações educacionais móveis. In Simpósio Brasileiro de Qualidade de Software, (pp. 165–179). [GS Search]
Soad, G. W., Duarte Filho, N. F., & Barbosa, E. F. (2016). Quality evaluation of mobile learning applications. In 2016 IEEE Frontiers in Education Conference (FIE) (pp. 1-8). IEEE. doi: 10.1109/FIE.2016.7757540 [GS Search]
Vargo, J., Nesbit, J. C., Belfer, K., & Archambault, A. (2003). Learning object evaluation: computer-mediated collaboration and inter-rater reliability. International Journal of Computers and Applications, 25(3), 198-205. doi: 10.1080/1206212X.2003.11441703 [GS Search]
Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision sciences, 39(2), 273-315. doi: 10.1111/j.1540-5915.2008.00192.x [GS Search]
Vermeeren, A. P., Law, E. L. C., Roto, V., Obrist, M., Hoonhout, J., & Väänänen-Vainio-Mattila, K. (2010, October). User experience evaluation methods: current state and development needs. In 6th Nordic conference on human-computer interaction: Extending boundaries (pp. 521-530). doi: 10.1145/1868914.1868973 [GS Search]
Vieira, M. M. S., Simões, L. L. F., & de Oliveira Barreto A. L. (s.d) Avaliação de software educativo: aspectos pedagógicos e técnicos. [GS Search]
Zaibon, S. B., & Shiratuddin, N. (2010). Heuristics evaluation strategy for mobile game-based learning. In 2010 6th IEEE International Conference on Wireless, Mobile, and Ubiquitous Technologies in Education (pp. 127-131). IEEE. doi: 10.1109/WMUTE.2010.27 [GS Search]
Additional Files
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
