Are scale-based techniques enough for learners to convey their UX when using a Learning Management System?
DOI:
https://doi.org/10.5753/rbie.2019.27.01.104Keywords:
User Experience, Learning Management System, LMS, UX, EdmodoAbstract
As technology evolves, learning institutions have sought to invest in new learning technologies. Many universities have been adopting platforms designed to support the learning process, called Learning Management Systems (LMSs), to complement face-to-face learning, as well as to support distance learning. This widespread adoption of LMSs raises the need for evaluating them regarding aspects related to quality in use, such as User eXperience (UX). UX is very important in e-learning context, given that it may influence learner’s attitude towards the use of the LMS. Despite its importance, few studies evaluated the UX of LMSs. Moreover, researchers have been using generic techniques to evaluate them. In this context, there is no evidence of studies conducted to identify whether these techniques can fully capture the experience conveyed by these platforms. This paper presents an analysis of two UX evaluation techniques (User Experience Questionnaire, and Integrated Experience Acceptance Model) applied to an LMS called Edmodo. We carried out a study with 34 students, divided into two groups, to evaluate the UX of Edmodo and obtain their perceptions about the technique they used. We also gathered the difficulties faced by the students during the execution of the tasks on the platform, while correlating those difficulties with learning issues. In addition, we performed a qualitative analysis in order to better understand the results. Regarding Edmodo, the results showed that the students perceived its UX as positive. Regarding the techniques, both groups considered them easy to use and useful. However, around 41% of the students in both groups reported that they were not able to fully evaluate their UX using the techniques. The qualitative results allowed the identification of positive and negative aspects that researchers may consider during the improvement or development of new UX evaluation techniques.
Downloads
Referências
Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., & Rossano, V. (2006). An approach to usability evaluation of e-learning applications. Universal access in the information society, 4(3), 270-283. doi: 10.1007/s10209-005-0008-6. [GS Search]
Corbin, J., & Strauss, A. (2014). Basics of qualitative research. Sage. [GS Search]
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management science, 35(8), 982-1003. [GS Search]
DiStefano, C., Zhu, M., & Mindrila, D. (2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation, 14(20), 1-11. [GS Search]
Debevc, M., Stjepanovič, Z., Povalej, P., Verlič, M., & Kokol, P. (2007, July). Accessible and adaptive e-learning materials: considerations for design and development. In International Conference on Universal Access in Human-Computer Interaction (pp. 549-558). Springer, Berlin, Heidelberg. doi: 10.1007/978-3-540-73283-9_61. [GS Search]
Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior, 61, 463-471. doi: 10.1016/j.chb.2016.03.051. [GS Search]
Hassenzahl, M. (2003). The thing and I: understanding the relationship between user and product. In Funology (pp. 31-42). Springer Netherlands. doi: 10.1007/1-4020-2967-5_4. [GS Search]
ISO 9241-210 (2010). Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. Available at [Link].
Kalyuga, S. (2007). Enhancing instructional efficiency of interactive e-learning environments: A cognitive load perspective. Educational Psychology Review, 19(3), 387-399. doi: 10.1007/s10648-007-9051-6. [GS Search]
Kirakowski, J., & Corbett, M. (1993). SUMI: The software usability measurement inventory. British journal of educational technology, 24(3), 210-212. doi: 10.1111/j.1467-8535.1993.tb00076.x. [GS Search]
Laugwitz, B., Held, T., & Schrepp, M. (2008, November). Construction and evaluation of a user experience questionnaire. In Symposium of the Austrian HCI and Usability Engineering Group (pp. 63-76). Springer, Berlin, Heidelberg. doi: 10.1007/978-3-540-89350-9_6. [GS Search]
Mehlenbacher, B., Bennett, L., Bird, T., Ivey, M., Lucas, J., Morton, J., & Whitman, L. (2005, July). Usable e-learning: A conceptual model for evaluation and design. In Proceedings of HCI International (Vol. 2005, p. 11th). [GS Search]
Mtebe, J. S., & Kissaka, M. M. (2015, May). Heuristics for evaluating usability of learning management systems in Africa. In IST-Africa Conference, 2015 (pp. 1-13). IEEE. doi: 10.1109/ISTAFRICA.2015.7190521. [GS Search]
Nakamura, W. T., de Oliveira, E. H. T., & Conte, T. (2017). Usability and User Experience Evaluation of Learning Management Systems – A Systematic Mapping Study. In Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS 2017), 97-108. doi: 10.5220/0006363100970108. [GS Search]
Nakamura, W., Marques, L., Rivero, L., Oliveira, E., & Conte, T. (2017, October). Are Generic UX Evaluation Techniques Enough? A study on the UX Evaluation of the Edmodo Learning Management System. In Brazilian Symposium on Computers in Education (SBIE), 28(1), 1007-1016. doi: 10.5753/cbie.sbie.2017.1007. [GS Search]
Nielsen, J. (1994a). Heuristic evaluation. In Nielsen, J., and Mack, R. L. (Eds.), Usability Inspection Methods, John Wiley & Sons, New York, 25-64.
Nielsen, J. (1994b). Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 152-158). ACM. doi: 10.1145/191666.191729. [GS Search]
Oztekin, A., Kong, Z. J., & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. International Journal of Industrial Ergonomics, 40(4), 455-469. doi: 10.1016/j.ergon.2010.04.001. [GS Search]
Rivero, L., & Conte, T. (2017, October). A systematic mapping study on research contributions on UX evaluation technologies. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems (p. 5). ACM. doi: 10.1145/3160504.3160512. [GS Search]
Reeves, T. C., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., ... & Loh, S. (2002). Usability and instructional design heuristics for e-learning evaluation. In EdMedia: World Conference on Educational Media and Technology (pp. 1615-1621). Association for the Advancement of Computing in Education (AACE). [GS Search]
Santoso, H. B., Schrepp, M., Isal, R., Utomo, A. Y., & Priyogi, B. (2016). Measuring User Experience of the Student-Centered e-Learning Environment. Journal of Educators Online, 13(1), 58-79. [GS Search]
Shiratuddin, N., Hassan, S. & Landoni, M. (2003). A Usability Study for Promoting eContent in Higher Education. Journal of Educational Technology & Society, 6(4), 112-124. [GS Search]
Ssemugabi, S., & De Villiers, R. (2007, October). A comparative study of two usability evaluation methods using a web-based e-learning application. In Proceedings of the 2007 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries (pp. 132-142). ACM. doi: 10.1145/1292491.1292507. [GS Search]
Stoney, S., & Wild, M. (1998). Motivation and interface design: maximising learning opportunities. Journal of Computer Assisted Learning, 14(1), 40-50. doi: 10.1046/j.1365-2729.1998.1410040.x. [GS Search]
Sullivan, G. M., & Artino Jr, A. R. (2013). Analyzing and interpreting data from Likert-type scales. Journal of graduate medical education, 5(4), 541-542. doi: 10.4300/JGME-5-4-18. [GS Search]
Van Der Linden, J., & Van De Leemput, C. (2015). Observatory of students’ uses of computer-based tools. Psychologie française, 60(2), 145-157. doi: 10.1016/j.psfr.2015.02.002. [GS Search]
Van Schaik, P., & Ling, J. (2010). An integrated model of interaction experience for information retrieval in a Web-based encyclopaedia. Interacting with Computers, 23(1), 18-32. doi: 10.1016/j.intcom.2010.07.002. [GS Search]
Zaharias, P., & Pappas, C. (2016). Quality Management of Learning Management Systems: A User Experience Perspective. Current Issues in Emerging eLearning, 3(1), 5. [GS Search]
Zaharias, P. (2009). Usability in the context of e-learning: A framework augmenting ‘traditional'usability constructs with instructional design and motivation to learn. International Journal of Technology and Human Interaction (IJTHI), 5(4), 37-59. doi: 10.4018/jthi.2009062503. [GS Search]
Arquivos adicionais
Published
Como Citar
Issue
Section
Licença
Copyright (c) 2019 Walter T. Nakamura, Leonardo C. Marques, Luis Rivero, Elaine H. T. de Oliveira, Tayana Conte
Este trabalho está licenciado sob uma licença Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
