Enhancing Usability Assessment with a Novel Heuristics-Based Approach Validated in an Actual Business Setting

Afra Pascual Almenara; Toni Granollers Saltiveri; Juan Enrique G. Navarro; Marta Albets Mitjaneta

doi:10.5753/jis.2024.4078

Authors

Afra Pascual Almenara Universitat de Lleida https://orcid.org/0000-0002-2368-755X
Toni Granollers Saltiveri Universitat de Lleida https://orcid.org/0000-0001-9189-7308
Juan Enrique G. Navarro Universidad de Castilla-La Mancha https://orcid.org/0000-0003-4402-1076
Marta Albets Mitjaneta Universitat de Lleida https://orcid.org/0000-0001-7855-0844

DOI:

https://doi.org/10.5753/jis.2024.4078

Keywords:

Heuristic evaluation, usability evaluation, experts of usability, usability percentage

Abstract

We have long been committed to improving Usability Evaluation and one of the proposals we have been working on the most is the use and improvement of the Heuristic Evaluation (HE) technique. With this in mind, we proposed an improvement which was tested in an experiment. This article describes an experiment carried out in a real business professional context. Fifteen usability experts from a reputable company evaluated eight websites (four supermarket and four bank platforms) using our HE proposal for the first time in real-world scenarios. This experimentation analyzed two main aspects: firstly, whether individual or group evaluations affect the final result, and secondly, whether the heuristic evaluation technique is effective in a real business and professional context. Regarding the Usability Percentage (UP) event, the results indicate that there was little difference between group and individual evaluations. The mean UP for the group was 57.88%, while the mean UP for individuals was 56.66%. The experiment provided sufficient information to suggest a new version of our HE methodology, specifically designed to improve results in real-life contexts. Furthermore, the experiment’s findings support the proposal of this new methodology, which is better suited to the business environment.

Downloads

Download data is not yet available.

References

Alcaraz, R., Ribera Turró, M., and Granollers, T. (2021a). Methodology for heuristic evaluation of the accessibility of statistical charts for people with low vision and color vision deficiency. Universal Access in the Information Society, 21. DOI: https://doi.org/10.1007/s10209-021-00816-0.

Alcaraz, R., Ribera Turró, M., Marcelino, J., PascualAlmenara, A., and Granollers, T. (2021b). Accessible charts are part of the equation of accessible papers: a heuristic evaluation of the highest impact lis journals. Library Hi Tech, ahead-of-print. DOI: https://doi.org/10.1108/LHT-08-2020-0188.

Allam, A. H., Hussin, A. R. C., and Dahlan, H. M. (2013). User experience: challenges and opportunities.

Bevan, N., Kirakowski, J., and Maissel, J. (1991). What is usability.

Bonastre, L. and Granollers, T. (2014). A set of heuristics for user experience evaluation in e-commerce websites. In International Conference on Advances in Computer-Human Interaction.

Cheiran, J. F. P., Rodrigues, A., and Pimenta, M. S. (2021). Virtual look around: comparing presence, cybersickness and usability for virtual tours across different devices. Journal on Interactive Systems, 12(1):191–205. DOI: https://doi.org/10.5753/jis.2021.2063.

Clark, L. A. and Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological assessment.

Derby, J. L. and Chaparro, B. S. (2021). The challenges of evaluating the usability of augmented reality (ar). Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 65(1):994–998. DOI: https://doi.org/10.1177/1071181321651315.

Dix, A., Finlay, J., Abowd, G. D., and Beale, R. (2003). Human Computer Interaction. Pearson Prentice Hall, Harlow, England, 3 edition.

Fernández, J. and Macías, J. A. (2021). Heuristic-based usability evaluation support: A systematic literature review and comparative study. In Proceedings of the XXI International Conference on Human Computer Interaction, Interacción ’21, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3471391.3471395.

Gerlach, J. H. and Kuo, F.-Y. (1991). Understanding human-computer interaction for information systems design. MIS Q., 15(4):527–549.

Goldbort, R. (2016). Usability evaluation, Chap. 15. Interaction Design Foundation.

Gonzalez MP, Granollers T, P. A. L. J. (2008). Testing website usability in spanish-speaking academia through heuristic evaluation and cognitive walkthroughs. In Journal Universal Comput Sci (JUCS), page 14(9):1513–29. Special Issue “Designing the Human–Computer Interaction: Trends and Challenges”. In: Bravo C, Redondo MA, Ortega M, editors. Graz University of Technology, ISSN: 0948-69.

González, M. P.; Pascual, A. L. J. (2001). Evaluación Heurística. Introducción a la Interacción Persona-Ordenador. AIPO: Asociación Interacción Persona-Ordenador.

Graham, L. M. (2007). Gestalt theory in interactive media design.

Granollers, T. (2010a). Template heuristic evaluation ecommerce environment. Available: [link]. Access: 01 July 2024.

Granollers, T. (2010b). Template heuristic evaluation generic environment. Available: [link]. Access: 01 July 2024.

Granollers, T. (2018a). Usability evaluation with heuristics , beyond nielsen ’ s list. Available: [link]. Access on 01 July 2024.

Granollers, T. (2018b). Usability evaluation with heuristics. new proposal from integrating two trusted sources. In Design, User Experience, and Usability: Theory and Practice: 7th International Conference, DUXU 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part I, page 396–405, Berlin, Heidelberg. Springer-Verlag. DOI: https://doi.org/10.1007/978-3-319-91797-9_28.

Hvannberg, E. T., Law, E. L.-C., and Lérusdóttir, M. K. (2006). Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with Computers, 19(2):225–240. DOI: https://doi.org/10.1016/j.intcom.2006.10.001.

ISO/IEC (2017). ISO/IEC TS 25011:2017(en). Information technology — Systems and software Quali-ty Requirements and Evaluation (SQuaRE) — Service quality models.

ISO/IEC (2020). ISO/CD 9241-11. Ergonomics of human-system interaction–Part11: Ergonomics of human-system interaction — Part 110: Interaction principles.

Lewis, J. R. (1991). Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The asq. SIGCHI Bull., 23(1):78–81. DOI: https://doi.org/10.1145/122672.122692.

Lorés J, Gonzalez MP, P. A. (2005). Primera etapa de la inicitativa usabaipo: usabilidad de páginas de inicio de las universidades españolas. VI Congreso de Interacción persona ordenador, 1(1):217–221.

Mutlu, T. (2023). Usability Evaluation Methods: How Usable Are They? Global Studies on Management Information Systems. DOI: https://doi.org/10.26650/B/SS28ET06.2023.006.16.

Nielsen, J. (1994). Usability Engineering. Capcher 5. Usability Heuristics. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Nielsen, J. and Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’90, page 249–256, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/97243.97281.

Pascual-Almenara, A. and Granollers-Saltiveri, T. (2021). Combining two inspection methods: Usability heuristic evaluation and wcag guidelines to assess e-commerce websites. In Ruiz, P. H., Agredo-Delgado, V., and Kawamoto, A. L. S., editors, Human-Computer Interaction, pages 1–16, Cham. Springer International Publishing.

Patnaik, M. and Adrian, A. M. (2022). Chapter 4 - a perspective depiction of heuristics in virtual reality. In Mishra, S., Tripathy, H. K., Mallick, P. K., Sangaiah, A. K., and Chae, G.-S., editors, Cognitive Big Data Intelligence with a Metaheuristic Approach, Cognitive Data Science in Sustainable Computing, pages 101–116. Academic Press. DOI: https://doi.org/10.1016/B978-0-323-85117-6.00006-6.

Preece, J., Rogers, Y., and Sharp, H. (2015). Interaction Design: Beyond Human-Computer Interaction. Wiley, Hoboken, NJ, 4 edition.

Yablonski, J. (2020). Laws of UX: Using psychology to design better products services. O’Reilly Media.