A Set of Heuristics for Evaluating Usability and User Experience in E-Government Information Systems
DOI:
https://doi.org/10.5753/isys.2026.6100Keywords:
Heuristic evaluation, User experience, Electronic government information systems, UsabilityAbstract
[Context] Given the expansion of digital public services and the need for more efficient, accessible, and citizen-centered systems, this article addresses the lack of domain-specific instruments to evaluate usability and user experience in electronic government systems. [Objective] The study aimed to propose and validate a set of heuristics designed exclusively for e-government information systems, capable of identifying domain-specific issues more accurately than generic heuristics. [Method] To achieve this goal, a formal eight-stage methodology was adopted, including a systematic literature review, heuristic evaluations of Brazilian government portals, correlational analysis between domain characteristics and UX/usability attributes, selection, specification, comparative validation, and refinement with specialists. [Results] The process resulted in 15 heuristics organized into thematic categories, covering aspects such as security and privacy, context, services, and navigation. In the comparative stage, the proposed set outperformed Nielsen’s traditional heuristics in identifying relevant issues in digital public systems. Focus group validation indicated strong alignment with the context, conceptual clarity, and practical applicability. [Conclusion/Contribution] It is concluded that the developed set enhances the evaluation of digital government systems by incorporating public-sector specificities, providing both methodological and practical contributions for researchers, designers, and managers interested in the continuous improvement of digital public services.
Downloads
References
Almarashdeh, I., & Alsmadi, M. (2016) Heuristic evaluation of mobile government portal services: An experts' review. In 2016 11th International Conference for Internet Technology and Secured Transactions (ICITST) (pp. 427-431). IEEE.
Alotaibi, S. J., & Wald, M. (2013) Evaluation of the UTAUT model for acceptable user experiences in Identity Access Management Systems. In 8th International Conference for Internet Technology and Secured Transactions (ICITST-2013) (pp. 232-237). IEEE.
Anganes, A., Pfaff, M. S., Drury, J. L., & O'toole, C. M. (2016) The heuristic quality scale. Interacting with computers, 28(5), 584-597.
Ardito, C., Costabile, M. F., Marsico, M. D., Lanzilotti, R., Levialdi, S., Roselli, T., & Rossano, V. (2006) An approach to usability evaluation of e-learning applications. Universal access in the information society, 4, 270-283.
Barbosa, S. D. J., Silva, B. D., Silveira, M. S., Gasparini, I., Darin, T., & Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuario. Auto publicação.
Bertot, J. C., & Jaeger, P. T. (2006) User-centered e-government: Challenges and benefits for government Web sites. Government Information Quarterly, 23(2), 163-168.
Bevan, N., & Curson, I. (1998) Planning and implementing user-centred design. In CHI 98 conference summary on Human factors in computing systems (pp. 111-112).
Bevan, N. (1995). Measuring usability as quality of use. Software Quality Journal, 4, 115-130.
Businesslink: Best practice in web design Crown. Disponível em: http://www.businesslink.gov.uk/bdotg/action/detail?r.l1=1073861197&r.l3=1075384855&r.t=RESOURCES&type=RESOURCES&itemId=1075384976&r.i=1075384964&r.l2=1074448623&r.s=m# Acessado em 01-Março-2025
Carroll, J. M. (1990) Infinite detail and emulation in an ontologically minimized HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 321-328).
Carroll, J. M., Kellogg, W. A., & Rosson, M. B. (1990) The task-artifact cycle (pp. 74-102). IBM Thomas J. Watson Research Division.
Coelho, J. I. M. Governo Eletrônico: um levantamento das iniciativas de e-Gov em municípios da Região Metropolitana de Manaus. Revista Científica Multidisciplinar Núcleo do Conhecimento, ano, 5, 31-56.
Dataprev. Disponível em https://www.dataprev.gov.br/conheca-dataprev-quem-somos/empresa. Acessado em 11-Setembro-2023.
Dilshad, R. M., & Latif, M. I. (2013) Focus group interview as a tool for qualitative research: An analysis. Pakistan Journal of Social Sciences, 33(1), 191-198.
Eidaroos, A., & Alkraiji, A. (2015) Evaluating the usability of library websites using an heuristic analysis approach on smart mobile phones: preliminary findings of a study in Saudi universities. In New Contributions in Information Systems and Technologies: Volume 1 (pp. 1141-1152). Springer International Publishing.
Fogg, B. J. (2002). Persuasive technology: using computers to change what we think and do. Ubiquity.
García, E., García, A., De-Marcos, L., Otón, S., & Hilera, J. R. (2012) Semiautomatic Evaluation of Websites Usability. In Proceedings of the Fifth International Conference on Advances in Computer-Human Interactions (ACHI) (p. 165169).
Garcia, A. C. B., Maciel, C., & Pinto, F. B. A (2005) quality inspection method to evaluate e-government sites. In International Conference on Electronic Government (pp. 198-209). Berlin, Heidelberg: Springer Berlin Heidelberg.
Garrido, M., Lavin, C., & Pena, N. R. (2014) Detecting usability problems and offering lines of solutions: An instrument' proposal for measuring usability in online services. IEEE Latin America Transactions, 12(1), 9-16.
Gerhardt‐Powals, J. (1996) Cognitive engineering principles for enhancing human‐computer performance. International Journal of Human‐Computer Interaction, 8(2), 189-211.
Governo Digital. Disponível em https://www.gov.br/governodigital/pt-br/transformacao-digital/ferramentas/pesquisa-com-usuarios. Acessado em 16-Nov-2021
Governo Digital. Disponível em https://www.gov.br/governodigital/pt-br/estrategia-de-governanca-digital/do-eletronico-ao-digital . Acessado em 28-Agosto-2023.
Governo Digital. Disponível em https://www.gov.br/governodigital/pt-br/estrategias-e-governanca-digital/estrategianacional. Acessado em 17-Janeiro-2025.
Huang, Z., & Benyoucef, M. (2014) Usability and credibility of e-government websites. Government information quarterly, 31(4), 584-595.
Hughes, J., Ahluwalia, P., & Midha, V. . (2013) A heuristic evaluation instrument for e–government online software. Electronic Government, An International Journal, 10(1), 1-18
Iso, B., & Standard, B. (2010) Ergonomics of human-system interaction. British Standards Institution.
Ivory, M. Y. (2001) An empirical foundation for automated web interface evaluation. University of California, Berkeley.
Júnior, S. F. R., & Leite, L. L. (2024.) Um estudo sobre problemas de usabilidade no software de processos administrativos eletrônicos do Governo Federal do Brasil. iSys-Brazilian Journal of Information Systems, 17(1), 6-1,
Kitchenham, B. (2004) Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004), 1-26.
Krueger, R. A., & Casey, M. A. (2000). Focus groups: A practical guide for applied research, 4th ed. Thousand Oaks, CA: Sage Publications Inc. NIELSEN, J. Usability engineering at a discount. In Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.) (pp. 394-401). (1989)
Nielsen, J., & Molich, R. (1990) Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 249-256).
Nielsen, J. (1994). Usability engineering. Morgan Kaufmann. (1994)
Nielsen, J. (2005). Ten usability heuristics. (2005)
Oliveira, R., & Cappelli, C. (2024) UX e Linguagem Simples na Web: Práticas para um Design de Interação mais compreensível. Minicursos do SBSI, 31.
Padrão Digital de Governo. Disponível em https://www.gov.br/ds/introducao/sobre. Acessado em 11-Setembro-2023.
Quiñones, D., & Rusu, C. (2019) Applying a methodology to develop user eXperience heuristics. Computer Standards & Interfaces, 66, 103345.
Quiñones, D., Rusu, C., & Roncagliolo, S. (2014) Redefining usability heuristics for transactional web applications. In 2014 11th International Conference on Information Technology: New Generations (pp. 260-265). IEEE.
Quiñones, D., Rusu, C., & Rusu, V. (2018) A methodology to develop usability/user experience heuristics. Computer standards & interfaces, 59, 109-129.
Quiñones, D., Rusu, C., Roncagliolo, S., Rusu, V., & Collazos, C. A. (2016) Developing usability heuristics: a formal or informal process?. IEEE Latin America Transactions, 14(7), 3400-3409.
Quiñones, D., & Rusu, C. (2017). How to develop usability heuristics: A systematic literature review. Computer standards & interfaces, 53, 89-122
Redish, J., & Barnum, C. (2011) Overlap, influence, intertwining: The interplay of UX and technical communication. Journal of Usability Studies, 6(3), 90-101
Rusu, C., Roncagliolo, S., Rusu, V., & Collazos, C. A (2011). Methodology to establish usability heuristics
Rusu, C., Rusu, V., Roncagliolo, S., & González, C. (2015). Usability and user experience: what should we care about? International Journal of Information Technologies and Systems Approach (IJITSA), 8(2), 1-12.
Sarantis, D., Soares, D., & Carvalho, J. (2020) HSWAI: a health sector website assessment instrument. In Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance (pp. 359-368).
Schulze, K., & Krömker, H. (2010) A framework to measure user experience of interactive online products. In Proceedings of the 7th international conference on methods and techniques in behavioral research (pp. 1-5).
Shneiderman, B., & Plaisant, C. (2010) Designing the user interface: strategies for effective human-computer interaction. Pearson Education India.
Smith, A. (2003) Homepage Usability: 50 Web sites Deconstructed. Online Information Review, 27(4), 293-294.
Ssemugabi, S., & De Villiers, M. R. (2016) Make your choice: dimensionality of an open integrated conceptual model for evaluating E-service quality, usability and user experience (e-SQUUX) of Web-based applications. In Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists (pp. 1-10).
Venkatesh, V., Hoehle, H., & Aljafari, R. (2014) A usability evaluation of the Obamacare website. Government information quarterly, 31(4), 669-680.
Wallace, R. (2002) Usability for the Web: Designing Web Sites that Work. Journal of the Association for Information Science and Technology, 53(9), 775.
Wharton, C., Bradford, J., Jeffries, R., & Franzke, M. (1992) Applying cognitive walkthroughs to more complex user interfaces: Experiences, issues, and recommendations. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 381-388).
Youngblood, N. E., & Mackiewicz, J. (2012) A usability analysis of municipal government website home pages in Alabama. Government Information Quarterly, 29(4), 582-588.
Zabed Ahmed, S. M. (2008) A comparison of usability techniques for evaluating information retrieval system interfaces. Performance Measurement and Metrics, 9(1), 48-58.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 iSys - Journal of Information Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
