Evaluation of the Learner eXperience of HCI course in a Collaborative Context: an Exploratory Study and Lessons Learned

Gabriela Corbari dos Santos; Deivid Eive S. Silva; Rachel Carlos Duque Reis; Natasha M. Costa Valentim

doi:10.5753/jis.2026.7447

Authors

Gabriela Corbari dos Santos Federal University of Paraná https://orcid.org/0000-0003-0864-1534
Deivid Eive S. Silva Federal University of Western Pará https://orcid.org/0000-0003-1066-0750
Rachel Carlos Duque Reis Federal University of Paraná https://orcid.org/0000-0002-3747-9508
Natasha M. Costa Valentim Federal University of Paraná https://orcid.org/0000-0002-6027-3452

DOI:

https://doi.org/10.5753/jis.2026.7447

Keywords:

Learner eXperience, HCI, Practical Work, Lessons Learned, Exploratory Study, Collaborative Context

Abstract

Evaluating Learner eXperience (LX) in collaborative educational settings allows identifying how communication, coordination, and cooperation among peers influence learning, engagement and the construction of knowledge. In the sinse, the aims of this article is to present a study that evaluates LX, as well as to share feedback and insights from the instructor and a learner assistant in a collaborative learning environment supported by interactive systems. This study was conducted the Human-Computer Interaction (HCI) course, involving learners and two practical works: a usability, UX, and accessibility evaluation of a Brazilian Sign Language (Libras) translation app; and the specification, modeling, prototyping, and evaluation of an app for use by visually impaired individuals. Perceptions were collected from learners, the professor, and a assistant learner in the course regarding the observed and lived experiences. Additionally, a questionnaire was applied to the learners for LX evaluation. The data were analyzed quantitatively and qualitatively. The results show that interactive systems positively supported learner interaction by fostering knowledge exchange, creativity, and a more critical perspective. These findings reinforce the importance of evaluating educational systems that promote communication, coordination, and cooperation, contributing to the advancement of LX research within the HCI field.

Downloads

Download data is not yet available.

References

Almeida, L. D. A. and Baranauskas, M. C. C. (2010). Universal design principles combined with web accessibility guidelines: a case study. In Proceedings of the IX Symposium on Human Factors in Computing Systems, IHC '10, page 169–178, Porto Alegre, BRA. Brazilian Computer Society. DOI: https://doi.org/10.5555/1999593.1999612.

Barbosa, S. and Silva, B. (2010). Interação humano-computador. Elsevier Brasil.

Barbosa, S. D. J., Silva, B. S. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação Humano-Computador e Experiência do Usuário. Autopublicação.

Bastien, C. and Scapin, D. (1993). Ergonomic criteria for the evaluation of human-computer interfaces. PhD thesis, Inria. [link], Accessed: 13 July 2026.

Benyon, D. I. H.-C. T. and de Souza, H. C. (2011). revisão técnica: Illana de almeida s. concílio. [link], Accessed: 13 July 2026.

Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2):77–101. DOI: https://doi.org/10.1191/1478088706qp063oa.

Brooke, J. et al. (1996). Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4–7. DOI: https://doi.org/10.1201/9781498710411-35.

Caldwell, B., Cooper, M., Reid, L. G., Vanderheiden, G., Chisholm, W., Slatin, J., and White, J. (2008). Web content accessibility guidelines (wcag) 2.0.

Coleti, T. A. and Morandini, M. (2012). Atividades de ensino de ihc em duas instituiçoes de ensino superior brasileiras. [link], Accessed: 13 July 2026.

Corbin, J. and Strauss, A. (2014). Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.

Correa, C. M., de Freitas, G. V. M., dos Santos Eberhardt, A. L., and Silveira, M. S. (2021). From now on: Experiences from user-based research in remote settings. In Proceedings of the XX Brazilian Symposium on Human Factors in Computing Systems, IHC '21, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3472301.3484334.

da Silva Junior, D. P., Alves, D. D., Carneiro, N., Matos, E. d. S., Baranauskas, M. C. C., and Mendoza, Y. L. M. (2024). Grandihc-br 2025-2035 - gc1: New theoretical and methodological approaches in hci. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC '24, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3702038.3702054.

De Souza, C. S. (2005). The semiotic engineering of human-computer interaction. MIT press.

Dos Santos, G. C., Dos S. Silva, D. E., and C. Valentim, N. M. (2023). Proposal and preliminary evaluation of a learner experience evaluation model in information systems. In Proceedings of the XIX Brazilian Symposium on Information Systems, SBSI '23, page 308–316, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3592813.3592919.

Dos Santos, G. C., Dos S. Silva, D. E., and M. C. Valentim, N. (2024a). Feasibility study of a model that evaluates the learner experience: A quantitative and qualitative analysis. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, IHC '23, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3638067.3638119.

dos Santos, G. C., Silva, D., Reis, R. D., and Valentim, N. (2025a). Exploratory study on the learner experience in a collaborative learning context using computational resources. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU, pages 950–957. INSTICC, SciTePress. DOI: https://doi.org/10.5220/0013477600003932.

dos Santos, G. C., Silva, D., and Valentim, N. (2025b). Teaching topics in human-computer interaction: A practical experience with a focus on experimental research. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU, pages 776–783. INSTICC, SciTePress. DOI: https://doi.org/10.5220/0013340300003932.

Dos Santos, G. C., Silva, D. E., Peres, L. M., and Valentim, N. M. C. (2024b). Case study of a model that evaluates the learner experience with dicts. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA '24, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3613905.3637138.

dos Santos, G. C. and Valentim, N. (2025). Percepções e experiências sobre o ensino e aprendizado de uma disciplina de ihc com foco em práticas colaborativas. In Anais do XXIV Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 173–195, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/ihc.2025.10847.

Ellis, C. A., Gibbs, S. J., and Rein, G. (1991). Groupware: some issues and experiences. Commun. ACM, 34(1):39–58. DOI: https://doi.org/10.1145/99977.99987.

Ferreira, K. G., de Curso, M. d. F., and da Silva, C. I. P. (2002). Teste de usabilidade. [link], Accessed: 13 July 2026.

Fuks, H., Raposo, A., Gerosa, M. A., Pimentel, M., Filippo, D., and Lucena, C. (2008). Inter- and intrarelationships between communication coordination and cooperation in the scope of the 3c collaboration model. In 2008 12th International Conference on Computer Supported Cooperative Work in Design, pages 148–153. DOI: https://doi.org/10.1109/CSCWD.2008.4536971.

Godoi, T. X., da Silva Junior, D. P., and Costa Valentim, N. M. (2020). A case study about usability, user experience and accessibility problems of deaf users with assistive technologies. In Universal Access in Human-Computer Interaction. Applications and Practice: 14th International Conference, UAHCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22, pages 73–91. Springer. DOI: https://doi.org/10.1007/978-3-030-49108-6_6.

Hanlon, R., Barry, M., Marrinan, F., and O'Sullivan, D. (2021). Towards an effective user interface for data exploration, data quality assessment and data integration. In 2021 IEEE 15th International Conference on Semantic Computing (ICSC), pages 431–436. DOI: https://doi.org/10.1109/ICSC50631.2021.00077.

Hartmann, A. C., Maronn, T. G., and Santos, E. G. (2019). A importância da aula expositiva dialogada no ensino de ciências e biologia. [link], Accessed: 13 July 2026.

Hassenzahl, M., Burmester, M., and Koller, F. (2003). Attrakdiff: Ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität. Mensch & Computer 2003: Interaktion in Bewegung, pages 187–196. DOI: https://doi.org/10.1007/978-3-322-80058-9_19.

Huang, R., Spector, J. M., and Yang, J. (2019). Educational Technology a Primer for the 21st Century. Springer.

Inostroza, R., Rusu, C., Roncagliolo, S., and Rusu, V. (2013). Usability heuristics for touchscreen-based mobile devices: update. In Proceedings of the 2013 Chilean Conference on Human - Computer Interaction, ChileCHI '13, page 24–29, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/2535597.2535602.

ISO 9241-210 (2019). Ergonomics of human-system interaction — part 210: Human-centred design for interactive systems. [link], Accessed: 13 July 2026.

John, B. (1995). Why goms? interactions, 2(4):80–89. DOI: https://doi.org/10.1145/225362.225374.

Kaptelinin, V. and Nardi, B. A. (2009). Acting with technology: Activity theory and interaction design. MIT press.

Kawano, A., Motoyama, Y., and Aoyama, M. (2019). A lx (learner experience)-based evaluation method of the education and training programs for professional software engineers. In Proceedings of the 2019 7th International Conference on Information and Education Technology, ICIET 2019, page 151–159, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3323771.3323789.

Lang, P. (1980). Behavioral treatment and biobehavioral assessment: Computer applications. [link], Accessed: 13 July 2026.

Laugwitz, B., Held, T., and Schrepp, M. (2008). Construction and evaluation of a user experience questionnaire. In Holzinger, A., editor, HCI and Usability for Education and Work, pages 63–76, Berlin, Heidelberg. Springer Berlin Heidelberg. DOI: https://doi.org/10.1007/978-3-540-89350-9_6.

Law, E., Roto, V., Vermeeren, A. P., Kort, J., and Hassenzahl, M. (2008). Towards a shared definition of user experience. In CHI '08 Extended Abstracts on Human Factors in Computing Systems, CHI EA '08, page 2395–2398, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/1358628.1358693.

Lazar, J., Feng, J. H., and Hochheiser, H. (2017). Research methods in human-computer interaction. Morgan Kaufmann.

Maeda, J. (2006). The laws of simplicity (simplicity: design, technology, business, life). The MIT Press Cambridge.

Marcus, A. (2002). Dare we define user-interface design? Interactions, 9(5):19–24. DOI: https://doi.org/10.1145/566981.566992.

Marques, A. B. d. S. and Melo, Á. H. d. S. (2015). Relato de experiência no ensino de ihc em nível de graduação através de uma abordagem interdisciplinar. [link], Accessed: 13 July 2026.

Melo, Áurea C. F., Jardim, M., and Barreto, R. (2017). Modelo 3c de colaboração aplicado ao uso de um repositório para o desenvolvimento de interfaces para autistas. In Anais do XIV Simpósio Brasileiro de Sistemas Colaborativos, pages 1471–1485, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/sbsc.2017.9967.

Miranda, P., Viana, J., Nascimento, E., and Portela, C. (2019). O uso de estratégias de gamificação em uma disciplina de ihc: Um relato de experiência. In Anais Estendidos do XVIII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 94–99, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/ihc.2019.8407.

MPO, M. d. P., e Gestão, O., de Logística, S., and da Informação, T. (2014). Modelo de Acessibilidade em Governo Eletrônico (eMAG). Brasília. [link], Accessed: 13 July 2026.

Neto, F. S. d. S. and da Silva, E. V. (2018). itimeline: Uma abordagem visual das interações sociais em ambientes virtuais de aprendizagem baseada no modelo 3c de colaboração. RENOTE, 16(1). DOI: https://doi.org/10.22456/1679-1916.85904.

Nielsen, J. (1995). Usability inspection methods. In Conference Companion on Human Factors in Computing Systems, CHI '95, page 377–378, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/223355.223730.

Nygren, E., Blignaut, A. S., Leendertz, V., and Sutinen, E. (2019). Quantitizing affective data as project evaluation on the use of a mathematics mobile game and intelligent tutoring system. Informatics in Education, 18(2):375–402. DOI: https://doi.org/10.15388/infedu.2019.18.

O'Brien, H. L. and Toms, E. G. (2008). What is user engagement? a conceptual framework for defining user engagement with technology. Journal of the American society for Information Science and Technology, 59(6):938–955. DOI: https://doi.org/10.1002/asi.20801.

Oliveira, A. and Eler, M. (2015). Acessibilidade em governo eletrônico: um estudo sobre a aplicação de padrões web em sítios gov.br. In Anais do XI Simpósio Brasileiro de Sistemas de Informação, pages 691–698, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/sbsi.2015.5877.

Oliveira, E. (2020). Ihc no ensino remoto emergencial: Relato de experiência. In Anais Estendidos do XIX Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 31–36, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/ihc.2020.14051.

Pereira, R., Darin, T., and Silveira, M. S. (2024). Grandihc-br: Grand research challenges in human-computer interaction in brazil for 2025-2035. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC '24, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3702038.3702061.

Pimentel, M., Gerosa, M. A., Filippo, D., Raposo, A., Fuks, H., and Lucena, C. J. P. d. (2006). Modelo 3c de colaboração para o desenvolvimento de sistemas colaborativos. [link], Accessed: 13 July 2026.

Preece, J., Rogers, Y., and Sharp, H. (2015). Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, 4st edition.

Rebelo, S. M. M. (2016). Abordagens participativas para os sistemas de orientação e sinalização: O caso de estudo do colégio das artes. Master's thesis, Universidade de Coimbra (Portugal). [link], Accessed: 13 July 2026.

Rosa, J. a. R. d. S. and Valentim, N. M. C. (2022). Investigating the use of uaudg-vi in the mobile applications design for visually impaired considering accessibility, usability and ux: a feasibility study and an evaluation with experts. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems, IHC '22, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3554364.3561829.

Schmidt, M. and Huang, R. (2022). Defining learning experience design: Voices from the field of learning design & technology. TechTrends, 66(2):141–158. DOI: https://doi.org/10.1007/s11528-021-00656-y.

Shi, L. (2014). Defining and Evaluating Learner Experience for Social Adaptive E-Learning. In Neykova, R. and Ng, N., editors, 2014 Imperial College Computing Student Workshop, volume 43 of OpenAccess Series in Informatics (OASIcs), pages 74–82, Dagstuhl, Germany. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik. DOI: https://doi.org/10.4230/OASIcs.ICCSW.2014.74.

Shneiderman, B. and Plaisant, C. (2010). Designing the user interface: strategies for effective human-computer interaction. Pearson Education.

Silva, D. E. d. S., Guerino, G. C., and Valentim, N. M. C. (2023). Analyzing the learners' experience of an experimental hci course in a remote context. Journal on Interactive Systems, 14(1):341–353. DOI: https://doi.org/10.5753/jis.2023.3243.

Soegaard, M. and Dam, R. F. (2012). The encyclopedia of human-computer interaction. [link], Accessed: 13 July 2026.

Soloway, E., Guzdial, M., and Hay, K. E. (1994). Learner-centered design: The challenge for hci in the 21st century. Interactions, 1(2):36–48. DOI: https://doi.org/10.1145/174809.174813.

Stanton, N. A. (2006). Hierarchical task analysis: Developments, applications, and extensions. Applied Ergonomics, 37(1):55–79. Special Issue: Fundamental Reviews. DOI: https://doi.org/10.1016/j.apergo.2005.06.003.

Swedberg, R. (2020). Exploratory research. [link], Accessed: 13 July 2026.

Vieira, H. and Baranauskas, M. C. C. (2003). Design e avaliação de interfaces humano-computador. Campinas: Unicamp. [link], Accessed: 13 July 2026.

Wharton, C., Rieman, J., Lewis, C., and Polson, P. (1994). The cognitive walkthrough method: a practitioner's guide, page 105–140. John Wiley & Sons, Inc., USA. DOI: https://doi.org/10.5555/189200.189214.

Zhang, Z., Sun, B., and An, P. (2025). Breaking barriers or building dependency? exploring teamllm collaboration in ai-infused classroom debate. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI '25, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3706598.3713853.