SIM Family: The Semiotic Inspection Method and Its Variants

Caíque Bruno Fortunato; Ricardo Bruno da Cruz Costa; Raquel Oliveira Prates

doi:10.5753/jis.2026.7619

Authors

Caíque Bruno Fortunato Federal University of Minas Gerais https://orcid.org/0009-0002-1876-7632
Ricardo Bruno da Cruz Costa Federal University of Minas Gerais https://orcid.org/0000-0002-7569-5391
Raquel Oliveira Prates Federal University of Minas Gerais https://orcid.org/0000-0002-7128-4974

DOI:

https://doi.org/10.5753/jis.2026.7619

Keywords:

Semiotic Inspection Method, SIM, Communicability, Evaluation, Semiotic Engineering

Abstract

Background: The Semiotic Inspection Method (SIM) is a qualitative evaluation method grounded in Semiotic Engineering that focuses on the assessment of communicability in interactive systems through the analysis of metalinguistic, static, and dynamic signs. As interactive technologies have evolved toward more complex scenarios, limitations in the scope of the original method have motivated the proposal of several adaptations. Purpose: This study aims to provide a systematic overview of the adaptations and modifications of the Semiotic Inspection Method, clarifying their motivations, methodological changes, and application contexts. Methods: A systematic literature review was conducted using Google Scholar as the primary search platform. The search strategy included the full name of the method in English and Portuguese and resulted in an initial corpus of 604 records. After processing and classification, studies proposing structural modifications to SIM were analyzed. A web-based platform, SIM Studies, was developed to support data organization, classification, and visualization. Results: We have identified five distinct variants: Scientific SIM, Cross-Platform SIM (CP-SIM), SIM with the addition of Deictic Signs, SIM mediated by Screen Reader (SIM-SR), and SIM for Human–Computer Integration (SIM-HInt). These variants introduce methodological extensions such as new procedural steps, additional classes of signs, mediated interaction analysis, and the inspection of distributed systems. Conclusion: The proposed variants maintain the SIM’s underlying theoretical grounding. Together with SIM, they can be considered a “family” of methods, as they apply to different contexts and are complementary to the original method. By identifying and conducting a comparative analysis of these adaptations, this study contributes to organize the “SIM family”.

Downloads

Download data is not yet available.

References

Barbosa, G. and Prates, R. (2022). Estendendo a IHC para HInt à luz da engenharia semiótica. In Anais Estendidos do XXI Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC 2022), pages 166–174, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/ihc_estendido.2022.226191.

Barbosa, G. A. R. (2022). Extending the ontology, meta-communication and communicability of semiotic engineering to the emerging paradigm of human-computer integration (HInt). In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (IHC '22), pages 1–11, Diamantina, Brazil. ACM. DOI: https://doi.org/10.1145/3554364.3559109.

Barbosa, G. A. R. (2024). Extensão da interação para a integração humano-computador à luz da teoria da engenharia semiótica. Tese de doutorado, Universidade Federal de Minas Gerais, Belo Horizonte.

Barbosa, G. A. R., Fernandes, U. d. S., Santos, N. S., and Prates, R. O. (2024). Human-computer integration as an extension of interaction: Understanding its state-of-the-art and the next challenges. International Journal of Human–Computer Interaction, 40(11):2761–2780. DOI: https://doi.org/10.1080/10447318.2023.2177797.

Barbosa, S. D. J. and Silva, B. S. d. (2010). Interação Humano-Computador. Elsevier, Rio de Janeiro.

Barbosa, S. D. J., Silva, B. S. d., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação Humano-Computador e Experiência do Usuário. Autopublicação, Rio de Janeiro.

Cardoso, D. d. A., Henriques, F. d. R., and Belloze, K. T. (2023). Ontology visualization in BioPortal: Methodological triangulation for analyzing accessibility, communicability, and usability. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems (IHC '23), pages 1–11, Maceió, Brazil. ACM. DOI: https://doi.org/10.1145/3638067.3638088.

Carroll, J. M., editor (2003). HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science. Morgan Kaufmann, San Francisco.

Carvalho, L. P. (2021). MIS-LT: um método para avaliação de acessibilidade e comunicabilidade de sistemas interativos. Dissertação de mestrado, Universidade Federal de Minas Gerais, Belo Horizonte. Available at: https://hdl.handle.net/1843/57622. Accessed on 21 September 2026.

Carvalho, L. P., Prates, R. O., and Freire, A. P. (2019). A proposal to adapt the semiotic inspection method to analyze screen reader mediated interaction. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (IHC '19), pages 1–11, Vitória, ES, Brazil. ACM. DOI: https://doi.org/10.1145/3357155.3358455.

Carvalho, L. P., Santos, M. A. d., Souza, J. G. R. d., Prates, R. O., and Freire, A. P. (2021a). Aplicação do método de inspeção semiótica mediado por leitores de tela ao site TudoGostoso. Relatório técnico, Departamento de Ciência da Computação (DCC), Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, MG, Brasil. Available at: [link]. Accessed on 21 September 2026.

Carvalho, L. P., Santos, M. A. d., Souza, J. G. R. d., Prates, R. O., and Freire, A. P. (2021b). Beyond barriers: A practical analysis of the SIM-SR method to inspect the communicability of interactive systems for visually impaired users. In Proceedings of the 20th Brazilian Symposium on Human Factors in Computing Systems (IHC '21), pages 1–11, Virtual Event, Brazil. ACM. DOI: https://doi.org/10.1145/3472301.3484365.

Carvalho, L. P., Souza, J. G. R. d., Prates, R. O., and Freire, A. P. (2023). SIM-SR—semiotic inspection method mediated by screen reader: Analysing the benefits of bringing communicability to the accessibility evaluation table. Interacting with Computers, 35(2):276–300. DOI: https://doi.org/10.1093/iwc/iwad023.

Costa, R. B. d. C., Fortunato, C. B., and Prates, R. O. (2025). The SIM studies platform. In Human-Computer Interaction – INTERACT 2025 (Demo Track), volume 16111 of Lecture Notes in Computer Science, pages 142–146, Cham. Springer. DOI: https://doi.org/10.1007/978-3-032-05008-3_28.

da Silva Junior, D. P., Alves, D. D., Carneiro, N., Matos, E. d. S., Baranauskas, M. C. C., and Mendoza, Y. L. M. (2024). GranDIHC-BR 2025–2035 – GC1: New theoretical and methodological approaches in HCI. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems (IHC '24), pages 1–31, Brasília, Brazil. ACM. DOI: https://doi.org/10.1145/3702038.3702054.

de Barros, G. F., da Silva Reis, D. B., Honorato, R. S., Alves, A. J. C., Dutra, T. C., Júnior, C. X. P., and Aranha, R. V. (2024). Interação também é inclusão na educação: avaliando a comunicabilidade mediada por leitor de tela em sistemas acadêmicos de instituições federais. In Anais do XXXV Simpósio Brasileiro de Informática na Educação (SBIE 2024), pages 1167–1177, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/sbie.2024.242163.

de Souza, C. S. (2005). The Semiotic Engineering of Human-Computer Interaction. MIT Press, Cambridge, MA.

de Souza, C. S. and Leitão, C. F. (2009). Semiotic Engineering Methods for Scientific Research in HCI. Synthesis Lectures on Human-Centered Informatics. Morgan & Claypool Publishers.

de Souza, C. S., Leitão, C. F., Prates, R. O., Bim, S. A., and da Silva, E. J. (2010). Can inspection methods generate valid new knowledge in HCI? the case of semiotic inspection. International Journal of Human-Computer Studies, 68(1-2):22–40. DOI: https://doi.org/10.1016/j.ijhcs.2009.08.006.

de Souza, C. S., Leitão, C. F., Prates, R. O., and da Silva, E. J. (2006). The semiotic inspection method. In Proceedings of VII Brazilian Symposium on Human Factors in Computing Systems (IHC '06), pages 148–157, New York, NY, USA. ACM. DOI: https://doi.org/10.1145/1298023.1298044.

Felizardo, K. R., Mendes, E., Kalinowski, M., Souza, É. F., and Vijaykumar, N. L. (2016). Using forward snowballing to update systematic reviews in software engineering. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '16), pages 1–6, Ciudad Real, Spain. ACM. DOI: https://doi.org/10.1145/2961111.2962630.

Fortunato, C., Costa, R., and Prates, R. (2025a). Communicability inspection: Which version of the semiotic inspection method should i use? In Anais do XXIV Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC 2025), pages 1025–1048, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/ihc.2025.10771.

Fortunato, C. B., Costa, R. B. C., and Prates, R. O. (2025b). Adaptations of the semiotic inspection method (SIM) across different contexts. In Human-Computer Interaction – INTERACT 2025, volume 16111 of Lecture Notes in Computer Science, pages 166–171, Cham. Springer. DOI: https://doi.org/10.1007/978-3-032-05008-3_32.

Gibbs, G. R. (2008). Analyzing Qualitative Data. SAGE Publications, London.

Gomes, L., Silveira, J. C. S., Martins, H., Lana, C. A., and Villela, M. L. B. (2025). Communicating ethical considerations in generative AI systems. In Anais do XXIV Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC 2025), pages 718–742, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/ihc.2025.10939.

Guerino, G., Rodrigues, L., Capeleti, B., Mello, R. F., Freire, A., and Zaina, L. (2025). Can GPT-4o evaluate usability like human experts? a comparative study on issue identification in heuristic evaluation. In Human-Computer Interaction – INTERACT 2025, volume 16110 of Lecture Notes in Computer Science, pages 381–402, Cham. Springer. DOI: https://doi.org/10.1007/978-3-032-05005-2_20.

Kitchenham, B. (2004). Procedures for performing systematic reviews. Technical Report TR/SE-0401, Keele University, Keele, UK.

Leitão, C. F. and Prates, R. O. (2017). A aplicação de métodos qualitativos em computação. In Delicato, F. C., Pires, P. F., and Silveira, I. F., editors, Jornadas de Atualização em Informática 2017 (JAI), pages 43–90. SBC, Porto Alegre.

Lopes, A. D., Maciel, C., and Pereira, V. C. (2015). Applicability of the semiotic inspection method in interactive electronic games. In Proceedings of the 14th Brazilian Symposium on Human Factors in Computing Systems (IHC '15), pages 1–10, Salvador, Brazil. ACM. DOI: https://doi.org/10.1145/3148456.3148487.

Lopes, A. D., Pereira, V. C., and Maciel, C. (2017). An analysis of deictic signs in computer interfaces: Contributions to the semiotic inspection method. Journal of Visual Languages & Computing, 40:51–64. DOI: https://doi.org/10.1016/j.jvlc.2017.01.001.

Maués, R. d. A. and Barbosa, S. D. J. (2013). Cross-communicability: Evaluating the meta-communication of cross-platform applications. In Human-Computer Interaction – INTERACT 2013: 14th IFIP TC 13 International Conference, Proceedings, Part III, volume 8119 of Lecture Notes in Computer Science, pages 241–258, Berlin, Heidelberg. Springer. DOI: https://doi.org/10.1007/978-3-642-40477-1_15.

Maués, R. d. A. and Barbosa, S. D. J. (2014). Reflections on the cross-platform semiotic inspection method. In Human-Computer Interaction. Theories, Methods, and Tools: 16th International Conference, HCI International 2014, Proceedings, Part I, volume 8510 of Lecture Notes in Computer Science, pages 533–544, Cham. Springer. DOI: https://doi.org/10.1007/978-3-319-07233-3_49.

McDonald, N., Schoenebeck, S., and Forte, A. (2019). Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–23. DOI: https://doi.org/10.1145/3359174.

Meira, G., da Cruz, B. B., Paulin, T., Valle, P., and Paschoal, L. (2026). A tool supported by LLM for heuristic usability evaluation. In Proceedings of the 28th International Conference on Enterprise Information Systems (ICEIS 2026) – Volume 3, pages 1916–1927. INSTICC, SciTePress. DOI: https://doi.org/10.5220/0014829900004018.

Melo, G., dos Santos, S., Teran, L., Monteiro, I., Rodrigues, K., and Mota, M. (2026). Investigating accessibility barriers in software development tools through the blind people's perspective. In Anais do XXI Simpósio Brasileiro de Sistemas Colaborativos (SBSC 2026), pages 171–185, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/sbsc.2026.20813.

Melo, G. L. N., Menezes, N. d. S., Maciel, A. C. d. S., Teran, L. A., Rocha, T. Á. d., Souza, C. R. B. d., and Mota, M. P. (2024). Accessibility issues in establishing awareness on remote collaborative software development. Journal on Interactive Systems, 15(1):294–310. DOI: https://doi.org/10.5753/jis.2024.3802.

Menezes, N. d. S., Rocha, T. Á. d., and Mota, M. P. (2023). Construção de sites acessíveis para pessoas cegas ou com baixa visão utilizando sistemas de gerenciamento de conteúdo: um estudo de caso com WordPress e Joomla. In Anais do Computer on the Beach, volume 14, pages 339–346, Florianópolis, SC, Brasil. DOI: https://doi.org/10.14210/cotb.v14.p339-346.

Mueller, F. F., Montoya, M. F., Pell, S. J., Oppermann, L., Blythe, M., Dietz, P. H., Marshall, J., Bateman, S., Smith, I., Ananthanarayan, S., Mazalek, A., Verni, A., Bakogeorge, A., Simonnet, M., Ellis, K., Semertzidis, N. A., Burleson, W., Quarles, J., Mann, S., Hill, C., Clashing, C., and Elvitigala, D. S. (2024). Grand challenges in WaterHCI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI '24), pages 1–18, Honolulu, HI, USA. ACM. DOI: https://doi.org/10.1145/3613904.3642052.

Oliveira, E. R., Luz, L. C. S., and Prates, R. O. (2008). Aplicação semi-estruturada do método de inspeção semiótica: estudo de caso para o domínio educacional. In Proceedings of the VIII Brazilian Symposium on Human Factors in Computing Systems (IHC '08), pages 50–59, Porto Alegre, RS, Brazil. SBC. DOI: https://dl.acm.org/doi/10.5555/1497470.1497477.

Oliveira, E. R. and Prates, R. O. (2018). Intermediated semiotic inspection method. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems (IHC 2018), New York, NY, USA. ACM. DOI: https://doi.org/10.1145/3274192.3274221.

Pang, R. Y., Schroeder, H., Smith, K. S., Barocas, S., Xiao, Z., Tseng, E., and Bragg, D. (2025). Understanding the LLM-ification of CHI: Unpacking the impact of LLMs at CHI through a systematic literature review. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25), pages 1–20, Yokohama, Japan. ACM. DOI: https://doi.org/10.1145/3706598.3713726.

Peirce, C. S. (1934). Collected Papers of Charles Sanders Peirce, Volume V: Pragmatism and Pragmaticism, volume 5. Harvard University Press, Cambridge, MA.

Pereira, R., Darin, T., and Silveira, M. S. (2024). GranDIHC-BR: Grand research challenges in human-computer interaction in Brazil for 2025–2035. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems (IHC '24), pages 1–24, Brasília, Brazil. ACM. DOI: https://doi.org/10.1145/3702038.3702061.

Prates, R. O., de Souza, C. S., and Barbosa, S. D. J. (2000). Methods and tools: A method for evaluating the communicability of user interfaces. Interactions, 7(1):31–38. DOI: https://doi.org/10.1145/328595.328608.

Reis, S. d. S. and Prates, R. O. (2011). Applicability of the semiotic inspection method: a systematic literature review. In Proceedings of the 10th Brazilian Symposium on Human Factors in Computing Systems and the 5th Latin American Conference on Human-Computer Interaction (IHC+CLIHC '11), pages 177–186, Porto Alegre, Brazil. Brazilian Computer Society. DOI: https://dl.acm.org/doi/10.5555/2254436.2254468.

Reis, S. d. S. and Prates, R. O. (2013). Assessing the semiotic inspection method – the evaluators' perspective. Revista de Informática Teórica e Aplicada (RITA), 20(3):155–182. DOI: https://doi.org/10.22456/2175-2745.40202.

Rogers, Y. (2012). HCI Theory: Classical, Modern, and Contemporary. Number 14 in Synthesis Lectures on Human-Centered Informatics. Morgan & Claypool Publishers.

Souza, J. G. R. d. and Prates, R. O. (2021). Games by end-users: Analyzing development environments. In Proceedings of the 20th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames 2021), pages 69–78. IEEE. DOI: https://doi.org/10.1109/SBGames54170.2021.00018.

Stephanidis, C., Salvendy, G., Antona, M., Chen, J. Y. C., Dong, J., Duffy, V. G., Fang, X., Fidopiastis, C., Fragomeni, G., Fu, L. P., et al. (2019). Seven HCI grand challenges. International Journal of Human–Computer Interaction, 35(14):1229–1269. DOI: https://doi.org/10.1080/10447318.2019.1619259.

Stephanidis, C., Salvendy, G., Antona, M., Duffy, V. G., Gao, Q., Karwowski, W., Konomi, S., Nah, F., Ntoa, S., Rau, P.-L. P., Siau, K., and Zhou, J. (2025). Seven HCI grand challenges revisited: Five-year progress. International Journal of Human–Computer Interaction, 41(19):11947–11995. DOI: https://doi.org/10.1080/10447318.2025.2450411.

Weiss, M. C. (2019). Sociedade sensoriada: a sociedade da transformação digital. Estudos Avançados, 33(95):203–214. DOI: https://doi.org/10.1590/s0103-4014.2019.3395.0013.

Zaina, L., Prates, R. O., Delabrida, S. E., Choma, J., Valentim, N. M. C., Frigo, L. B., and Bicho, A. d. L. (2024). GranDIHC-BR 2025–2035 – GC7: Interaction with emerging technologies: An ecosystem integrating humans, technologies and contexts. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems (IHC '24), pages 1–21, Brasília, Brazil. ACM. DOI: https://doi.org/10.1145/3702038.3702060.