A Guide-based Approach for IoT Interoperability Testing
DOI:
https://doi.org/10.5753/jbcs.2026.5473Keywords:
Internet of Things, Guidelines, Interoperability TestingAbstract
Technology has significantly transformed human interactions with everyday objects, expanding their communication capabilities. This broad connectivity to the Internet has given rise to the "Internet of Things (IoT)", extending the boundaries of the Internet and evolving those objects into smart objects. When interconnected, these smart objects can collect and share information to provide services effectively. However, the intense data traffic and the diversity of interaction methods among these objects bring challenges related to interoperability. Interoperability refers to the ability of distinct systems to communicate effectively, ensuring data integrity. In this context, interoperability testing assesses the capability of systems and devices to cooperate efficiently. Regarding the testing challenges faced in IoT, we can highlight (i) the complexity of the IoT architecture, which involves multiple layers and system components; (ii) the diversity of IoT protocols; and (iii) the heterogeneity of devices, which poses challenges in testing their compatibility and managing their integration to ensure effective connectivity. To address these challenges, this paper proposes a guide-based approach designed to systematize IoT interoperability testing. The guide was developed following a systematic methodology, including literature reviews, data extraction and analysis, and observations in real IoT environments. The guide covers 12 topics, such as the definition and correlation of IoT characteristics, subcharacteristics, abstract test cases, measures, cost-benefit analysis, and tool suggestions. The guide evaluation was conducted in three stages: (1) a structural evaluation using the Technology Acceptance Model (TAM); (2) a controlled experiment applying the guide to test the interoperability of a real IoT application; and (3) an evaluation of the guide with domain experts. The results of the three evaluations were positive, indicating that the guide is effective in test planning, execution, and identifying IoT-specific failures.
Downloads
References
Almeida, R. L. A. (2018). CHASE: checklist para avaliação da experiência do usuário em ambientes de internet das coisas. Dissertação (mestrado em ciência da computação), Universidade Federal do Ceará, Fortaleza. Avaialable at:[link].
Andrade, S. and Luque, D. (2022). Interoperabilidade de sistemas aplicados às cidades inteligentes: Um estudo de mapeamento sistemático. In Anais do X Workshop de Computação Aplicada em Governo Eletrônico, pages 97-108. SBC. DOI: 10.5753/wcge.2022.222970.
Atzori, L., Iera, A., and Morabito, G. (2010). The internet of things: A survey. Computer networks, 54(15):2787-2805. DOI: 10.1016/j.comnet.2010.05.010.
Branco, K. C., Dantas, V., and Carvalho, L. (2024). Interoperability testing guide for the internet of things. In Proceedings of the 30th Brazilian Symposium on Multimedia and the Web, pages 188-196, Porto Alegre, RS, Brasil. SBC. DOI: 10.5753/webmedia.2024.242058.
Caldas, E. A. L. (2023). Checklist para avaliação da interoperabilidade em dispositivos iot com foco em casas inteligentes. Trabalho de conclusão de curso, Universidade Federal do Ceará, Campus de Quixadá, Quixadá. Available at:[link].
Cañas Betancur, D. C. and Hernández Sánchez, J. (2019). Comunicação assertiva em professores: diagnóstico e proposta educativa. Praxis & Saber, 10(24):143-165. DOI: 10.19053/22160159.v10.n25.2019.8936.
Carvalho, L. (2022). Wiki of the Performance Testing Guide for IoT Applications. Available at:[link]. Accessed on: April 29, 2025.
Carvalho, L. M., Lelli, V., and Andrade, R. M. (2022). Performance testing guide for iot applications. In ICEIS (1), pages 667-678. INSTICC, SciTePress. DOI: 10.5220/0011090800003179.
Coutinho, A., Carneiro, E. O., and Greve, F. G. P. (2016). Computaçao em névoa: Conceitos, aplicações e desafios. Minicursos do XXXIV SBRC, pages 266-315. Available at:[link].
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3):319-340. DOI: 10.2307/249008.
dos Santos, M. C. (2016). Internet das coisas e sistemas inteligentes no jornalismo: o conceito de presença diluído entre as narrativas da complexidade urbana. Comunicação & Inovação, 17(34):21-39. DOI: 10.13037/ci.vol17n34.3769.
Ferreira, H. G. C. (2014). Arquitetura de middleware para Internet das Coisas. PhD thesis, University of Brasília Brasília, Brazil. Available at:[link].
Fienberg, S. E. (2006). When did bayesian inference become" bayesian"?. DOI: 10.1214/06-ba101.
Filho, F. G. S., Lelli, V., Santos, I. d. S., and Andrade, R. M. C. (2021). Correlations among software testability metrics. In Proceedings of the XIX Brazilian Symposium on Software Quality, SBQS '20, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3439961.3439972.
Ghasemi, A. and Zahediasl, S. (2012). Normality tests for statistical analysis: a guide for non-statisticians. International Journal of Endocrinology and Metabolism, 10(2):486-489. DOI: 10.5812/ijem.3505.
Giusto, D., Iera, A., Morabito, G., and Atzori, L. (2010). The internet of things: 20th Tyrrhenian workshop on digital communications. Springer Science & Business Media. DOI: 10.1007/978-1-4419-1674-7.
Gulla, J. A., Tomassen, S. L., and Strasunskas, D. (2006). Semantic interoperability in the norwegian petroleum industry. In ISTA, pages 81-93. Available at:[link].
Gunathilaka, P., Mashima, D., and Chen, B. (2016). Softgrid: A software-based smart grid testbed for evaluating substation cybersecurity solutions. In Proceedings of the 2nd ACM Workshop on Cyber-Physical Systems Security and Privacy, pages 113-124. DOI: 10.1145/2994487.2994494.
ISO 15926 (2011). Iso 15926 - industrial automation systems and integration - integration of life-cycle data for process plants including oil and gas production facilities. Available at:[link]. Accessed on 28 Mar. 2023.
ISO/IEC 21823 (2022). Internet of Things (IoT) - Interoperability for IoT systems. (ISO/IEC 21823:2022). DOI: 10.3403/bsisoiec21823.
ISO/IEC 25000 (2005). ISO/IEC 25000:2005, software engineering - software product quality requirements and evaluation (SQuaRE). Available at:[link].
ISO/IEC 25010 (2011). Systems and software engineering — systems and software quality requirements and evaluation (square) — system and software quality models. DOI: 10.3403/30215101.
ISO/IEC 25019 (2023). Systems and software engineering — systems and software quality requirements and evaluation (square) — quality-in-use model. DOI: 10.3403/30421230u.
ISO/IEC 30141 (2018). Information technology - Internet of Things (IoT) - Reference architecture. Available at:[link].
Kim, T. K. (2015). T test as a parametric statistic. Korean Journal of Anesthesiology, 68(6):540-546. DOI: 10.4097/kjae.2015.68.6.540.
Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M., Bailey, J., and Linkman, S. (2009). Systematic literature reviews in software engineering-a systematic literature review. Information and software technology, 51(1):7-15. DOI: 10.1016/j.infsof.2008.09.009.
Kuzniar, M., Peresini, P., Canini, M., Venzano, D., and Kostic, D. (2012). A soft way for openflow switch interoperability testing. In Proceedings of the 8th international conference on Emerging networking experiments and technologies, pages 265-276. DOI: 10.1145/2413176.2413207.
Legner, C. and Wende, K. (2006). Towards an excellence framework for business interoperability. In 19th Bled eConference eValues, Bled, Slovenia. Available at:[link].
Levene, H. (1960). Robust tests for equality of variances. pages 278-292. DOI: 10.2307/2285659.
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140):1-55. Available at:[link].
Maia, N., Macedo, G., Collins, E., and Dias-Neto, A. (2012). Aplicando testes ágeis com equipes distribuídas: Um relato de experiência. In Anais do XI Simpósio Brasileiro de Qualidade de Software, pages 365-372. SBC. DOI: 10.5753/sbqs.2012.15330.
Mann, H. B. and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1):50-60. DOI: 10.1214/aoms/1177730491.
Mattern, F. and Floerkemeier, C. (2010). From the internet of computers to the internet of things. In From active data management to event-based systems and more, pages 242-259. Springer. DOI: 10.1007/978-3-642-17226-7_15.
Mattiello-Francisco, M. d. F. (2009). Inrob-uma abordagem para testes de interoperabilidade e de robustez de subsistemas de temporeal intensivos em software. Doutorado em engenharia eletrônica e computação, Instituto de Tecnológico de Aeronáutica, São José dos Campos, SP. Available at:[link].
Motta, R. C., De Oliveira, K. M., and Travassos, G. H. (2017). Rethinking interoperability in contemporary software systems. In 2017 IEEE/ACM Joint 5th International Workshop on Software Engineering for Systems-of-Systems and 11th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems (JSOS), pages 9-15. IEEE. DOI: 10.1109/jsos.2017.5.
Mujjiga, S. and Sukumaran, S. (2007). Modelling and test generation using sal for interoperability testing in consumer electronics. In Proceedings of the second workshop on Automated formal methods, pages 32-40. DOI: 10.1145/1345169.1345173.
Myers, G. J., Sandler, C., and Badgett, T. (2011). The art of software testing. John Wiley & Sons. DOI: 10.1002/9781119202486.
Noura, M., Atiquzzaman, M., and Gaedke, M. (2019). Interoperability in internet of things: Taxonomies and open challenges. Mobile networks and applications, 24:796-809. DOI: 10.1007/s11036-018-1089-9.
Nunes, P. R. d. A. F. (2011). Validação de padrões de web services transacionais. PhD thesis, Universidade de São Paulo. Avaialable at:[link].
Perera, C., Zaslavsky, A., Christen, P., and Georgakopoulos, D. (2014). Context aware computing for the internet of things: A survey. IEEE Communications Surveys & Tutorials, 16(1):414-454. DOI: 10.1109/surv.2013.042313.00197.
Rath, F., Schemmel, D., and Wehrle, K. (2018). Interoperability-guided testing of quic implementations using symbolic execution. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC, pages 15-21. DOI: 10.1145/3284850.3284853.
Rayes, A. and Salam, S. (2019). Internet of things from hype to reality: The road to digitization. Springer. Book.
Rowland, C., Goodman, E., Charlier, M., Light, A., and Lui, A. (2015). Designing Connected Products: UX for the Consumer Internet of Things. O'Reilly Media, Inc., first edition. Book.
Shapiro, S. S. and Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3-4):591-611. DOI: 10.1093/biomet/52.3-4.591.
Sicari, S., Rizzardi, A., Grieco, L. A., and Coen-Porisini, A. (2015). Security, privacy and trust in internet of things: The road ahead. Computer networks, 76:146-164. DOI: 10.1016/j.comnet.2014.11.008.
Silva, V. M. d. (2019). Scenariot: support for scenario specification of internet of things-based software systems. DOI: 10.5753/cbsoft_estendido.2020.14628.
Soares, D. (2009). Interoperabilidade entre sistemas de informação na administração pública. sl] Universidade do Minho. Available at:[link].
Venkatesh, V. and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science, 46(2):186-204. DOI: 10.1287/mnsc.46.2.186.11926.
Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly, pages 425-478. Available at:[link].
Vermesan, O. and Friess, P. (2011). Internet of Things: Global technological and societal trends. River Publishers. Book.
Weiss, L. C. et al. (2019). Interoperabilidade semântica: uma análise sob a perspectiva da abordagem ontológica de willard van orman quine. Available at:[link].
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in Software Engineering. Springer Science & Business Media. DOI: 10.1007/978-1-4615-4625-2.
Zaidi, F., Bayse, E., and Cavalli, A. (2009). Network protocol interoperability testing based on contextual signatures and passive testing. In Proceedings of the 2009 ACM symposium on Applied Computing, pages 2-7. DOI: 10.1145/1529282.1529284.
Zanella, A., Bui, N., Castellani, A., Vangelista, L., and Zorzi, M. (2014). Internet of things for smart cities. IEEE Internet of Things journal, 1(1):22-32. DOI: 10.1109/jiot.2014.2306328.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Karina da Silva Castelo Branco, Valéria Lelli Leitão Dantas, Lavínia Matoso Freitas, Liana Mara Carvalho, Paulo Ricardo Fernandes Rodrigues
This work is licensed under a Creative Commons Attribution 4.0 International License.
