Software testing for peer-to-peer systems: Challenges and the state-of-the-art
DOI:
https://doi.org/10.5753/jserd.2025.4917Keywords:
peer-to-peer, distributed systems, software testing, systematic mappingAbstract
Peer-to-peer (P2P) systems, characterized by their decentralized architecture, have become crucial to various applications such as file sharing, content distribution, and blockchain technologies. Due to their dynamic and heterogeneous nature, ensuring their reliability and robustness presents challenges. Such an issue has motivated researchers to develop testing techniques for these systems. Despite advances in research about software testing for distributed systems, there is a lack of data about their applicability and which features from P2P systems are highlighted by the proposed testing approaches. This paper provides a comprehensive overview of the state-of-the-art software testing for P2P systems. We systematically examine the inherent challenges and situations. Our findings show that some aspects of P2P systems can make the software testing activity difficult, such as validation of global properties and consensus, peer churn, a large volume of data, malformed data, reproducibility, and non-determinism. Several testing techniques have been used in this context, such as model-based testing, fault injection, fuzzing, stress testing, differential testing, conformance testing, and concurrency testing. We highlight fuzzing for testing malformed data, global properties, parallel events, and model-based testing as a powerful technique to abstract the complexities and varieties of P2P applications during the testing activities. Moreover, blockchain applications are the most addressed P2P application explored by the studies, where most papers were published in 2023, highlighting the theme’s relevance.
Downloads
References
Adamsky, F., Kaiser, D., Steglich, M., and Engel, T. (2021). Locust: Highly concurrent DHT experimentation framework for security evaluations. In Proceedings of the 2020 10th International Conference on Communication and Network Security, ICCNS ’20, page 115–122, New York, NY, USA. Association for Computing Machinery.
Ahmad, A. A.-S., Brereton, P., and Andras, P. (2017). A systematic mapping study of empirical studies on software cloud testing methods. In 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), pages 555–562.
Alshahwan, N., Chheda, J., Finogenova, A., Gokkaya, B., Harman, M., Harper, I., Marginean, A., Sengupta, S., and Wang, E. (2024). Automated unit test improvement using large language models at Meta. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering, FSE 2024, page 185–196, New York, NY, USA. Association for Computing Machinery.
Amaral, M., Pardal, M. L., Mercier, H., and Matos, M. (2020). Faultsee: Reproducible fault injection in distributed systems. In 2020 16th European Dependable Computing Conference (EDCC), pages 25–32.
Arsat, N., Bakar, N. S. A. A., and Yahya, N. (2022). Testing in blockchain-based systems: A systematic review. In 2022 10th International Conference on Cyber and IT Service Management (CITSM), pages 1–6.
Aziz, A., Riaz, M. T., Jahan, M. S., and Ayub, K. (2020). Meta-model for stress testing on blockchain nodes. In 2020 3rd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), pages 1–4.
Boumaiza, A. and Sanfilippo, A. (2024a). A testing framework for blockchain-based energy trade microgrids applications. IEEE Access, 12:27465–27483.
Boumaiza, A. and Sanfilippo, A. (2024b). A testing framework for blockchain-based energy trade microgrids applications. IEEE Access, 12:27465–27483.
Chen, Y. (2024). Chronos: Finding timeout bugs in practical distributed systems by deep-priority fuzzing with transient delay. In 2024 IEEE Symposium on Security and Privacy (SP), pages 108–108, Los Alamitos, CA, USA. IEEE Computer Society.
Chen, Y., Ma, F., Zhou, Y., Jiang, Y., Chen, T., and Sun, J. (2023). Tyr: Finding consensus failure bugs in blockchain system with behaviour divergent model. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2517–2532.
Chepurnoy, A. and Rathee, M. (2018). Checking laws of the blockchain with property-based testing. In 2018 International Workshop on Blockchain Oriented Software Engineering (IWBOSE), pages 40–47.
Cvitic, P. H., Dobslaw, F., and de Oliveira Neto, F. G. (2023). Investigating software testing and maintenance of open-source distributed ledger. In 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 886–896.
Dalal, S. R., Jain, A., Karunanithi, N., Leaton, J. M., Lott, C. M., Patton, G. C., and Horowitz, B. M. (1999). Model-based testing in practice. In Proceedings of the 21st International Conference on Software Engineering, ICSE ’99, page 285–294, New York, NY, USA. Association for Computing Machinery.
Dinesh, B., Kavya, B., Sivakumar, D., and Ahmed, M. R. (2019). Conforming test of blockchain for 5G-enabled IoT. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), pages 1153–1157.
Eisenbarth, J.-P., Cholez, T., and Perrin, O. (2021a). A comprehensive study of the Bitcoin P2P network. In 2021 3rd Conference on Blockchain Research Applications for Innovative Networks and Services (BRAINS), pages 105–112.
Eisenbarth, J.-P., Cholez, T., and Perrin, O. (2021b). An open measurement dataset on the Bitcoin P2P network. In 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM), pages 643–647.
Etemadi, K., Mohammadi, B., Su, Z., and Monperrus, M. (2024). Mokav: Execution-driven differential testing with LLMs.
Fox, G. (2001). Peer-to-peer networks. Computing in Science Engineering, 3(3):75–77.
Garcia, B. E. R., Delamaro, M. E., and Souza, S. d. R. S. d. (2024). Towards differential fuzzing to reduce manual efforts to identify equivalent mutants: A preliminary study. Anais.
Groce, A., Jain, K., van Tonder, R., Kalburgi, G. T., and Goues, C. L. (2022). Looking for lacunae in Bitcoin Core’s fuzzing efforts. In Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP ’22, page 185–186, New York, NY, USA. Association for Computing Machinery.
Gulzar, M. A., Zhu, Y., and Han, X. (2019). Perception and practices of differential testing. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 71–80.
Hassanzadeh-Nazarabadi, Y., Rybalov, M., and Claybon, K. (2023). BFT testing framework for Flow blockchain. In Machado, J. M., Prieto, J., Vieira, P., Peixoto, H., Abelha, A., Arroyo, D., and Vigneri, L., editors, Blockchain and Applications, 5th International Congress, pages 338–347, Cham. Springer Nature Switzerland.
Hsueh, M.-C., Tsai, T., and Iyer, R. (1997). Fault injection techniques and tools. Computer, 30(4):75–82.
Imtiaz, M. A., Starobinski, D., Trachtenberg, A., and Younis, N. (2019). Churn in the Bitcoin network: Characterization and impact. In 2019 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), pages 431–439.
Jabbar, R., Krichen, M., Shinoy, M., Kharbeche, M., Fetais, N., and Barkaoui, K. (2020). A model-based and resource-aware testing framework for parking system payment using blockchain. In 2020 International Wireless Communications and Mobile Computing (IWCMC), pages 1252–1259.
Jain, K., Kalburgi, G., Goues, C. L., and Groce, A. (2023). Mind the gap: The difference between coverage and mutation score can guide testing efforts. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE), pages 102–113, Los Alamitos, CA, USA. IEEE Computer Society.
Kim, S. and Hwang, S. (2023). Etherdiffer: Differential testing on rpc services of ethereum nodes. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2023, page 1333–1344, New York, NY, USA. Association for Computing Machinery.
Li, R., Zhu, L., Li, C., Wu, F., and Xu, D. (2023). Bns: A detection system to find nodes in the bitcoin network. Mathematics, 11(24).
Li, S. and Su, Z. (2023). Finding unstable code via compiler-driven differential testing. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, ASPLOS 2023, page 238–251, New York, NY, USA. Association for Computing Machinery.
Li, W., Feng, C., Zhang, L., Xu, H., Cao, B., and Imran, M. A. (2021). A scalable multi-layer pbft consensus for blockchain. IEEE Transactions on Parallel and Distributed Systems, 32(5):1146–1160.
Long, N. and Thomas, R. (2001). Trends in denial of service attack technology. CERT Coordination Center, 648(651):569.
Ma, F., Chen, Y., Ren, M., Zhou, Y., Jiang, Y., Chen, T., Li, H., and Sun, J. (2023a). Loki: State-aware fuzzing framework for the implementation of blockchain consensus protocols. Proceedings 2023 Network and Distributed System Security Symposium.
Ma, F., Chen, Y., Zhou, Y., Sun, J., Su, Z., Jiang, Y., Sun, J., and Li, H. (2023b). Phoenix: Detect and locate resilience issues in blockchain via context-sensitive chaos. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS ’23, page 1182–1196, New York, NY, USA. Association for Computing Machinery.
Machado, N., Maia, F., Neves, F., Coelho, F., and Pereira, J. (2020). Minha: Large-Scale Distributed Systems Testing Made Practical. In Felber, P., Friedman, R., Gilbert, S., and Miller, A., editors, 23rd International Conference on Principles of Distributed Systems (OPODIS 2019), volume 153 of Leibniz International Proceedings in Informatics (LIPIcs), pages 11:1–11:17, Dagstuhl, Germany. Schloss Dagstuhl – Leibniz-Zentrum für Informatik.
Meftah, L., Gomez, M., Rouvoy, R., and Chrisment, I. (2017). Androfleet: Testing wifi peer-to-peer mobile apps in the large. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 961–966.
Milka, G. and Rzadca, K. (2018). Dfuntest: A testing framework for distributed applications. In Wyrzykowski, R., Dongarra, J., Deelman, E., and Karczewski, K., editors, Parallel Processing and Applied Mathematics, pages 395–405, Cham. Springer International Publishing.
Miller, B. P., Fredriksen, L., and So, B. (1990). An empirical study of the reliability of unix utilities. Communications of the ACM, 33(12):32–44.
Motlagh, S. G., Mišić, J., and Mišić, V. B. (2020). Impact of node churn in the bitcoin network with compact blocks. In GLOBECOM 2020 - 2020 IEEE Global Communications Conference, pages 1–6.
Pankov, K. N. (2020). Testing, verification and validation of distributed ledger systems. In 2020 Systems of Signals Generating and Processing in the Field of on Board Communications, pages 1–9.
Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008). Systematic mapping studies in software engineering. Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, 17.
Plein, L., Ouédraogo, W. C., Klein, J., and Bissyandé, T. F. (2024). Automatic generation of test cases based on bug reports: a feasibility study with large language models. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, ICSE-Companion ’24, page 360–361, New York, NY, USA. Association for Computing Machinery.
Reddivari, S., Orr, J., and Reddy, R. (2023). Blockchain-oriented software testing: A preliminary literature review. In 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), pages 974–975.
Rozsíval, M. and Smrčka, A. (2023). Netloiter: A tool for automated testing of network applications using fault-injection. In 2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSNW), pages 207–210.
Saad, M., Chen, S., and Mohaisen, D. (2021). Root cause analyses for the deteriorating bitcoin network synchronization. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS), pages 239–249.
Sakib, N., Wuthier, S., Zhang, K., Zhou, X., and Chang, S.-Y. (2024). From slow propagation to partition: Analyzing bitcoin over anonymous routing. In 2024 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), pages 377–385.
Schollmeier, R. (2001). A definition of peer-to-peer networking for the classification of peer-to-peer architectures and applications. In Proceedings First International Conference on Peer-to-Peer Computing, pages 101–102.
Schäfer, M., Nadi, S., Eghbali, A., and Tip, F. (2024). An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50(1):85–105.
Shala, B., Wacht, P., Trick, U., Lehmann, A., Shala, B., Ghita, B., and Shiaeles, S. (2017). Framework for automated functional testing of p2p-based m2m applications. In 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN), pages 916–921.
Sit, E. and Morris, R. (2002). Security considerations for peer-to-peer distributed hash tables. In International Workshop on Peer-to-Peer Systems, pages 261–269. Springer.
Sommerville, I. (2011). Software Engineering, 9/E. Pearson Education India.
Steinmetz, R. and Wehrle, K. (2005a). Peer-to-peer systems and applications, volume 3485. Springer.
Steinmetz, R. and Wehrle, K. (2005b). Peer-to-Peer Systems and Applications. Information Systems and Applications, incl. Internet/Web, and HCI. Springer.
Sunyé, G., de Almeida, E. C., Le Traon, Y., Baudry, B., and Jézéquel, J.-M. (2014). Model-based testing of global properties on large-scale distributed systems. Information and Software Technology, 56(7):749–762.
van Meerten, M., Ozkan, B. K., and Panichella, A. (2023). Evolutionary approach for concurrency testing of ripple blockchain consensus algorithm. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 36–47.
Veldkamp, L., Olsthoorn, M., and Panichella, A. (2023). Grammar-based evolutionary fuzzing for json-rpc apis. In 2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT), pages 33–36.
Wang, J., Zhang, B., Wang, K., Wang, Y., and Han, W. (2024). Bftdiagnosis: An automated security testing framework with malicious behavior injection for bft protocols. Computer Networks, 249:110404.
Winter, L. N., Buse, F., de Graaf, D., von Gleissenthall, K., and Kulahcioglu Ozkan, B. (2023). Randomized testing of byzantine fault tolerant algorithms. Proc. ACM Program. Lang., 7(OOPSLA1).
Yang, Y., Kim, T., and Chun, B.-G. (2021). Finding consensus bugs in ethereum via multi-transaction differential fuzzing. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21), pages 349–365. USENIX Association.
Yutia, S. N. and Fathiyana, R. Z. (2022). Performance analysis of blockchain in tendering process based on hyperledger framework. In 2022 10th International Conference on Information and Communication Technology (ICoICT), pages 190–193.
Zhong, W., Yang, C., Liang, W., Cai, J., Chen, L., Liao, J., and Xiong, N. (2023). Byzantine fault-tolerant consensus algorithms: A survey. Electronics, 12(18).
Zhou, X., Jin, Y., Zhang, H., Li, S., and Huang, X. (2016). A map of threats to validity of systematic literature reviews in software engineering. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC), pages 153–160.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Bruno Garcia, Simone do Rocio Senger de Souza
This work is licensed under a Creative Commons Attribution 4.0 International License.
