Cybersecurity Testbeds for IoT: A Systematic Literature Review and Taxonomy
DOI:
https://doi.org/10.5753/jisa.2024.4363Keywords:
Testbed, IoT, Cybersecurity, Systematic Literature Review, TaxonomyAbstract
Researchers across the globe are carrying out numerous experiments related to cybersecurity, such as botnet dispersion, intrusion detection systems powered by machine learning, and others, to explore these topics in many different contexts and environmental settings. One current research topic is the behavior of Internet of Things (IoT) devices, as they increasingly become a common feature of homes, offices, and companies.. Network testing environments which are designated as testbeds, are boosting the effectiveness of network research. However, exploratory studies in IoT cybersecurity may include a wide range of requirements. This article seeks to carry out a survey of IoT cybersecurity testbeds. A critical systematic literature review was conducted to select relevant articles, by applying a novel taxonomy to classify the testbeds. The surveyed testbeds are classified in terms of their primary target domain and other features such as fidelity, heterogeneity, scalability, security, reproducibility, flexibility, and measurability. Furthermore, we have compared the testbeds with regard to each feature. Thus, the main contribution made by this study lies in a) the insights it provides into the state-of-the-art in IoT cybersecurity testbeds, and b) the emphasis laid on the main benefits and limitations that were found in the surveyed testbeds.
Downloads
References
Adjih, C., Baccelli, E., Fleury, E., Harter, G., Mitton, N., Noel, T., Pissard-Gibollet, R., Saint-Marcel, F., Schreiner, G., Vandaele, J., et al. (2015). Fit iot-lab: A large scale open experimental iot testbed. In 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT), pages 459-464. IEEE. DOI: 10.1109/WF-IoT.2015.7389098.
Agrawal, N. and Kumar, R. (2022). Security perspective analysis of industrial cyber physical systems (i-cps): A decade-wide survey. ISA transactions, 130:10-24. DOI: 10.1016/j.isatra.2022.03.018.
Al-Hawawreh, M. and Sitnikova, E. (2020). Developing a security testbed for industrial internet of things. IEEE Internet of Things Journal, 8(7):5558-5573. DOI: 10.1109/JIOT.2020.3032093.
Babun, L., Aksu, H., Ryan, L., Akkaya, K., Bentley, E. S., and Uluagac, A. S. (2020). Z-iot: Passive device-class fingerprinting of zigbee and z-wave iot devices. In ICC 2020-2020 IEEE International Conference on Communications (ICC), pages 1-7. IEEE. DOI: 10.1109/ICC40277.2020.9149285.
Baldin, I., Nikolich, A., Griffioen, J., Monga, I. I. S., Wang, K.-C., Lehman, T., and Ruth, P. (2019). Fabric: A national-scale programmable experimental network infrastructure. IEEE Internet Computing, 23(6):38-47. DOI: 10.1109/MIC.2019.2958545.
Beauchaine, A., Macchiaroli, M., and Yun, M. (2021). ibot: Iot botnet testbed. In 2021 16th International Conference on Computer Science & Education (ICCSE), pages 822-827. IEEE. DOI: 10.1109/ICCSE51940.2021.9569298.
Bettayeb, M., Waraga, O. A., Talib, M. A., Nasir, Q., and Einea, O. (2019). Iot testbed security: Smart socket and smart thermostat. In 2019 IEEE Conference on Application, Information and Network Security (AINS), pages 18-23. IEEE. DOI: 10.1109/AINS47559.2019.8968694.
Cappos, J., Hemmings, M., McGeer, R., Rafetseder, A., and Ricart, G. (2018). Edgenet: a global cloud that spreads by local action. In ACM Symposium on Edge Computing (SEC), pages 359-360. DOI: 10.1109/SEC.2018.00045.
Chernyshev, M., Baig, Z., Bello, O., and Zeadally, S. (2017). Internet of things (iot): Research, simulators, and testbeds. IEEE Internet of Things Journal, 5(3):1637-1647. DOI: 10.1109/JIOT.2017.2786639.
Cintuglu, M. H., Mohammed, O. A., Akkaya, K., and Uluagac, A. S. (2016). A survey on smart grid cyber-physical system testbeds. IEEE Communications Surveys & Tutorials, 19(1):446-464. DOI: 10.1109/COMST.2016.2627399.
Conti, M., Donadel, D., and Turrin, F. (2021). A survey on industrial control system testbeds and datasets for security research. IEEE Communications Surveys & Tutorials, 23(4):2248-2294. DOI: 10.1109/COMST.2021.3094360.
Demeester, P., Van Daele, P., Wauters, T., and Hrasnica, H. (2022). Fed4fire-the largest federation of testbeds in europe. In Building the future internet through FIRE, pages 87-109. River Publishers. Available online [link].
Gardiner, J., Craggs, B., Green, B., and Rashid, A. (2019). Oops i did it again: Further adventures in the land of ics security testbeds. In Proceedings of the ACM Workshop on Cyber-Physical Systems Security & Privacy, pages 75-86. DOI: 10.1145/3338499.3357355.
Gomez, J., Kfoury, E. F., Crichigno, J., and Srivastava, G. (2023). A survey on network simulators, emulators, and testbeds used for research and education. Computer Networks, 237:110054. DOI: 10.1016/j.comnet.2023.110054.
Grossmann, J. and Duponchelle, J. (2008). Graphical network simulator-3. Available online [link]. Accessed in: 20-02-2024.
Hemminger, S. et al. (2005). Network emulation with netem. In Linux conf au, volume 5, page 2005. Available online [link].
Henderson, T. R., Lacage, M., Riley, G. F., Dowell, C., and Kopena, J. (2008). Network simulations with the ns-3 simulator. SIGCOMM demonstration, 14(14):527. Available online [link].
IBM (2024). Cost of a data breach 2023 | ibm -- ibm.com. Available online [link]. Accessed in: 15-02-2024.
Imperva (2023). 2023 Imperva Bad Bot Report | Resource Library -- imperva.com. Available online [link]. Accessed in: Accessed 15-02-2024.
Kampourakis, V., Gkioulos, V., and Katsikas, S. (2023). A systematic literature review on wireless security testbeds in the cyber-physical realm. Computers & Security, page 103383. DOI: 10.1016/j.cose.2023.103383.
Kitchenham, B. (2007). Guidelines for performing Systematic Literature Reviews in software engineering. EBSE Technical Report EBSE-2007-01. Technical report, EBSE Technical Report EBSE-2007-01. Book.
Koroniotis, N., Moustafa, N., Schiliro, F., Gauravaram, P., and Janicke, H. (2021). The sair-iiot cyber testbed as a service: A novel cybertwins architecture in iiot-based smart airports. IEEE Transactions on Intelligent Transportation Systems. DOI: 10.1109/TITS.2021.3106378.
Kumar, A. and Lim, T. J. (2019). A secure contained testbed for analyzing iot botnets. In Testbeds and Research Infrastructures for the Development of Networks and Communities: 13th EAI International Conference, TridentCom 2018, Shanghai, China, December 1-3, 2018, Proceedings 13, pages 124-137. Springer. DOI: 10.1007/978-3-030-12971-2_8.
Lee, G., Lee, J., Kim, Y., and Park, J.-G. (2021). Network flow data re-collecting approach using 5g testbed for labeled dataset. In 2021 23rd International Conference on Advanced Communication Technology (ICACT), pages 254-258. IEEE. DOI: 10.23919/ICACT51234.2021.9370561.
Lee, S., Lee, S., Yoo, H., Kwon, S., and Shon, T. (2018). Design and implementation of cybersecurity testbed for industrial iot systems. The Journal of Supercomputing, 74:4506-4520. DOI: 10.1007/s11227-017-2219-z.
Lochin, E., Perennou, T., and Dairaine, L. (2012). When should i use network emulation? annals of telecommunications-annales des télécommunications, 67:247-255. DOI: 10.1007/s12243-011-0268-5.
Mirkovic, J. and Benzel, T. (2012). Teaching cybersecurity with deterlab. IEEE Security & Privacy, 10(1):73-76. DOI: 10.1109/MSP.2012.23.
Moustafa, N. (2021). A new distributed architecture for evaluating ai-based security systems at the edge: Network ton_iot datasets. Sustainable Cities and Society, 72:102994. DOI: 10.1016/j.scs.2021.102994.
Nock, O., Starkey, J., and Angelopoulos, C. M. (2020). Addressing the security gap in iot: towards an iot cyber range. Sensors, 20(18):5439. DOI: 10.3390/s20185439.
Oliver, I., Kalliola, A., Holtmanns, S., Miche, Y., Limonta, G., Vigmostad, B., and Muller, K. (2018). A testbed for trusted telecommunications systems in a safety critical environment. In Computer Safety, Reliability, and Security: SAFECOMP 2018 Workshops, ASSURE, DECSoS, SASSUR, STRIVE, and WAISE, Västeraas, Sweden, September 18, 2018, Proceedings 37, pages 87-98. Springer. DOI: 10.1007/978-3-319-99229-7_9.
OWASP (2018). IoT Top 10. Technical report, OWSAP. Available online [link].
Peterson, L. and Culler, D. (2002). PlanetLab | An open platform for developing, deploying, and accessing planetary-scale services. Available online [link].
Rampfl, S. (2013). Network simulation and its limitations. In Proceeding zum seminar future internet (FI), Innovative Internet Technologien und Mobilkommunikation (IITM) und autonomous communication networks (ACN), volume 57. Citeseer. DOI: 10.2313/NET-2013-08-1_08.
Rizzo, L. (1997). Dummynet: a simple approach to the evaluation of network protocols. ACM SIGCOMM Computer Communication Review, 27(1):31-41. DOI: 10.1145/251007.251012.
Sáez-de Cámara, X., Flores, J. L., Arellano, C., Urbieta, A., and Zurutuza, U. (2023). Gotham testbed: a reproducible iot testbed for security experiments and dataset generation. IEEE Transactions on Dependable and Secure Computing. DOI: 10.1109/TDSC.2023.3247166.
Siaterlis, C., Garcia, A. P., and Genge, B. (2012). On the use of emulab testbeds for scientifically rigorous experiments. IEEE Communications Surveys & Tutorials, 15(2):929-942. DOI: 10.1109/SURV.2012.0601112.00185.
Siaterlis, C., Genge, B., and Hohenadel, M. (2013). Epic: A testbed for scientifically rigorous cyber-physical security experimentation. IEEE Transactions on Emerging Topics in Computing, 1(2):319-330. DOI: 10.1109/TETC.2013.2287188.
Siboni, S., Sachidananda, V., Meidan, Y., Bohadana, M., Mathov, Y., Bhairav, S., Shabtai, A., and Elovici, Y. (2018). Security testbed for internet-of-things devices. IEEE transactions on reliability, 68(1):23-44. DOI: 10.1109/TR.2018.2864536.
Thom, J., Das, T., Shrestha, B., Sengupta, S., and Arslan, E. (2021). Casting a wide net: An internet of things testbed for cybersecurity education and research. In 2021 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), pages 1-8. IEEE. DOI: 10.23919/SPECTS52716.2021.9639278.
Ukwandu, E., Farah, M. A. B., Hindy, H., Brosset, D., Kavallieros, D., Atkinson, R., Tachtatzis, C., Bures, M., Andonovic, I., and Bellekens, X. (2020). A review of cyber-ranges and test-beds: Current and future trends. Sensors, 20(24):7148. DOI: 10.3390/s20247148.
University of Utah and Flux Research Group (2024). Emulab.Net - Bibliography. Available online [link]. Accessed in: 20-02-2024.
Veksler, V. D., Buchler, N., Hoffman, B. E., Cassenti, D. N., Sample, C., and Sugrim, S. (2018). Simulations in cyber-security: a review of cognitive modeling of network attackers, defenders, and users. Frontiers in psychology, 9:691. DOI: 10.3389/fpsyg.2018.00691.
Waraga, O. A., Bettayeb, M., Nasir, Q., and Talib, M. A. (2020). Design and implementation of automated iot security testbed. Computers & security, 88:101648. DOI: 10.1016/j.cose.2019.101648.
Wroclawski, J., Benzel, T., Blythe, J., Faber, T., Hussain, A., Mirkovic, J., and Schwab, S. (2016). Deterlab and the deter project. The GENI Book, pages 35-62. DOI: 10.1007/978-3-319-33769-2_3.
Xavier, M. G., Neves, M. V., Rossi, F. D., Ferreto, T. C., Lange, T., and De Rose, C. A. (2013). Performance evaluation of container-based virtualization for high performance computing environments. In 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, pages 233-240. IEEE. DOI: 10.1109/PDP.2013.41.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Internet Services and Applications
This work is licensed under a Creative Commons Attribution 4.0 International License.
