Insights from the application of Exploratory Tests in the daily life of distributed teams: an experience report
DOI:
https://doi.org/10.5753/jserd.2023.2646Keywords:
Software testing, Exploratory Testing, Testing Education, Testing Learning and Teaching, Active Learning, JiTT, Just-in-Time Teaching, PBL, Problem Based LearningAbstract
The Exploratory Testing (ET) approach has been adopted in the context of agile development due to the effectiveness of its application. Due to these benefits, the need arose to train agile professionals based on the practical application of this type of test to contribute to its incorporation into the daily work of teams. In this sense, the objective of this article is to investigate the contributions and limitations of adopting Problem-Based Learning (PBL) and Just-in-Time Teaching (JiTT) in ET teaching-learning, and the main aspects that favor or limit the incorporation of ET into the day-to-day of agile teams. For this, we conducted a course in remote teaching format with agile professionals from a software development company, distributed geographically. At the end of the course, data were collected through an online questionnaire and examined with quantitative and qualitative analysis. Then, the ET activities performed by the participants in their daily lives were monitored and a brainstorming session was conducted to evaluate this experience. Our main findings are that (1) the collaboration between participants and the adoption of a real problem, along with (2) activities and resources made available before the class, and (3) the existence of specific tool support for ET sessions optimized learning in the context of remote teaching. Other main results refer to the planning and registration of ET and the need for guidelines to guide the execution of ET. Therefore, integrating theory and practice in ET is necessary for a better understanding of the effects of tests in the agile environment. Additionally, it is necessary to investigate specific approaches and tools that contribute to the execution of the ET and, consequently, to the incorporation of this test into the daily lives of the teams.
Downloads
References
Afzal, W., Ghazi, A. N., Itkonen, J., Torkar, R., Andrews, A., and Bhatti, K. (2015). An experiment on the effectiveness and efficiency of exploratory testing. Empirical Software Engineering, 20:844–878.
Alliance, A. (2016). Agile glossary. URL: [link] Accessed on August 13, 2020.
Andrews, D. H., Hull, T. D., and Donahue, J. A. (2009). Storytelling as an instructional method: Descriptions and research questions. Technical report, Oak Ridge Inst for Science and Education Tn.
Aniche, M., Hermans, F., and van Deursen, A. (2019). Pragmatic software testing education. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, SIGCSE ’19, page 414–420, New York, NY, USA. ACM.
Bach, J. (2003). Exploratory testing explained. Online: [link]
Bonnardel, N. and Didier, J. (2020). Brainstorming variants to favor creative design. Applied ergonomics, 83:102987.
Bonwell, C. C. and Eison, J. A. (1991). Active Learning: Creating Excitement in the Classroom. 1991 ASHE-ERIC Higher Education Reports. ERIC, sl.
Brown, T. and Katz, B. (2011). Change by design. Journal of product innovation management, 28(3):381–383.
Castro, A. K. S. d. (2018). Testes exploratórios: Características, problemas e soluções. B.S. thesis, Universidade Federal do Rio Grande do Norte.
Cheiran, J. F. P., de M. Rodrigues, E., de S. Carvalho, E. L., and da Silva, J. a. P. S. (2017). Problem-based learning to align theory and practice in software testing teaching. In Proceedings of the 31st Brazilian Symposium on Software Engineering, SBES’17, page 328–337, New York, NY, USA. ACM.
Corbin, J. and Strauss, A. (2014). Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.
Costa, I., Oliveira, S., Cardoso, L., Ramos, A., and Sousa, R. (2019). Uma gamificação para ensino e aprendizagem de teste exploratório de software: Aplicação em um estudo experimental. XVIII Simpósio Brasileiro de Jogos e Entretenimento Digital (Education Track–Short Papers), 2019(1):1232–1235.
Coutinho, E. F. and Bezerra, C. I. M. (2018). Uma avaliação inicial do jogo para o ensino de testes de software itestleaening sob a ótica de um software educativo. In Congresso sobre Tecnologias na Educação, volume 3, pages 11–22, Fortaleza,CE. SBC Open Library.
Coutinho, J., Andrade, W., and Machado, P. (2021). Teaching exploratory tests through pbl and jitt: An experience report in a context of distributed teams. In Proceedings of the XXXV Brazilian Symposium on Software Engineering, SBES ’21, page 205–214. Association for Computing Machinery.
Crispin, L. and Gregory, J. (2009). Agile testing: A practical guide for testers and agile teams. Pearson Education, sl.
Crouch, C. H. and Mazur, E. (2001). Peer instruction: Ten years of experience and results. American journal of physics, 69(9):970–977.
de Andrade, S. a. A., de Oliveira Neves, V., and Delamaro, M. E. (2019). Software testing education: Dreams and challenges when bringing academia and industry closer together. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering, SBES 2019, page 47–56, New York, NY, USA. ACM.
Ferreira Costa, I. E. and Oliveira, S. R. B. (2020). The use of gamification to support the teaching-learning of software exploratory testing: an experience report based on the application of a framework. In 2020 IEEE Frontiers in Education Conference (FIE), pages 1–9, Uppsala, Sweden. IEEE.
Figuerêdo, C. d. O., dos Santos, S. C., Borba, P., and Alexandre, G. (2011). Using pbl to develop software test engineers. In International Conference on Computers and Advanced Technology in Education, pages 305–322, Cambridge, United Kingdom. sn.
Garousi, V., Felderer, M., Kuhrmann, M., and Herkiloğlu, K. (2017). What industry wants from academia in software testing? hearing practitioners’ opinions. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, EASE’17, page 65–69, New York, NY, USA. ACM.
Garousi, V., Rainer, A., Lauvås, P., and Arcuri, A. (2020). Software-testing education: A systematic literature mapping. Journal of Systems and Software, 165:110570.
Gebizli, C. and Sözer, H. (2017). Impact of education and experience level on the effectiveness of exploratory testing: An industrial case study. In 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 23–28.
Ghazi, A. N. (2017). Structuring Exploratory Testing Through Test Charter Design and Decision Support. PhD thesis, Blekinge Tekniska Högskola.
Hendrickson, E. (2013). Explore it!: reduce risk and increase confidence with exploratory testing. Pragmatic Bookshelf, sl.
Leite, F. T., Coutinho, J. C. S., and de Sousa, R. R. (2020). An experience report about challenges of software engineering as a second cycle course. In Proceedings of the 34th Brazilian Symposium on Software Engineering, SBES ’20, page 824–833, New York, NY, USA. ACM.
Mårtensson, T., Martini, A., Ståhl, D., and Bosch, J. (2019). Excellence in exploratory testing: Success factors in large-scale industry projects. In Product-Focused Software Process Improvement, pages 299–314. Springer International Publishing.
Martinez, A. (2018). Use of jitt in a graduate software testing course: an experience report. In 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET), pages 108–115, Gothenburg, Sweden. IEEE.
McConnell, J. J. (1996). Active learning and its use in computer science. In Proceedings of the 1st Conference on integrating Technology into Computer Science Education, pages 52–54, Barcelona, Spain. ACM.
Milne, A., Riecke, B., and Antle, A. (2014). Exploring maker practice: Common attitudes, habits and skills from vancouver’s maker community. Studies, 19(21):23.
Novak, G. M. (2011). Just-in-time teaching. New directions for teaching and learning, 2011(128):63–73.
Paiva, M. R. F., Parente, J. R. F., Brandão, I. R., and Queiroz, A. H. B. (2016). Metodologias ativas de ensino-aprendizagem: revisão integrativa. SANARE-Revista de Políticas Públicas, 15(2):145–153.
Paschoal, L. N. and de Souza, S. d. R. S. (2018). A survey on software testing education in brazil. In Proceedings of the 17th Brazilian Symposium on Software Quality, SBQS, page 334–343, New York, NY, USA. ACM.
Paschoal, L. N., Silva, L., and Souza, S. (2017). Abordagem flipped classroom em comparação com o modelo tradicional de ensino: uma investigação empírica no âmbito de teste de software. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), page 476, Recife, PE. SBC Open Library.
Paschoal, L. N. and Souza, S. R. (2018). Planejamento e aplicação de flipped classroom para o ensino de teste de software. RENOTE, 16(2):606–614.
Pfahl, D., Yin, H., Mäntylä, M. V., and Münch, J. (2014). How is exploratory testing used? a state-of-the-practice survey. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’14, New York, NY, USA. Association for Computing Machinery.
Queiroz, R., Pinto, F., and Silva, P. (2019). Islandtest: jogo educativo para apoiar o processo ensino-aprendizagem de testes de software. In Anais do XXVII Workshop sobre Educação em Computação, pages 533–542, Belém,PA. SBC Open Library.
Raappana, P., Saukkoriipi, S., Tervonen, I., and Mäntylä, M. V. (2016). The effect of team exploratory testing–experience report from f-secure. In 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 295–304, Chicago, IL, USA. IEEE.
Scatalon, L. P., Carver, J. C., Garcia, R. E., and Barbosa, E. F. (2019). Software testing in introductory programming courses: A systematic mapping study. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, SIGCSE ’19, page 421–427, New York, NY, USA. ACM.
Suranto, B. (2015). Exploratory software testing in agile project. In 2015 International Conference on Computer, Communications, and Control Technology (I4CT), pages 280–283, Kuching, Malaysia. IEEE.
Whittaker, J. A. (2009). Exploratory software testing: tips, tricks, tours, and techniques to guide test design. Pearson Education, sl.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media, sl.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Jarbele C. S. Coutinho, Wilkerson L. Andrade, Patrícia D. L. Machado
This work is licensed under a Creative Commons Attribution 4.0 International License.
