Characterization of software testing practices: A replicated survey in Costa Rica
DOI:
https://doi.org/10.5753/jserd.2019.472Keywords:
Software Testing, Industry Practices, Survey, Costa Rica, Replication, Empirical Software EngineeringAbstract
Software testing is an essential activity in software development projects for delivering high quality products. In a previous study, we reported the results of a survey of software engineering practices in the Costa Rican industry. To make a more in depth analysis of the specific software testing practices among practitioners, we replicated a previous survey conducted in South America. Our objective was to characterize the state of the practice based on practitioners use and perceived importance of those practices. This survey evaluated 42 testing practices grouped in three categories: processes, activities and tools. A total of 92 practitioners responded to the survey. The participants indicated that: (1) tasks for recording of the results of tests, documentation of test procedures and cases, and re-execution of tests when the software is modified are useful and important for software testing practitioners. (2) Acceptance and system testing are the two most useful and important testing types. (3) Tools for recording defects and the effort to fix them (bug tracking) and the availability of a test database for reuse are useful and important. Regarding use and implementation of practices, the participants stated that (4) Planning and designing of software testing before coding and evaluating the quality of test artifacts are not a regular practice. (5) There is a lack of measurement of defect density and test coverage in the industry; and (6) tools for automatic generation of test cases and for estimating testing effort are rarely used. This study gave us a first glance at the state of the practice in software testing in a thriving and very dynamic industry that currently employs most of our computer science professionals. The benefits are twofold: for academia, it provides us with a road map to revise our academic offer, and for practitioners it provides them with a first set of data to benchmark their practices.
Downloads
Download data is not yet available.
References
Andersson, C. and Runeson, P. (2002). Verification and validation in industry-a qualitative survey on the state of practice. In Proceedings International Symposium on Empirical Software Engineering, pages 37–47. IEEE.
Aymerich, B., Díaz-Oreiro, I., Guzmán, J. C., López, G., and Garbanzo, D. (2018). Software development practices in costa rica: A survey. In International Conference on Applied Human Factors and Ergonomics, pages 122–132. Springer.
Basili, V., Gianluigi, C., and Rombach, D. (1994). The goal question metric approach. Encyclopedia of software engineering, pages 528–532.
Beck, L. L. and Perkins, T. E. (1983). A survey of software engineering practice: tools, methods, and results. IEEE Transactions on Software Engineering, (5):541–561.
Bhuiyan, S. A. R., Rahim, M. S., Chowdhury, A. E., and Hasan, M. H. (2018). A survey of software quality assurance and testing practices and challenges in bangladesh. International Journal of Computer Applications, 975:8887.
Carver, J. C. (2010). Towards reporting guidelines for experimental replications: A proposal. In 1st international workshop on replication in empirical software engineering, pages 2–5. Citeseer.
Carver, J. C., Juristo, N., Baldassarre, M. T., and Vegas, S. (2014). Replications of software engineering experiments. Causevic, A., Sundmark, D., and Punnekkat, S. (2010). An
industrial survey on contemporary aspects of software testing. In 2010 Third International Conference on Software Testing, Verification and Validation, pages 393–401.
IEEE.
Chan, F., Tse, T., Tang, W., and Chen, T. (2005). Software testing education and training in hong kong. In Fifth International Conference on Quality Software (QSIC’05), pages 313–316. IEEE.
Daka, E. and Fraser, G. (2014). A survey on unit testing practices and problems. In 2014 IEEE 25th International Symposium on Software Reliability Engineering, pages 201–211. IEEE.
De Greca, F., Rossi, B. D., Robiolo, G., and Travassos, G. H. (2015). Aplicación y valoración de la verificación y validación de software: una encuesta realizada en buenos aires. In Simposio Argentino de Ingeniería de Software (ASSE 2015)-JAIIO 44 (Rosario, 2015).
Deak, A. (2014). A comparative study of testers’ motivation in traditional and agile software development. In International Conference on Product-Focused Software Process Improvement, pages 1–16. Springer.
Deak, A. and Stålhane, T. (2013). Organization of testing activities in norwegian software companies. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, pages 102–107. IEEE.
Deak, A., Stålhane, T., and Cruzes, D. (2013). Factors influencing the choice of a career in software testing among norwegian students. Software Engineering, page 796.
Dias-Neto, A., Natali, A. C. C., Rocha, A. R., and Travassos, G. H. (2006). Caracterização do estado da prática das atividades de teste em um cenário de desenvolvimento de software brasileiro. V Simpósio Brasileiro de Qualidade de Software, Vila Velha, ES.
Dias-Neto, A. C., Matalonga, S., Solari, M., Robiolo, G., and Travassos, G. H. (2017). Toward the characterization of software testing practices in south america: looking at brazil and uruguay. Software Quality Journal, 25(4):1145–1183.
Engström, E. and Runeson, P. (2010). A qualitative survey of regression testing practices. In International Conference on Product Focused Software Process Improvement, pages 3–16. Springer.
Fernández-Sanz, L. (2005). Un sondeo sobre la práctica actual de pruebas de software en españa. REICIS. Revista Española de Innovación, Calidad e Ingeniería del Software, 1(2).
Fernández-Sanz, L., Villalba, M. T., Hilera, J. R., and Lacuesta, R. (2009). Factors with negative influence on software testing practice in spain: A survey. In European conference on software process improvement, pages 1–12. Springer.
Garousi, V., Coşkunçay, A., Betin-Can, A., and Demirörs, O. (2015). A survey of software engineering practices in turkey. Journal of Systems and Software, 108:148–177.
Garousi, V., Coşkunçay, A., Demirörs, O., and Yazici, A. (2016). Cross-factor analysis of software engineering practices versus practitioner demographics: An exploratory study in turkey. Journal of Systems and Software, 111:49–73.
Garousi, V. and Felderer, M. (2017). Living in two different worlds: A comparison of industry and academic focus areas in software testing. IEEE Software, (1):1–1.
Garousi, V., Felderer, M., Kuhrmann, M., and Herkiloğlu, K. (2017). What industry wants from academia in software testing?: Hearing practitioners’ opinions. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, pages 65–69. ACM.
Garousi, V. and Varma, T. (2010). A replicated survey of software testing practices in the canadian province of alberta: What has changed from 2004 to 2009? Journal of Systems and Software, 83(11):2251–2262.
Garousi, V. and Zhi, J. (2013). A survey of software testing practices in canada. Journal of Systems and Software, 86(5):1354–1376.
Gelperin, D. and Hetzel, B. (1988). The growth of software testing. Communications of the ACM, 31(6):687–695.
Geras, A. M., Smith, M. R., and Miller, J. (2004). A survey of software testing practices in alberta. Canadian Journal of Electrical and Computer Engineering, 29(3):183–191.
Ghazi, A. N., Petersen, K., and Börstler, J. (2015). Heterogeneous systems testing techniques: An exploratory survey. In International Conference on Software Quality, pages 67–85. Springer.
Ghazi, A. N., Petersen, K., Reddy, S. S. V. R., and Nekkanti, H. (2017). Survey research in software engineering: problems and strategies. arXiv preprint arXiv:1704.01090.
Greiler, M., Deursen, A. v., and Storey, M.-A. (2012). Test confessions: A study of testing practices for plug-in systems. In Proceedings of the 34th International Conference on Software Engineering, pages 244–254. IEEE Press.
Grindal, M., Offutt, J., and Mellin, J. (2006). On the testing maturity of software producing organizations. In Testing: Academic & Industrial Conference-Practice And Research Techniques (TAIC PART’06), pages 171–180. IEEE.
Hynninen, T., Kasurinen, J., Knutas, A., and Taipale, O. (2018). Software testing: Survey of the industry practices. In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1449–1454. IEEE.
Juristo, N. and Gómez, O. S. (2010). Replication of software engineering experiments. In Empirical software engineering and verification, pages 60–88. Springer.
Juristo, N., Moreno, A. M., and Vegas, S. (2004). Reviewing 25 years of testing technique experiments. Empirical Software Engineering, 9(1-2):7–44.
Kanij, T., Merkel, R., and Grundy, J. (2014). A preliminary survey of factors affecting software testers. In 2014 23rd Australian Software Engineering Conference, pages 180–189. IEEE.
Kassab, M. (2018). Testing practices of software in safety critical systems: Industrial survey. In 20th International Conference on Enterprise Information Systems, ICEIS 2018, pages 359–367. SciTePress.
Kassab, M., DeFranco, J. F., and Laplante, P. A. (2017). Software testing: The state of the practice. IEEE Software, 34(5):46–52.
Kasurinen, J., Taipale, O., and Smolander, K. (2010). Software test automation in practice: empirical observations.
Advances in Software Engineering, 2010.
Kirk, D. and Tempero, E. (2012). Software development practices in new zealand. In 2012 19th Asia-Pacific Software Engineering Conference, volume 1, pages 386–395. IEEE.
Kochhar, P. S., Thung, F., Nagappan, N., Zimmermann, T., and Lo, D. (2015). Understanding the test automation culture of app developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pages 1–10. IEEE.
Kochhar, P. S., Xia, X., and Lo, D. (2019). Practitioners’ views on good software testing practices. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, pages 61–70. IEEE Press.
Kuhrmann, M., Diebold, P., Münch, J., Tell, P., Garousi, V., Felderer, M., Trektere, K., McCaffery, F., Linssen, O., Hanser, E., et al. (2017). Hybrid software and system development in practice: waterfall, scrum, and beyond. In Proceedings of the 2017 International Conference on Software and System Process, pages 30–39. ACM.
Lee, J., Kang, S., and Lee, D. (2012). Survey on software testing practices. IET software, 6(3):275–282.
Lima, B. and Faria, J. P. (2016). A survey on testing distributed and heterogeneous systems: The state of the practice. In International Conference on Software Technologies, pages 88–107. Springer.
Linåker, J., Sulaman, S. M., Maiani de Mello, R., and Höst, M. (2015). Guidelines for conducting surveys in software engineering.
Lindsay, R. M. and Ehrenberg, A. S. (1993). The design of replicated studies. The American Statistician, 47(3):217–228.
Molléri, J. S., Petersen, K., and Mendes, E. (2016). Survey guidelines in software engineering: An annotated review. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, page 58. ACM.
Ng, S., Murnane, T., Reed, K., Grant, D., and Chen, T. (2004). A preliminary survey on software testing practices in australia. In 2004 Australian Software Engineering Conference. Proceedings., pages 116–125. IEEE.
Park, J., Ryu, H., Choi, H.-J., and Ryu, D.-K. (2008). A survey on software test maturity in korean defense industry. In Proceedings of the 1st India software engineering conference, pages 149–150. ACM.
Pérez, J., Mens, T., and Kamseu, F. (2013). A pilot study on software quality practices in belgian industry. In 2013 17th European Conference on Software Maintenance and Reengineering, pages 395–398. IEEE.
Pfahl, D., Yin, H., Mäntylä, M. V., and Münch, J. (2014). How is exploratory testing used? a state-of-the-practice survey. In Proceedings of the 8th ACM/IEEE international symposium on empirical software engineering and measurement, page 5. ACM.
Pham, R., Singer, L., Liskin, O., Figueira Filho, F., and Schneider, K. (2013). Creating a shared understanding of testing culture on a social coding site. In Proceedings of the 2013 International Conference on Software Engineering, pages 112–121. IEEE Press.
Quesada-López, C., Hernandez-Aguero, E., and Jenkins, M. (2019). A survey of software testing practices in costa rica. In Proceedings of the XXII Ibero-American Conference on Software Engineering (CibSE 2019). La Habana, Cuba, 23-27 Abril 2019, pages 107–145.
Quesada-López, C. and Jenkins, M. (2017). Estudio sobre las prácticas de la ingeniería de software en costa rica: Resultados preliminares. In Proceedings of the XX IberoAmerican Conference on Software Engineering (CibSE 2017). Buenos Aires, Argentina, 22-23 May 2017, pages 107–145.
Quesada-López, C. and Jenkins, M. (2018). Factores asociados a prácticas de desarrollo y pruebas de software en costa rica: Un estudio exploratorio. In Proceedings of the XXI Ibero-American Conference on Software Engineering (CibSE 2018). Bogotá, Colombia, 23-27 Abril 2018, pages 107–145.
Rafi, D. M., Moses, K. R. K., Petersen, K., and Mäntylä, M. V. (2012). Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. In Proceedings of the 7th International Workshop on Automation of Software Test, pages 36–42. IEEE Press.
Raulamo-Jurvanen, P., Hosio, S., and Mäntylä, M. V. (2019). Practitioner evaluations on software testing tools. In Proceedings of the Evaluation and Assessment on Software Engineering, pages 57–66. ACM.
Robiolo, G., M, M., Rossi, B., and Travassos, G. H. (2017). Aplicación e importancia de las pruebas de software: una encuesta realizada en buenos aires en el Ámbito público. In XX Ibero-American Conference on Software Engineering (CibSE 2017). Argentina, 22-23 May 2017.
Rodrigues, A. and Dias-Neto, A. (2016). Relevance and impact of critical factors of success in software test automation lifecycle: A survey. In Proceedings of the 1st Brazilian Symposium on Systematic and Automated Software Testing, page 6. ACM.
Runeson, P. (2006). A survey of unit testing practices. IEEEsoftware, 23(4):22–29.
Runeson, P., Andersson, C., and Höst, M. (2003). Test processes in software product evolution—a qualitative survey on the state of practice. Journal of software maintenance and evolution: Research and practice, 15(1):41–59.
Scatalon, L. P., Fioravanti, M. L., Prates, J. M., Garcia, R. E., and Barbosa, E. F. (2018). A survey on graduates’ curriculum-based knowledge gaps in software testing. In 2018 IEEE Frontiers in Education Conference (FIE), pages 1–8. IEEE.
Smolander, K., Taipale, O., and Kasurinen, J. (2016). Software test automation in practice: Empirical observations. In Data Structure and Software Engineering, pages 107–145. Apple Academic Press.
Sung, P. W.-B. and Paynter, J. (2006). Software testing practices in new zealand. In In Proceedings of the 19th Annual Conference of the National Advisory Committee on Computing Qualifications, pages 273–282.
Taipale, O., Smolander, K., and Kälviäinen, H. (2005). Finding and ranking research directions for software testing. In European Conference on Software Process Improvement,pages 39–48. Springer.
Taipale, O., Smolander, K., and Kälviäinen, H. (2006). A survey on software testing. 6th International SPICE.
Torkar, R. and Mankefors, S. (2003). A survey on testing and reuse. In Proceedings 2003 Symposium on Security and Privacy, pages 164–173. IEEE.
Vasanthapriyan, S. (2018). A study of software testing practices in sri lankan software companies. In 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), pages 339–344. IEEE.
Vonken, F., Brunekreef, J., Zaidman, A., and Peeters, F. (2012). Software engineering in the netherlands: the state of the practice. Technical Report Series TUD-SERG-2012-022.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering, page 38. Citeseer.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media.
Wojcicki, M. A. and Strooper, P. (2006). A state-of-practice questionnaire on verification and validation for concurrent programs. In Proceedings of the 2006 workshop on Parallel and distributed systems: testing and debugging, pages 1–10. ACM.
Yli-Huumo, J., Taipale, O., and Smolander, K. (2014). Software development methods and quality assurance: Special focus on south korea. In European Conference on Software Process Improvement, pages 159–169. Springer
Aymerich, B., Díaz-Oreiro, I., Guzmán, J. C., López, G., and Garbanzo, D. (2018). Software development practices in costa rica: A survey. In International Conference on Applied Human Factors and Ergonomics, pages 122–132. Springer.
Basili, V., Gianluigi, C., and Rombach, D. (1994). The goal question metric approach. Encyclopedia of software engineering, pages 528–532.
Beck, L. L. and Perkins, T. E. (1983). A survey of software engineering practice: tools, methods, and results. IEEE Transactions on Software Engineering, (5):541–561.
Bhuiyan, S. A. R., Rahim, M. S., Chowdhury, A. E., and Hasan, M. H. (2018). A survey of software quality assurance and testing practices and challenges in bangladesh. International Journal of Computer Applications, 975:8887.
Carver, J. C. (2010). Towards reporting guidelines for experimental replications: A proposal. In 1st international workshop on replication in empirical software engineering, pages 2–5. Citeseer.
Carver, J. C., Juristo, N., Baldassarre, M. T., and Vegas, S. (2014). Replications of software engineering experiments. Causevic, A., Sundmark, D., and Punnekkat, S. (2010). An
industrial survey on contemporary aspects of software testing. In 2010 Third International Conference on Software Testing, Verification and Validation, pages 393–401.
IEEE.
Chan, F., Tse, T., Tang, W., and Chen, T. (2005). Software testing education and training in hong kong. In Fifth International Conference on Quality Software (QSIC’05), pages 313–316. IEEE.
Daka, E. and Fraser, G. (2014). A survey on unit testing practices and problems. In 2014 IEEE 25th International Symposium on Software Reliability Engineering, pages 201–211. IEEE.
De Greca, F., Rossi, B. D., Robiolo, G., and Travassos, G. H. (2015). Aplicación y valoración de la verificación y validación de software: una encuesta realizada en buenos aires. In Simposio Argentino de Ingeniería de Software (ASSE 2015)-JAIIO 44 (Rosario, 2015).
Deak, A. (2014). A comparative study of testers’ motivation in traditional and agile software development. In International Conference on Product-Focused Software Process Improvement, pages 1–16. Springer.
Deak, A. and Stålhane, T. (2013). Organization of testing activities in norwegian software companies. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, pages 102–107. IEEE.
Deak, A., Stålhane, T., and Cruzes, D. (2013). Factors influencing the choice of a career in software testing among norwegian students. Software Engineering, page 796.
Dias-Neto, A., Natali, A. C. C., Rocha, A. R., and Travassos, G. H. (2006). Caracterização do estado da prática das atividades de teste em um cenário de desenvolvimento de software brasileiro. V Simpósio Brasileiro de Qualidade de Software, Vila Velha, ES.
Dias-Neto, A. C., Matalonga, S., Solari, M., Robiolo, G., and Travassos, G. H. (2017). Toward the characterization of software testing practices in south america: looking at brazil and uruguay. Software Quality Journal, 25(4):1145–1183.
Engström, E. and Runeson, P. (2010). A qualitative survey of regression testing practices. In International Conference on Product Focused Software Process Improvement, pages 3–16. Springer.
Fernández-Sanz, L. (2005). Un sondeo sobre la práctica actual de pruebas de software en españa. REICIS. Revista Española de Innovación, Calidad e Ingeniería del Software, 1(2).
Fernández-Sanz, L., Villalba, M. T., Hilera, J. R., and Lacuesta, R. (2009). Factors with negative influence on software testing practice in spain: A survey. In European conference on software process improvement, pages 1–12. Springer.
Garousi, V., Coşkunçay, A., Betin-Can, A., and Demirörs, O. (2015). A survey of software engineering practices in turkey. Journal of Systems and Software, 108:148–177.
Garousi, V., Coşkunçay, A., Demirörs, O., and Yazici, A. (2016). Cross-factor analysis of software engineering practices versus practitioner demographics: An exploratory study in turkey. Journal of Systems and Software, 111:49–73.
Garousi, V. and Felderer, M. (2017). Living in two different worlds: A comparison of industry and academic focus areas in software testing. IEEE Software, (1):1–1.
Garousi, V., Felderer, M., Kuhrmann, M., and Herkiloğlu, K. (2017). What industry wants from academia in software testing?: Hearing practitioners’ opinions. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, pages 65–69. ACM.
Garousi, V. and Varma, T. (2010). A replicated survey of software testing practices in the canadian province of alberta: What has changed from 2004 to 2009? Journal of Systems and Software, 83(11):2251–2262.
Garousi, V. and Zhi, J. (2013). A survey of software testing practices in canada. Journal of Systems and Software, 86(5):1354–1376.
Gelperin, D. and Hetzel, B. (1988). The growth of software testing. Communications of the ACM, 31(6):687–695.
Geras, A. M., Smith, M. R., and Miller, J. (2004). A survey of software testing practices in alberta. Canadian Journal of Electrical and Computer Engineering, 29(3):183–191.
Ghazi, A. N., Petersen, K., and Börstler, J. (2015). Heterogeneous systems testing techniques: An exploratory survey. In International Conference on Software Quality, pages 67–85. Springer.
Ghazi, A. N., Petersen, K., Reddy, S. S. V. R., and Nekkanti, H. (2017). Survey research in software engineering: problems and strategies. arXiv preprint arXiv:1704.01090.
Greiler, M., Deursen, A. v., and Storey, M.-A. (2012). Test confessions: A study of testing practices for plug-in systems. In Proceedings of the 34th International Conference on Software Engineering, pages 244–254. IEEE Press.
Grindal, M., Offutt, J., and Mellin, J. (2006). On the testing maturity of software producing organizations. In Testing: Academic & Industrial Conference-Practice And Research Techniques (TAIC PART’06), pages 171–180. IEEE.
Hynninen, T., Kasurinen, J., Knutas, A., and Taipale, O. (2018). Software testing: Survey of the industry practices. In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1449–1454. IEEE.
Juristo, N. and Gómez, O. S. (2010). Replication of software engineering experiments. In Empirical software engineering and verification, pages 60–88. Springer.
Juristo, N., Moreno, A. M., and Vegas, S. (2004). Reviewing 25 years of testing technique experiments. Empirical Software Engineering, 9(1-2):7–44.
Kanij, T., Merkel, R., and Grundy, J. (2014). A preliminary survey of factors affecting software testers. In 2014 23rd Australian Software Engineering Conference, pages 180–189. IEEE.
Kassab, M. (2018). Testing practices of software in safety critical systems: Industrial survey. In 20th International Conference on Enterprise Information Systems, ICEIS 2018, pages 359–367. SciTePress.
Kassab, M., DeFranco, J. F., and Laplante, P. A. (2017). Software testing: The state of the practice. IEEE Software, 34(5):46–52.
Kasurinen, J., Taipale, O., and Smolander, K. (2010). Software test automation in practice: empirical observations.
Advances in Software Engineering, 2010.
Kirk, D. and Tempero, E. (2012). Software development practices in new zealand. In 2012 19th Asia-Pacific Software Engineering Conference, volume 1, pages 386–395. IEEE.
Kochhar, P. S., Thung, F., Nagappan, N., Zimmermann, T., and Lo, D. (2015). Understanding the test automation culture of app developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pages 1–10. IEEE.
Kochhar, P. S., Xia, X., and Lo, D. (2019). Practitioners’ views on good software testing practices. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, pages 61–70. IEEE Press.
Kuhrmann, M., Diebold, P., Münch, J., Tell, P., Garousi, V., Felderer, M., Trektere, K., McCaffery, F., Linssen, O., Hanser, E., et al. (2017). Hybrid software and system development in practice: waterfall, scrum, and beyond. In Proceedings of the 2017 International Conference on Software and System Process, pages 30–39. ACM.
Lee, J., Kang, S., and Lee, D. (2012). Survey on software testing practices. IET software, 6(3):275–282.
Lima, B. and Faria, J. P. (2016). A survey on testing distributed and heterogeneous systems: The state of the practice. In International Conference on Software Technologies, pages 88–107. Springer.
Linåker, J., Sulaman, S. M., Maiani de Mello, R., and Höst, M. (2015). Guidelines for conducting surveys in software engineering.
Lindsay, R. M. and Ehrenberg, A. S. (1993). The design of replicated studies. The American Statistician, 47(3):217–228.
Molléri, J. S., Petersen, K., and Mendes, E. (2016). Survey guidelines in software engineering: An annotated review. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, page 58. ACM.
Ng, S., Murnane, T., Reed, K., Grant, D., and Chen, T. (2004). A preliminary survey on software testing practices in australia. In 2004 Australian Software Engineering Conference. Proceedings., pages 116–125. IEEE.
Park, J., Ryu, H., Choi, H.-J., and Ryu, D.-K. (2008). A survey on software test maturity in korean defense industry. In Proceedings of the 1st India software engineering conference, pages 149–150. ACM.
Pérez, J., Mens, T., and Kamseu, F. (2013). A pilot study on software quality practices in belgian industry. In 2013 17th European Conference on Software Maintenance and Reengineering, pages 395–398. IEEE.
Pfahl, D., Yin, H., Mäntylä, M. V., and Münch, J. (2014). How is exploratory testing used? a state-of-the-practice survey. In Proceedings of the 8th ACM/IEEE international symposium on empirical software engineering and measurement, page 5. ACM.
Pham, R., Singer, L., Liskin, O., Figueira Filho, F., and Schneider, K. (2013). Creating a shared understanding of testing culture on a social coding site. In Proceedings of the 2013 International Conference on Software Engineering, pages 112–121. IEEE Press.
Quesada-López, C., Hernandez-Aguero, E., and Jenkins, M. (2019). A survey of software testing practices in costa rica. In Proceedings of the XXII Ibero-American Conference on Software Engineering (CibSE 2019). La Habana, Cuba, 23-27 Abril 2019, pages 107–145.
Quesada-López, C. and Jenkins, M. (2017). Estudio sobre las prácticas de la ingeniería de software en costa rica: Resultados preliminares. In Proceedings of the XX IberoAmerican Conference on Software Engineering (CibSE 2017). Buenos Aires, Argentina, 22-23 May 2017, pages 107–145.
Quesada-López, C. and Jenkins, M. (2018). Factores asociados a prácticas de desarrollo y pruebas de software en costa rica: Un estudio exploratorio. In Proceedings of the XXI Ibero-American Conference on Software Engineering (CibSE 2018). Bogotá, Colombia, 23-27 Abril 2018, pages 107–145.
Rafi, D. M., Moses, K. R. K., Petersen, K., and Mäntylä, M. V. (2012). Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. In Proceedings of the 7th International Workshop on Automation of Software Test, pages 36–42. IEEE Press.
Raulamo-Jurvanen, P., Hosio, S., and Mäntylä, M. V. (2019). Practitioner evaluations on software testing tools. In Proceedings of the Evaluation and Assessment on Software Engineering, pages 57–66. ACM.
Robiolo, G., M, M., Rossi, B., and Travassos, G. H. (2017). Aplicación e importancia de las pruebas de software: una encuesta realizada en buenos aires en el Ámbito público. In XX Ibero-American Conference on Software Engineering (CibSE 2017). Argentina, 22-23 May 2017.
Rodrigues, A. and Dias-Neto, A. (2016). Relevance and impact of critical factors of success in software test automation lifecycle: A survey. In Proceedings of the 1st Brazilian Symposium on Systematic and Automated Software Testing, page 6. ACM.
Runeson, P. (2006). A survey of unit testing practices. IEEEsoftware, 23(4):22–29.
Runeson, P., Andersson, C., and Höst, M. (2003). Test processes in software product evolution—a qualitative survey on the state of practice. Journal of software maintenance and evolution: Research and practice, 15(1):41–59.
Scatalon, L. P., Fioravanti, M. L., Prates, J. M., Garcia, R. E., and Barbosa, E. F. (2018). A survey on graduates’ curriculum-based knowledge gaps in software testing. In 2018 IEEE Frontiers in Education Conference (FIE), pages 1–8. IEEE.
Smolander, K., Taipale, O., and Kasurinen, J. (2016). Software test automation in practice: Empirical observations. In Data Structure and Software Engineering, pages 107–145. Apple Academic Press.
Sung, P. W.-B. and Paynter, J. (2006). Software testing practices in new zealand. In In Proceedings of the 19th Annual Conference of the National Advisory Committee on Computing Qualifications, pages 273–282.
Taipale, O., Smolander, K., and Kälviäinen, H. (2005). Finding and ranking research directions for software testing. In European Conference on Software Process Improvement,pages 39–48. Springer.
Taipale, O., Smolander, K., and Kälviäinen, H. (2006). A survey on software testing. 6th International SPICE.
Torkar, R. and Mankefors, S. (2003). A survey on testing and reuse. In Proceedings 2003 Symposium on Security and Privacy, pages 164–173. IEEE.
Vasanthapriyan, S. (2018). A study of software testing practices in sri lankan software companies. In 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), pages 339–344. IEEE.
Vonken, F., Brunekreef, J., Zaidman, A., and Peeters, F. (2012). Software engineering in the netherlands: the state of the practice. Technical Report Series TUD-SERG-2012-022.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering, page 38. Citeseer.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media.
Wojcicki, M. A. and Strooper, P. (2006). A state-of-practice questionnaire on verification and validation for concurrent programs. In Proceedings of the 2006 workshop on Parallel and distributed systems: testing and debugging, pages 1–10. ACM.
Yli-Huumo, J., Taipale, O., and Smolander, K. (2014). Software development methods and quality assurance: Special focus on south korea. In European Conference on Software Process Improvement, pages 159–169. Springer
Downloads
Published
2019-12-19
How to Cite
Quesada-López, C., Hernandez-Agüero, E., & Jenkins, M. (2019). Characterization of software testing practices: A replicated survey in Costa Rica. Journal of Software Engineering Research and Development, 7, 6:1 – 6:20. https://doi.org/10.5753/jserd.2019.472
Issue
Section
Research Article
License
Copyright (c) 2019 Christian Quesada-López
This work is licensed under a Creative Commons Attribution 4.0 International License.
