Using evidence from systematic studies to guide a PhD research in Requirements Engineering: an experience report
DOI:
https://doi.org/10.5753/jserd.2021.1978Keywords:
Evidence-based software engineering, Graduate education, Tertiary study, Secondary studyAbstract
Conducting systematic studies during a postgraduate program, such as systematic review, systematic mapping, and tertiary review, can benefit the project’s success. They provide an overview of the literature considering currently available research findings, establish baselines for other research activities, and support decisions made throughout the research project. However, there is a shortage of research that presents systematic studies experiences in supporting academic projects. This paper’s main contribution is reporting our experience on how the evidence found in tertiary and secondary studies positively influenced a PhD project’s decisions. Initially, a tertiary study was conducted, followed by a systematic mapping. The evidence returned by the tertiary study led to the definition of the PhD research proposal in the Requirement Engineering field. Moreover, a systematic mapping contributed to the definition of the PhD research problem. From this experience in undertaking systematic studies to support a PhD project, the paper also presents lessons learned and recommendations to guide PhD students’ decisions.
Downloads
References
Barcelos, L. and Penteado, R. (2017). Elaboration of software requirements documents by means of patterns instantiation. J Softw Eng Res Dev, 5(3):1–23.
Beckers, K., Côté, I., and Goeke, L. (2014). A catalog of security requirements patterns for the domain of cloud computing systems. In Proceedings of the ACM Symposium on Applied Computing, pages 337–342.
Bjarnason, E. and Borg, M. (2017). Aligning requirements and testing: Working together toward the same goal. IEEE Software, 34(1):20–23.
Brereton, P. O., Turner, M., and Kaur, R. (2009). Pair programming as a teaching tool: a student review of empirical studies. In 22nd Conference on Software Engineering Education and Training (CSEE&T’ 09).
Centre for Reviews and Dissemination (2002). title=The Database of Abstracts of Reviews of Effects (DARE). Effectiveness Matters, 6(2):1–4.
Chelimsky, D., Astels, D., Helmkamp, B., North, D., Dennis, Z., and Hellesoy, A. (2010). The RSpec Book: Behaviour Driven Development with Rspec, Cucumber, and Friends. Pragmatic Bookshelf, Raleigh, NC, 1st edition.
Clarke, M. and Chalmers, I. (2018). Reflections on the history of systematic reviews. BMJ Evidence-Based Medicine, 23:121–122.
Clear, T. (2015). Follow the moon’ development: Writing a systematic literature review on global software engineering education. In 15th Koli Calling Conference on Computing Education Research, Koli Calling ’15, pages 1–4. ACM.
Costal, D., Franch, X., López, L., Palomares, C., and Quer, C. (2019). On the use of requirement patterns to analyse request for proposal documents. In Laender, A. H. F., Pernici, B., Lim, E., and de Oliveira, J. P. M., editors, Conceptual Modeling - 38th International Conference, ER 2019, Salvador, Brazil, November 4-7, 2019, Proceedings, volume 11788 of Lecture Notes in Computer Science, pages 549–557. Springer.
Cruzes, D. and Dybå, T. (2011). Research synthesis in software engineering: A tertiary study. Information & Software Technology, 53(5):440–455.
Da Silva, R. C. and Benitti, F. B. V. (2011). Writing standards requirements: A systematic literature mapping. In Proceedings of the 14th Workshop on Requirements Engineering, pages 259–272, Rio de Janeiro, RJ, Brazil.
Dybå, T., Kitchenham, B. A., and Jørgensen, M. (2005). Evidence-based software engineering for practitioners. IEEE Software, 22(1):58–65.
Ebert, C. and Ray, R. (2021). Test-driven requirements engineering. IEEE Software, 38(01):16–24.
Egger, M., Smith, G., and Philips, A. (1997). Meta-analysis: Principles and procedures. BMJ, 315(1533–1537).
Fabbri, S. C. P. F., Silva, C., Hernandes, E. M., Octaviano, F., Di Thommazo, A., and Belgamo, A. (2016). Improvements in the start tool to better support the systematic review process. In 20th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 16), pages 21:1–21:5.
Felizardo, K. R., de Souza, E. F., Napoleão, B. M., Vijaykumar, N. L., and Baldassarre, M. T. (2020). Secondary studies in the academic context: A systematic mapping and survey. Journal of Systems and Software, 170:110734.
Franch, X. (2015). Software requirements patterns: A state of the art and the practice. In Proceedings of the 37th International Conference on Software Engineering - Volume 2, ICSE ’15, pages 943–944, Piscataway, NJ, USA. IEEE Press.
Irshad, M., Petersen, K., and Poulding, S. (2018). A systematic literature review of software requirements reuse approaches. Inf. Softw. Technol., 93(C):223–245.
Jamshidi, P., Ghafari, M., Ahmad, A., and Pahl, C. (2013). A framework for classifying and comparing architecturecentric software evolution research. In 2013 17th European Conference on Software Maintenance and Reengineering, pages 305–314.
Justo, J. L. B., Benitti, F. B. V., and Leal, A. C. (2018). Software patterns and requirements engineering activities in real-world settings: A systematic mapping study. Computer Standards & Interfaces, 58:23–42.
Kaijanaho, A.-J. (2017). Teaching master’s degree students to read research literature: Experience in a programming languages course 2002–2017. In 17th Koli Calling Int. Conference on Computing Education Research (Koli Calling ’17), pages 143–147, New York, NY, USA. ACM.
Kitchenham, B. and Brereton, O. (2013). A systematic review of systematic review process research in software engineering. Information and software technology, 55(12):2049–2075.
Kitchenham, B., Budgen, D., and Brereton, O. (2011). Using mapping studies as the basis for further research - a participant-observer case study. Information and Software Technology, 53(6):638–651.
Kitchenham, B., Budgen, D., and Brereton, P. (2015). Evidence-Based Software Engineering and Systematic Reviews. Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series. Chapman & Hall/CRC.
Kitchenham, B. A., Pretorius, R., Budgen, D. an Brereton, P. O., Turner, M., Niazi, M., and Linkman, S. G. (2010). Systematic literature reviews in software engineering - A tertiary study. Information & Software Technology, 52(8):792–805.
Konrad, S. and Cheng, B. H. (2002). Requirements patterns for embedded systems. In Proceedings IEEE Joint International Conference on Requirements Engineering, pages 127–136, Essen, Germany. IEEE.
Kudo, T. N. (2021). A metamodel for the alignment of requirement patterns and test patterns and a metamodel evaluation framework. Phd thesis, Federal University of São Carlos, São Carlos-SP, Brazil. (In Portuguese).
Kudo, T. N., Bulcão-Neto, R. F., Macedo, A. A., and Vincenzi, A. M. R. (2019a). Padrão de requisitos no ciclo de vida de software: Um mapeamento sistemático. In Proceedings of the XXII Iberoamerican Conference on Software Engineering (CIbSE’ 19), pages 420–433.
Kudo, T. N., Bulcão-Neto, R. F., Macedo, A. A., and Vincenzi, A. M. R. (2019b). A revisited systematic literature mapping on the support of requirement patterns for the software development life cycle. Journal of Software Engineering Research and Development, 7:9:1–9:11.
Kudo, T. N., Bulcão Neto, R. F., and Vincenzi, A. M. R. (2019c). A conceptual metamodel to bridging requirement patterns to test patterns. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering, pages 155–160, New York, NY, USA. ACM.
Kudo, T. N., Bulcão-Neto, R. F., and Vincenzi, A. M. R. (2020a). Requirement patterns: a tertiary study and a research agenda. IET Software, 14(1):18–26.
Kudo, T. N., Bulcão-Neto, R. F., and Vincenzi, A. M. R. (2020b). Uma ferramenta para construção de catálogos de padrões de requisitos com comportamento. In Anais do WER20 - Workshop em Engenharia de Requisitos, São José dos Campos, SP, Brasil, August 24-28, 2020. Editora PUC-Rio.
Kudo, T. N., Bulcão-Neto, R. F., Graciano Neto, V. V., and Vincenzi, A. M. R. (2022). Aligning requirements and testing through metamodeling and patterns: Design and evaluation. Requirements Engineering Journal, pages 1–25. (to be published).
Kudo, T. N., Bulcão-Neto, R. F., and Vincenzi, A. M. R. (2020c). Toward a metamodel quality evaluation framework: Requirements, model, measures, and process. In Proceedings of the XXXIV Brazilian Symposium on Software Engineering, SBES 2020, pages 102–107.
Kuhrmann, M. (2017). Teaching empirical software engineering using expert teams. In SEUH, pages 20–31.
Martins, M. C., Kudo, T. N., and Bulcão-Neto, R. F. (2021). Padrões de requisitos para sistemas de registro eletrônico de saúde. In Anais do WER21 - Workshop em Engenharia de Requisitos, Brasília, DF, Brasil, August 23-27, 2021. Editora PUC-Rio.
Mendes, E., Wohlin, C., Felizardo, K. R., and Kalinowski, M. (2020). When to update systematic literature reviews in software engineering. Journal of Systems and Software, 167:110607.
Napoleão, B., Felizardo, K. R., Souza, E. F., and Vijaykumar, N. L. (2017). Practical similarities and differences between systematic literature reviews and systematic mappings: a tertiary study. In 29th International Conference on Software Engineering and Knowledge Engineering (SEKE’ 17), pages 1–10.
Palomares, C., Quer, C., and Franch, X. (2011). Pabre-Man: Management of a requirement patterns catalogue. In 2011 IEEE 19th International Requirements Engineering Conference, pages 341–342.
Palomares, C., Quer, C., and Franch, X. (2017). Requirements reuse and requirement patterns: a state of the practice survey. Empirical Software Engineering, 22(6):2719–2762.
Pejcinovic, B. (2015). Development and uses of iterative systematic literature reviews in electrical engineering education. Electrical and Computer Engineering Faculty Publications and Presentations, 327(1):1–10.
Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.
Souza, E. F., Falbo, R. A., and Vijaykumar, N. L. (2015). Using the findings of a mapping study to conduct a research project: A case in knowledge management in software testing. In 41st Euromicro Conference on Software Engineering and Advanced Applications (SEAA’15), pages 208–215.
Withall, S. (2007). Software Requirement Patterns. Best practices. Microsoft Press, Redmond, Washington.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In 18th International Conference on Evaluation and Assessment in Software Engineering, EASE ’14, London, England, United Kingdom, May 13-14, 2014, pages 38:1–38:10.
Zhang, H. and Babar, M. (2011). An empirical investigation of systematic reviews in software engineering. In 5th International Symposium on Empirical Software Engineering and Measurement (ESEM’ 11), pages 1–10.
Zhou, Y., Zhang, H., Huang, X., Yang, S., Babar, M. A., and Tang, H. (2015). Quality assessment of systematic reviews in software engineering: a tertiary study. In Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, EASE 2015, Nanjing, China, April 27-29, 2015, pages 14:1–14:14.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Taciana Novo Kudo, Renato F. Bulcão-Neto, Auri Marcelo Rizzo Vincenzi, Érica Ferreira de Souza, Katia Romero Felizardo
This work is licensed under a Creative Commons Attribution 4.0 International License.
