Specifying the Process Model for Systematic Reviews: An Augmented Proposal
DOI:
https://doi.org/10.5753/jserd.2019.460Keywords:
Systematic Literature Review; Process Modeling Perspectives; SPEM; Process Improvement; Software Testing OntologyAbstract
Context: Systematic Literature Review (SLR) is a research methodology intended to obtain evidence from scientific articles stored in digital libraries. SLRs can be performed on primary and secondary studies. Although there are guidelines to the SLR process in Software Engineering, the SLR process is not fully and rigorously specified yet. Moreover, it can often be observed a lack of a clear separation of concerns between what to do (process) and how to do it (methods). Objective: To specify the SLR process in a more detailed and rigorous manner by considering different process modeling perspectives, such as functional, behavioral, organizational and informational. The main objective in this work is specifying the SLR activities rather than their methods. Method: The SPEM (Software & Systems Process Engineering Metamodel) language is used to model the SLR process from different perspectives. In addition, we illustrate aspects of the proposed process by using a recently conducted SLR on software testing ontologies. Results: Our SLR process model specifications favor a clear identification of what task/activities should be performed, in which order, by whom, and which are the consumed and produced artifacts as well as their inner structures. Also, we explicitly specify activities related to the SLR pilot test, analyzing the gains. Conclusion: The proposed SLR process considers with higher rigor the principles and benefits of process modeling backing SLRs to be more systematic, repeatable and auditable for researchers and practitioners. In fact, the rigor provided by process modeling, where several perspectives are combined, but can also be independently detached, provides a greater richness of expressiveness in sequences and decision flows, while representing different levels of granularity in the work definitions, such as activity, sub-activity and task.
Downloads
Download data is not yet available.
References
Arnicans, G., Romans, D., & Straujums, U. (2013). Semi-automatic Generation of a Software Testing Lightweight Ontology from a Glossary Based on the ONTO6 Methodology, Frontiers in Artificial Intelligence and Applications, V.249, pp. 263-276.
Asman, A., & Srikanth, R. M. (2016). A Top Domain Ontology for Software Testing, Master Thesis, Jönköping University, Sweden, pp. 1-74.
Bai, X., Lee, S., Tsai, W. T., & Chen, Y. (2008). OntologyBased Test Modeling and Partition Testing of Web Services, IEEE Int’l Conference on Web Services (ICWS'08), pp. 465-472.
Barbosa, E. F., Nakagawa, E. Y., Riekstin, A. C., & Maldonado, J. C. (2008). Ontology-based Development of Testing Related Tools, 20th International Conference on Software Engineering and Knowledge Engineering (SEKE'08), pp. 697-702.
Becker, P., Lew, P., & Olsina, L. (2012). Specifying Process Views for a Measurement, Evaluation, and Improvement Strategy, Advances in Software Engineering Journal, Academic Editor: Osamu Mizuno, Hindawi Publishing Corporation, USA, V.2012, 27 pgs.
Becker, P., Papa, F., & Olsina, L. (2015). Process Ontology Specification for Enhancing the Process Compliance of a Measurement and Evaluation Strategy, CLEI eJnal., 18:(1), pp. 1-26.
Biolchini, J., Mian, P.G., Natali A.C.C., & Travassos, G. (2005). Systematic Review in Software Engineering. Technical Report RT-ES 679-05. PESC, COPPE/UFRJ.
Brereton, P., Kitchenham, B., Budgen, D., Turner, M., & Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain, Journal of Systems and Software, 80:(4), pp. 571–583.
Cai, L., Tong, W., Liu, Z., & Zhang, J. (2009). Test Case Reuse Based on Ontology, 15th IEEE Pacific Rim International Symposium on Dependable Computing, pp. 103-108.
Campos, H., Acácio, C., Braga, R., Araújo, M. A. P., David, J. M. N., & Campos, F. (2017). Regression Tests Provenance Data in the Continuous Software Engineering Context, 2nd Brazilian Symposyum on Systematic and Automated Software Testing (SAST), Paper 10, pp. 1-6.
Curtis, B., Kellner, M., & Over, J. (1992). Process Modelling, Communications of ACM, 35:(9), pp. 75-90.
Freitas, A., & Vieira, R. (2014). An Ontology for Guiding Performance Testing, IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), (WI-IAT '14), V.1, pp. 400-407.
Garousi, V., & Mäntylä, M. (2016). A systematic literature review of literature reviews in software testing, Information and Software Technology, V.80, pp. 195-216.
Irshad, M., Petersen, K., & Poulding, S. (2018). A systematic literature review of software requirements reuse approaches, Information and Software Technology, V.93, pp. 223-245.
Kitchenham, B. (2004). Procedures for Undertaking Systematic Reviews, Joint TR, Comp. Science Dep., Keele University (TR/SE-0401) and National ICT Australia Ltd. (0400011T.1).
Kitchenham, B., & Brereton, P. (2013). A systematic review of systematic review process research in software engineering, Information and Software Technology, 55:(12), pp. 2049-2075.
Kitchenham, B., & Charters, S. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering, EBSE Technical Report, Software Engineering Group, School of Computer Science and Mathematics Keele University and Department of Computer Science University of Durham, UK, V. 2.3.
Kitchenham, B., Budgen, D., & Brereton, P. (2010a). The value of mapping studies-a participant-observer case study, 4th International Conference on Evaluation and Assessment in Software Engineering, British Computer Society, pp. 25–33.
Kitchenham, B., Pretorius, R., Budgen, D., Brereton, P., Turner, M., Niazi, M., & Linkman, S. (2010b). Systematic literature reviews in software engineering - A tertiary study, Information and Software Technology, 52:(8), pp. 792–805.
Kitchenham, B.A., Dyba, T., & Jorgensen, M. (2004). Evidence-based software engineering. 26th International Conference on Software Engineering, pp. 273-281.
Lam, R. W., & Kennedy, S. H. (2005). Using metaanalysis to evaluate evidence: Practical tips and traps. Canadian Journal of Psychiatry, 50, pp. 167–174.
Lebo, T., Sahoo, S., McGuinness, D., Belhajjame, K., Cheney, J., Corsar, D., Garijo, D., Soiland-Reyes, S., Specifying the Process Model for Systematic Reviews: An Augmented Proposal Becker et al. 2019 Zednik, S., & Zhao, J. (2013). PROV-O: The PROV Ontology. Retrieved by July 1st, 2019, from https://www.w3.org/TR/2013/REC-prov-o-20130430/
Marshall, C., & Brereton, P. (2013). Tools to Support Systematic Literature Reviews in Software Engineering: A Mapping Study. ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Baltimore, MD, pp. 296-299. DOI: <a href="http://dx.doi.org/10.1109/ESEM.2013.32">http://dx.doi.org/10.1109/ESEM.2013.32</a>
Meline, T. (2006). Selecting studies for systematic review: Inclusion and exclusion criteria. Contemporary issues in communication science and disorders, 33, pp. 21-27.
Napoleão, B.M., Felizardo, K.R., de Souza, E.F., & Vijaykumar, N. (2017). Practical similarities and differences between Systematic Literature Reviews and Systematic Mappings: a tertiary study, The 29th International Conference on Software Engineering and Knowledge Engineering, pp.
85-90, DOI: <a href="http://dx.doi.org/10.18293/SEKE2017-069">http://dx.doi.org/10.18293/SEKE2017-069</a>
Olsina, L. (1998). Functional View of the Hypermedia Process Model, 5th International Workshop on Engineering Hypertext Functionality, at ICSE’98, Kyoto, Japan, pp. 1-10.
Olsina, L., & Becker, P. (2017). Family of Strategies for different Evaluation Purposes, XX CIbSE’17, CABA, Argentina, Published by Curran Associates, pp. 221-234.
Olsina, L., & Becker, P. (2018). Linking Business and Information Need Goals with Functional and Non-functional Requirements, XXI CIbSE’18, Bogotá, Colombia, Published by Curran Associates, pp. 381-394.
Olsina, L., Covella, G., & Dieser, A. (2013). Metrics and Indicators as Key Organizational Assets for ICT Security Assessment, Chapter 2, in the book titled "Emerging Trends in ICT Security", Elsevier (Morgan Kaufmann), 1st Edition, Akhgar & Arabnia (Eds.), pp. 25-44. ISBN: 9780124114746
OMG (2008). Software & Systems Process Engineering Meta-Model (SPEM) Specification, Version 2.0.
OMG (2011). Business Process Model and Notation (BPMN) Specification, Version 2.0.
OMG (2017). Unified Modeling Language (UML) Specification, Version 2.5.1.
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update, Information and Software Technology, V.64, pp. 1–18.
Portela, C., Vasconcelos, A., Silva, A., Sinimbú, A., Silva, E., Ronny, M., Lira, W., & Oliveira, S. (2012). A Comparative Analysis between BPMN and SPEM Modeling Standards in the Software Processes Context, Journal of Software Engineering and Applications, 5(5), pp. 330-339.
Rosenberg, D.S.W., Gray, J., Hayes, R., & Richardson, W. (1996). Evidence-based medicine: what it is and what it isn’t, British Medical Journal, Vol. 312, No. 7023, p. 71.
Russel, N., van der Aalst, W., Hofstede, A., & Wohed, P. (2006). On the suitability of uml activity diagrams for business process modelling, Third Asia-Pacific Conference on Conceptual Modelling (APCCM), Hobart, V.53, pp. 195–204.
Sapna, P. G., & Mohanty, H. (2011). An Ontology Based Approach for Test Scenario Management, 5th International Conference on Information Intelligence, Systems, Technology and Management (ICISTM'2011), V.141, pp. 91-100.
Sepúlveda, S., Cravero, A., & Cachero, C. (2016). Requirements modeling languages for software product lines: A systematic literature review, Information and Software Technology, V.69, pp. 16-36.
Souza, E. F., Falbo, R. A., & Vijaykumar, N. L. (2017). ROoST: Reference Ontology on Software Testing, Applied Ontology Journal, 12:(1), pp. 1-30.
Tahir, T., Rasool, G., & Gencel, C. (2016). A systematic literature review on software measurement programs, Information and Software Technology, V.73, pp. 101-121.
Tebes, G., Peppino, D., Dameno, J., Becker, P., & Olsina L. (2018). Diseñando una Revisión Sistemática de Literatura sobre Ontologías de Testing de Software. (In English: Designing a Systematic Literature Review on Software Testing Ontologies), VI Congreso Nacional de Ingeniería Informática/Sistemas de Información (CoNaIISI), Mar del Plata, Argentina, pp. 1-13.
Tebes, G., Peppino, D., Becker, P., & Olsina, L. (2019a). Especificación del Modelo de Proceso para una Revisión Sistemática de Literatura. (In English: Specifying the Process Model for a Systematic Literature Review), XXII CIbSE’19, La Habana, Cuba, Published by Curran Associates 2019, pp. 391-404, ISBN 978-1-5108-8795-4.
Tebes, G., Peppino, D., Becker, P., Matturro, G., Solari, M., & Olsina, L. (2019b). A Systematic Review on Software Testing Ontologies, 12th International Conference on the Quality of Information and Communications Technology, Springer book, CCIS V.1010, M. Piattini et al. (Eds.):
QUATIC 2019, Ciudad Real, Spain, pp. 144-160,<a href="https://doi.org/10.1007/978-3-030-29238-6_11">https://doi.org/10.1007/978-3-030-29238-6_11</a>
Torrecilla-Salinas, C.J., Sedeño, J., Escalona, M.J., & Mejías, M. (2016). Agile, Web Engineering and Capability Maturity Model Integration: A systematic literature review, Information and Software Technology, V.71, pp. 92-107.
Vasanthapriyan, S., Tian, J., & Xiang J. (2017a). An Ontology-Based Knowledge Framework for Software Testing, Communications in Computer and Information Science, V.780, pp. 212-226.
Vasanthapriyan, S., Tian, J., Zhao, D., Xiong, S., & Xiang, J. (2017b). An Ontology-Based Knowledge Sharing Portal for Software Testing, IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C'17), pp. 472-479.
White, S. A. (2004). Process modeling notations and workflow patterns, Workflow Handbook 2004, pp. 265–294.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering, Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE '14), ACM, New York, NY, USA, Article 38, pp. 1-38.
Zhu, H., & Huo, Q. (2005). Developing A Software Testing Ontology in UML for a Software Growth Environment of Web-Based Applications, Software Evolution with UML and XML, IDEA Group, pp. 263-295
Asman, A., & Srikanth, R. M. (2016). A Top Domain Ontology for Software Testing, Master Thesis, Jönköping University, Sweden, pp. 1-74.
Bai, X., Lee, S., Tsai, W. T., & Chen, Y. (2008). OntologyBased Test Modeling and Partition Testing of Web Services, IEEE Int’l Conference on Web Services (ICWS'08), pp. 465-472.
Barbosa, E. F., Nakagawa, E. Y., Riekstin, A. C., & Maldonado, J. C. (2008). Ontology-based Development of Testing Related Tools, 20th International Conference on Software Engineering and Knowledge Engineering (SEKE'08), pp. 697-702.
Becker, P., Lew, P., & Olsina, L. (2012). Specifying Process Views for a Measurement, Evaluation, and Improvement Strategy, Advances in Software Engineering Journal, Academic Editor: Osamu Mizuno, Hindawi Publishing Corporation, USA, V.2012, 27 pgs.
Becker, P., Papa, F., & Olsina, L. (2015). Process Ontology Specification for Enhancing the Process Compliance of a Measurement and Evaluation Strategy, CLEI eJnal., 18:(1), pp. 1-26.
Biolchini, J., Mian, P.G., Natali A.C.C., & Travassos, G. (2005). Systematic Review in Software Engineering. Technical Report RT-ES 679-05. PESC, COPPE/UFRJ.
Brereton, P., Kitchenham, B., Budgen, D., Turner, M., & Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain, Journal of Systems and Software, 80:(4), pp. 571–583.
Cai, L., Tong, W., Liu, Z., & Zhang, J. (2009). Test Case Reuse Based on Ontology, 15th IEEE Pacific Rim International Symposium on Dependable Computing, pp. 103-108.
Campos, H., Acácio, C., Braga, R., Araújo, M. A. P., David, J. M. N., & Campos, F. (2017). Regression Tests Provenance Data in the Continuous Software Engineering Context, 2nd Brazilian Symposyum on Systematic and Automated Software Testing (SAST), Paper 10, pp. 1-6.
Curtis, B., Kellner, M., & Over, J. (1992). Process Modelling, Communications of ACM, 35:(9), pp. 75-90.
Freitas, A., & Vieira, R. (2014). An Ontology for Guiding Performance Testing, IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), (WI-IAT '14), V.1, pp. 400-407.
Garousi, V., & Mäntylä, M. (2016). A systematic literature review of literature reviews in software testing, Information and Software Technology, V.80, pp. 195-216.
Irshad, M., Petersen, K., & Poulding, S. (2018). A systematic literature review of software requirements reuse approaches, Information and Software Technology, V.93, pp. 223-245.
Kitchenham, B. (2004). Procedures for Undertaking Systematic Reviews, Joint TR, Comp. Science Dep., Keele University (TR/SE-0401) and National ICT Australia Ltd. (0400011T.1).
Kitchenham, B., & Brereton, P. (2013). A systematic review of systematic review process research in software engineering, Information and Software Technology, 55:(12), pp. 2049-2075.
Kitchenham, B., & Charters, S. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering, EBSE Technical Report, Software Engineering Group, School of Computer Science and Mathematics Keele University and Department of Computer Science University of Durham, UK, V. 2.3.
Kitchenham, B., Budgen, D., & Brereton, P. (2010a). The value of mapping studies-a participant-observer case study, 4th International Conference on Evaluation and Assessment in Software Engineering, British Computer Society, pp. 25–33.
Kitchenham, B., Pretorius, R., Budgen, D., Brereton, P., Turner, M., Niazi, M., & Linkman, S. (2010b). Systematic literature reviews in software engineering - A tertiary study, Information and Software Technology, 52:(8), pp. 792–805.
Kitchenham, B.A., Dyba, T., & Jorgensen, M. (2004). Evidence-based software engineering. 26th International Conference on Software Engineering, pp. 273-281.
Lam, R. W., & Kennedy, S. H. (2005). Using metaanalysis to evaluate evidence: Practical tips and traps. Canadian Journal of Psychiatry, 50, pp. 167–174.
Lebo, T., Sahoo, S., McGuinness, D., Belhajjame, K., Cheney, J., Corsar, D., Garijo, D., Soiland-Reyes, S., Specifying the Process Model for Systematic Reviews: An Augmented Proposal Becker et al. 2019 Zednik, S., & Zhao, J. (2013). PROV-O: The PROV Ontology. Retrieved by July 1st, 2019, from https://www.w3.org/TR/2013/REC-prov-o-20130430/
Marshall, C., & Brereton, P. (2013). Tools to Support Systematic Literature Reviews in Software Engineering: A Mapping Study. ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Baltimore, MD, pp. 296-299. DOI: <a href="http://dx.doi.org/10.1109/ESEM.2013.32">http://dx.doi.org/10.1109/ESEM.2013.32</a>
Meline, T. (2006). Selecting studies for systematic review: Inclusion and exclusion criteria. Contemporary issues in communication science and disorders, 33, pp. 21-27.
Napoleão, B.M., Felizardo, K.R., de Souza, E.F., & Vijaykumar, N. (2017). Practical similarities and differences between Systematic Literature Reviews and Systematic Mappings: a tertiary study, The 29th International Conference on Software Engineering and Knowledge Engineering, pp.
85-90, DOI: <a href="http://dx.doi.org/10.18293/SEKE2017-069">http://dx.doi.org/10.18293/SEKE2017-069</a>
Olsina, L. (1998). Functional View of the Hypermedia Process Model, 5th International Workshop on Engineering Hypertext Functionality, at ICSE’98, Kyoto, Japan, pp. 1-10.
Olsina, L., & Becker, P. (2017). Family of Strategies for different Evaluation Purposes, XX CIbSE’17, CABA, Argentina, Published by Curran Associates, pp. 221-234.
Olsina, L., & Becker, P. (2018). Linking Business and Information Need Goals with Functional and Non-functional Requirements, XXI CIbSE’18, Bogotá, Colombia, Published by Curran Associates, pp. 381-394.
Olsina, L., Covella, G., & Dieser, A. (2013). Metrics and Indicators as Key Organizational Assets for ICT Security Assessment, Chapter 2, in the book titled "Emerging Trends in ICT Security", Elsevier (Morgan Kaufmann), 1st Edition, Akhgar & Arabnia (Eds.), pp. 25-44. ISBN: 9780124114746
OMG (2008). Software & Systems Process Engineering Meta-Model (SPEM) Specification, Version 2.0.
OMG (2011). Business Process Model and Notation (BPMN) Specification, Version 2.0.
OMG (2017). Unified Modeling Language (UML) Specification, Version 2.5.1.
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update, Information and Software Technology, V.64, pp. 1–18.
Portela, C., Vasconcelos, A., Silva, A., Sinimbú, A., Silva, E., Ronny, M., Lira, W., & Oliveira, S. (2012). A Comparative Analysis between BPMN and SPEM Modeling Standards in the Software Processes Context, Journal of Software Engineering and Applications, 5(5), pp. 330-339.
Rosenberg, D.S.W., Gray, J., Hayes, R., & Richardson, W. (1996). Evidence-based medicine: what it is and what it isn’t, British Medical Journal, Vol. 312, No. 7023, p. 71.
Russel, N., van der Aalst, W., Hofstede, A., & Wohed, P. (2006). On the suitability of uml activity diagrams for business process modelling, Third Asia-Pacific Conference on Conceptual Modelling (APCCM), Hobart, V.53, pp. 195–204.
Sapna, P. G., & Mohanty, H. (2011). An Ontology Based Approach for Test Scenario Management, 5th International Conference on Information Intelligence, Systems, Technology and Management (ICISTM'2011), V.141, pp. 91-100.
Sepúlveda, S., Cravero, A., & Cachero, C. (2016). Requirements modeling languages for software product lines: A systematic literature review, Information and Software Technology, V.69, pp. 16-36.
Souza, E. F., Falbo, R. A., & Vijaykumar, N. L. (2017). ROoST: Reference Ontology on Software Testing, Applied Ontology Journal, 12:(1), pp. 1-30.
Tahir, T., Rasool, G., & Gencel, C. (2016). A systematic literature review on software measurement programs, Information and Software Technology, V.73, pp. 101-121.
Tebes, G., Peppino, D., Dameno, J., Becker, P., & Olsina L. (2018). Diseñando una Revisión Sistemática de Literatura sobre Ontologías de Testing de Software. (In English: Designing a Systematic Literature Review on Software Testing Ontologies), VI Congreso Nacional de Ingeniería Informática/Sistemas de Información (CoNaIISI), Mar del Plata, Argentina, pp. 1-13.
Tebes, G., Peppino, D., Becker, P., & Olsina, L. (2019a). Especificación del Modelo de Proceso para una Revisión Sistemática de Literatura. (In English: Specifying the Process Model for a Systematic Literature Review), XXII CIbSE’19, La Habana, Cuba, Published by Curran Associates 2019, pp. 391-404, ISBN 978-1-5108-8795-4.
Tebes, G., Peppino, D., Becker, P., Matturro, G., Solari, M., & Olsina, L. (2019b). A Systematic Review on Software Testing Ontologies, 12th International Conference on the Quality of Information and Communications Technology, Springer book, CCIS V.1010, M. Piattini et al. (Eds.):
QUATIC 2019, Ciudad Real, Spain, pp. 144-160,<a href="https://doi.org/10.1007/978-3-030-29238-6_11">https://doi.org/10.1007/978-3-030-29238-6_11</a>
Torrecilla-Salinas, C.J., Sedeño, J., Escalona, M.J., & Mejías, M. (2016). Agile, Web Engineering and Capability Maturity Model Integration: A systematic literature review, Information and Software Technology, V.71, pp. 92-107.
Vasanthapriyan, S., Tian, J., & Xiang J. (2017a). An Ontology-Based Knowledge Framework for Software Testing, Communications in Computer and Information Science, V.780, pp. 212-226.
Vasanthapriyan, S., Tian, J., Zhao, D., Xiong, S., & Xiang, J. (2017b). An Ontology-Based Knowledge Sharing Portal for Software Testing, IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C'17), pp. 472-479.
White, S. A. (2004). Process modeling notations and workflow patterns, Workflow Handbook 2004, pp. 265–294.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering, Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE '14), ACM, New York, NY, USA, Article 38, pp. 1-38.
Zhu, H., & Huo, Q. (2005). Developing A Software Testing Ontology in UML for a Software Growth Environment of Web-Based Applications, Software Evolution with UML and XML, IDEA Group, pp. 263-295
Downloads
Published
2019-12-21
How to Cite
Olsina, L., Becker, P., Peppino, D., & Tebes, G. (2019). Specifying the Process Model for Systematic Reviews: An Augmented Proposal. Journal of Software Engineering Research and Development, 7, 7:1 – 7:23. https://doi.org/10.5753/jserd.2019.460
Issue
Section
Research Article
