Extraction of test cases procedures from textual use cases: is it worth it?
DOI:
https://doi.org/10.5753/jserd.2020.731Keywords:
Software Testing, Test Generation, Test FactoryAbstract
Software testing plays a major role in the quality of software since it assures that the software complies with its expected behavior. However, this is an expensive activity and, thus, companies usually do not perform testing activities on software projects due to the time required. These costs may be even higher in testing processes that rely only on manual test execution, which is both time-consuming and error-prone. One strategy commonly used to mitigate these costs is to use tools to automate testing activities such as test execution, test documentation, and test case generation. This paper presents an experience report in the context of a Test Factory about the use of a tool that partially automates the specification of test procedures from textual use cases. This tool automatically retrieves use cases from the requirement management system, generates the test case procedures, requires inputs from the tester, and then sends the test cases to the test management system. This paper details how this tool was used in real releases of an industrial software project. We also evaluated our tool with four test analysts from a different project to gather more data regarding its efficiency to support the test case documentation.
Downloads
Download data is not yet available.
References
R. M. d. C. Andrade, I. d. S. Santos, V. Lelli, K. M.de Oliveira, and A. R. Rocha. Software testing process ina test factoryfrom ad hoc activities to an organizational standard. In ICEIS (2), pages 132–143, 2017.
B. Aragão, I. Santos, T. Nogueira, L. Mesquita, and R. Andrade. Modelagem interativa de um processo de desenvolvimento com base na percepção da equipe: Um relatode experiência. InAnais do XIII Simpósio Brasileiro deSistemas de Informação, pages 428–435. SBC, 2017.
B. S. Aragão, R. M. C. Andrade, I. S. Santos, R. N. S. Castro, V. Lelli, and T. G. R. Darin. Testdcat: Catalog of testdebt subtypes and management activities. In Testing Software and Systems, pages 279–295, Cham, 2019. Springer International Publishing. ISBN 9783030312800.
A.Bertolino. Softwaretestingresearch: Achievements, challenges, dreams. In 2007 Future of Software Engineering,FOSE ’07, pages 85–103, Washington, DC, USA, 2007. IEEE Computer Society. ISBN 0769528295. URL http://dx.doi.org/10.1109/FOSE.2007.25.
R. M. de Castro Andrade, I. de Sousa Santos, V. Lelli, K. M. de Oliveira, and A. R. C. da Rocha. Software testing process in a test factory from ad hoc activities to an organizational standard. In ICEIS, 2017.
D. Freudenstein, M. Junker, J. Radduenz, S. Eder, and B.Hauptmann. Automatedtestdesignfromrequirementsthe specmate tool. In 018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET), pages 5–8. IEEE, 2018.
V. Garousi and M. Felderer. Living in two different worlds: A comparison of industry and academic focus areas in software testing.IEEE Software, (1):1–1, 2017.
V. Garousi and M. V. Mäntylä. When and what to automatein software testing? a multivocal literature review.Information and Software Technology, 76:92–117, 2016.
T. Gorschek, P. Garre, S. Larsson, and C. Wohlin. A modelfor technology transfer in practice.IEEE software, 23(6):88–95, 2006.
J. Gutiérrez, M. Escalona, and M. Mejías. A modeldrivenapproach for functional test case generation.Journal ofSystems and Software, 109:214–228, 2015.
ISO/IEC291192. Iso/iec/ieee international standard software and systems engineering software testing part2:test processes.ISO/IEC/IEEE 291192:2013(E), pages1–68, Sept 2013.
D. N. Jorge, P. D. Machado, E. L. Alves, and W. L. Andrade. Integrating requirements specification and modelbased testing in agile development. In2018 IEEE 26thInternational Requirements Engineering Conference(RE),pages 336–346. IEEE, 2018.
L. Lazicand N. Mastorakis. Cost effective software test metrics. WSEAS Transactions on Computers, 7(6):599–619,2008.
J. L. Massollar, R. M. de Mello, and G. H. Travassos. Structuring and verifying requirement specifications through activity diagrams to support the semiautomated generation of functional test procedures. In 2012 Eighth International Conference on the Quality of Information and Communications Technology, pages 239–244. IEEE, 2012.
A. Mette and J. Hass. Testing processes. In Software Testing Verification and Validation Workshop, 2008. ICSTW’08.IEEE International Conference on, pages 321–327. IEEE,2008.
M. A. Montoni, A. R. Rocha, and K. C. Weber. Mps. br: a successful program for software process improvement in brazil Software Process: Improvement and Practice, 14(5):289–300, 2009
G. J. Myers, T. Badgett, T. M. Thomas, and C. Sandler.Theart of software testing, volume 2. Wiley Online Library,2004.
S. Nogueira, H. Araujo, R. Araujo, J. Iyoda, and A.Sampaio. Test case generation, selection and coverage from natural language. Science of Computer Programming, pages 84–110, 2019.
J. Preece, Y. Rogers, and H. Sharp. Interaction design. Apogeo Editore, 2004.
G. Samarthyam, M. Muralidharan, and R. K. Anna. Understanding test debt. In Trends in Software Testing, pages1–17. Springer, 2017.
E. B. d. Santos, L. S. d. Costa, B. S. Aragão, I. d. S. Santos,and R. M. d. C. Andrade. Extraction of test cases procedures from textual use cases to reduce test effort: Test factory experience report. In Proceedings of the XVIII Brazilian Symposiumon Software Quality,pages266–275,2019.
K. Schwaber and M. Beedle.Agile software developmentwith Scrum, volume 1. Prentice Hall Upper Saddle River,2002.
S. Seela and R. Yackel. 64 essential testing metrics for measuring quality assurance success. URL https://www.qasymphony.com/blog/64-test-metrics/.
H. M. Sneed. Requirementbased testingextracting logicaltest cases from requirement documents. In International Conference on Software Quality, pages 60–79. Springer,2018.
S. S. Some and X. Cheng. An approach for supporting systemlevel test scenarios generation from textual usecases. InProceedings of the 2008 ACM symposium onApplied computing, pages 724–729. ACM, 2008.
L. S. Vieira, C. G. L. Barreto, E. B. dos Santos, B. S. Aragão, I. de Sousa Santos, and R. M. C. Andrade. Automaçãode testes em uma fábrica de testes: Um relato de experiência. InAnais do XIV Simpósio Brasileiro de Sistemas deInformação, pages 80–73. SBC, 2018
L. S. Vieira, C. G. L. Barreto, E. B. dos Santos, B. S. Aragão, I. de Sousa Santos, and R. M. C. Andrade. Automaçãode testes em uma fábrica de testes: Um relato de experiência. In Anais do XIV Simpósio Brasileiro de Sistemas deInformação, pages 80–73. SBC, 2018b.
K. Wiegers and J. Beatty. Software requirements. Pearson Education, 2013.
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. Experimentation in software engineering. Springer Science & Business Media, 2012.
T. Yue, S. Ali, and M. Zhang. Rtcm: a natural language based, automated, and practical test case generation framework. In Proceedings of the 2015 International Symposium on Software Testing and Analysis, pages 397–408.ACM, 2015
B. Aragão, I. Santos, T. Nogueira, L. Mesquita, and R. Andrade. Modelagem interativa de um processo de desenvolvimento com base na percepção da equipe: Um relatode experiência. InAnais do XIII Simpósio Brasileiro deSistemas de Informação, pages 428–435. SBC, 2017.
B. S. Aragão, R. M. C. Andrade, I. S. Santos, R. N. S. Castro, V. Lelli, and T. G. R. Darin. Testdcat: Catalog of testdebt subtypes and management activities. In Testing Software and Systems, pages 279–295, Cham, 2019. Springer International Publishing. ISBN 9783030312800.
A.Bertolino. Softwaretestingresearch: Achievements, challenges, dreams. In 2007 Future of Software Engineering,FOSE ’07, pages 85–103, Washington, DC, USA, 2007. IEEE Computer Society. ISBN 0769528295. URL http://dx.doi.org/10.1109/FOSE.2007.25.
R. M. de Castro Andrade, I. de Sousa Santos, V. Lelli, K. M. de Oliveira, and A. R. C. da Rocha. Software testing process in a test factory from ad hoc activities to an organizational standard. In ICEIS, 2017.
D. Freudenstein, M. Junker, J. Radduenz, S. Eder, and B.Hauptmann. Automatedtestdesignfromrequirementsthe specmate tool. In 018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET), pages 5–8. IEEE, 2018.
V. Garousi and M. Felderer. Living in two different worlds: A comparison of industry and academic focus areas in software testing.IEEE Software, (1):1–1, 2017.
V. Garousi and M. V. Mäntylä. When and what to automatein software testing? a multivocal literature review.Information and Software Technology, 76:92–117, 2016.
T. Gorschek, P. Garre, S. Larsson, and C. Wohlin. A modelfor technology transfer in practice.IEEE software, 23(6):88–95, 2006.
J. Gutiérrez, M. Escalona, and M. Mejías. A modeldrivenapproach for functional test case generation.Journal ofSystems and Software, 109:214–228, 2015.
ISO/IEC291192. Iso/iec/ieee international standard software and systems engineering software testing part2:test processes.ISO/IEC/IEEE 291192:2013(E), pages1–68, Sept 2013.
D. N. Jorge, P. D. Machado, E. L. Alves, and W. L. Andrade. Integrating requirements specification and modelbased testing in agile development. In2018 IEEE 26thInternational Requirements Engineering Conference(RE),pages 336–346. IEEE, 2018.
L. Lazicand N. Mastorakis. Cost effective software test metrics. WSEAS Transactions on Computers, 7(6):599–619,2008.
J. L. Massollar, R. M. de Mello, and G. H. Travassos. Structuring and verifying requirement specifications through activity diagrams to support the semiautomated generation of functional test procedures. In 2012 Eighth International Conference on the Quality of Information and Communications Technology, pages 239–244. IEEE, 2012.
A. Mette and J. Hass. Testing processes. In Software Testing Verification and Validation Workshop, 2008. ICSTW’08.IEEE International Conference on, pages 321–327. IEEE,2008.
M. A. Montoni, A. R. Rocha, and K. C. Weber. Mps. br: a successful program for software process improvement in brazil Software Process: Improvement and Practice, 14(5):289–300, 2009
G. J. Myers, T. Badgett, T. M. Thomas, and C. Sandler.Theart of software testing, volume 2. Wiley Online Library,2004.
S. Nogueira, H. Araujo, R. Araujo, J. Iyoda, and A.Sampaio. Test case generation, selection and coverage from natural language. Science of Computer Programming, pages 84–110, 2019.
J. Preece, Y. Rogers, and H. Sharp. Interaction design. Apogeo Editore, 2004.
G. Samarthyam, M. Muralidharan, and R. K. Anna. Understanding test debt. In Trends in Software Testing, pages1–17. Springer, 2017.
E. B. d. Santos, L. S. d. Costa, B. S. Aragão, I. d. S. Santos,and R. M. d. C. Andrade. Extraction of test cases procedures from textual use cases to reduce test effort: Test factory experience report. In Proceedings of the XVIII Brazilian Symposiumon Software Quality,pages266–275,2019.
K. Schwaber and M. Beedle.Agile software developmentwith Scrum, volume 1. Prentice Hall Upper Saddle River,2002.
S. Seela and R. Yackel. 64 essential testing metrics for measuring quality assurance success. URL https://www.qasymphony.com/blog/64-test-metrics/.
H. M. Sneed. Requirementbased testingextracting logicaltest cases from requirement documents. In International Conference on Software Quality, pages 60–79. Springer,2018.
S. S. Some and X. Cheng. An approach for supporting systemlevel test scenarios generation from textual usecases. InProceedings of the 2008 ACM symposium onApplied computing, pages 724–729. ACM, 2008.
L. S. Vieira, C. G. L. Barreto, E. B. dos Santos, B. S. Aragão, I. de Sousa Santos, and R. M. C. Andrade. Automaçãode testes em uma fábrica de testes: Um relato de experiência. InAnais do XIV Simpósio Brasileiro de Sistemas deInformação, pages 80–73. SBC, 2018
L. S. Vieira, C. G. L. Barreto, E. B. dos Santos, B. S. Aragão, I. de Sousa Santos, and R. M. C. Andrade. Automaçãode testes em uma fábrica de testes: Um relato de experiência. In Anais do XIV Simpósio Brasileiro de Sistemas deInformação, pages 80–73. SBC, 2018b.
K. Wiegers and J. Beatty. Software requirements. Pearson Education, 2013.
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. Experimentation in software engineering. Springer Science & Business Media, 2012.
T. Yue, S. Ali, and M. Zhang. Rtcm: a natural language based, automated, and practical test case generation framework. In Proceedings of the 2015 International Symposium on Software Testing and Analysis, pages 397–408.ACM, 2015
Downloads
Published
2020-11-29
How to Cite
Santos, E. B. dos, da Costa, L. S., de Amorim, T. M., Aragão, B. S., Santos, I. de S., de Vasconcelos, D. R., & Andrade, R. M. de C. (2020). Extraction of test cases procedures from textual use cases: is it worth it?. Journal of Software Engineering Research and Development, 8, 9:1 – 9:17. https://doi.org/10.5753/jserd.2020.731
Issue
Section
Research Article
License
Copyright (c) 2020 Erick Barros dos Santos, Lucas Simão da Costa, Thaís Marinho de Amorim, Bruno Sabóia Aragão, Ismayle de Sousa Santos, Danilo Reis de Vasconcelos, Rossana Maria de Castro Andrade
This work is licensed under a Creative Commons Attribution 4.0 International License.
