On the use of Change History Data to Enhance Class Change-Proneness Prediction Models

Rogério de Carvalho Silva; Paulo Roberto Farah; Silvia Regina Vergilio

doi:10.5753/jbcs.2024.3782

Authors

Rogério de Carvalho Silva Federal University of Paraná https://orcid.org/0000-0002-5975-1277
Paulo Roberto Farah Santa Catarina Sate University https://orcid.org/0000-0001-8023-7611
Silvia Regina Vergilio Federal University of Paraná https://orcid.org/0000-0003-3139-6266

DOI:

https://doi.org/10.5753/jbcs.2024.3782

Keywords:

class change-proneness, software maintenance, software evolution

Abstract

As software evolves, new artifacts are created, modified, or removed. One of these main artifacts generated in the development of object-oriented software is the class. Classes have a very dynamic life cycle that can result in additional costs to the project. One way to mitigate this is to detect, in the early stages of the development, classes that are prone to change. Some approaches in the literature adopt Machine Learning (ML) algorithms to predict the change-proneness of a class. However, most of these approaches do not consider the temporal dependency between training instances, i.e., they consider that the instances are independent. To overcome such a limitation, this study presents an approach for predicting change-proneness based on the class change history. The approach adopts the sliding window method and is evaluated to obtain six kinds of models, which are derived by using, as predictors, different sets of metrics: structural, evolutionary, and smell-based. The evaluation uses five systems, four ML algorithms, and also explores some resample techniques to deal with imbalanced data. Regardless of the kind of model analyzed and the algorithm used, our approach overcomes the traditional one in 378 (~80) cases, out of 420, considering all systems, kinds of models, indicators, and algorithms. Moreover, the results show that our approach presents the best performance when the set of evolutionary metrics is used as predictors. There is no improvement when smell-based metrics are added. The Random Forest algorithm with the resampling technique ADA reaches the best performance among the ML algorithms evaluated.

Downloads

Download data is not yet available.

Author Biographies

Rogério de Carvalho Silva, Federal University of Paraná

He has a bachelor's degree in Software Engineering from the Federal University of Ceará (2016), with academic mobility at Technische Universitaet Muenchen (TUM, Germany). Master's degree in Software Engineering from the Federal University of Paraná (2022). Has experience in research in the areas of Artificial Intelligence, Web Systems, Databases and Graphs

Silvia Regina Vergilio, Federal University of Paraná

Silvia Regina Vergilio received Master and Doctoral degress from University of Campinas (UNICAMP), Brazil. She is currently a professor of Software Engineering in the Computer Science Department of Federal University of Paraná (UFPR), Brasil, where she leads the Research Group on Software Engineering. She has involved in several projects and her research is mainly supported by CNPq (PQ Level 1D). Her research interests include software testing, software reliability, Software Product Lines (SPLs) and Search-based Software Engineering (SBSE). She serves as assistant editor of the Journal of Software Engineering: Research and Development, and acts as peer reviewer for diverse international journals. She serves on the Program Committe of the main Brazilian Software Engineering conferences and other international ones, mainly related to Search-Based Software Engineering and software testing.

References

Al-Khiaty, M., Abdel-Aal, R., and Elish, M. (2017). Abductive network ensembles for improved prediction of future change-prone classes in object-oriented software. International Arab Journal of Information Technology (IAJIT), 14(6). Available online [link].

Aniche, M. (2015). Java code metrics calculator (CK). Available online [link].

Arcuri, A. and Briand, L. (2011). A practical guide for using statistical tests to assess randomized algorithms in software engineering. In Proceedings of the 33rd International Conference on Software Engineering, ICSE’11, page 1–10, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/1985793.1985795.

Arisholm, E., Briand, L., and Foyen, A. (2004). Dynamic coupling measurement for object-oriented software. IEEE Transactions on Software Engineering, 30(8):491-506. DOI: 10.1109/TSE.2004.41.

Bansiya, J. and Davis, C. (2002). A hierarchical model for object-oriented design quality assessment. IEEE Transactions on Software Engineering, 28(1):4-17. DOI: 10.1109/32.979986.

Batista, G. E. A. P. A., Prati, R. C., and Monard, M. C. (2004). A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor. Newsl., 6(1):20–29. DOI: 10.1145/1007730.1007735.

Bieman, J., Straw, G., Wang, H., Munger, P., and Alexander, R. (2003). Design patterns and change proneness: an examination of five evolving systems. In Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717), pages 40-49. DOI: 10.1109/METRIC.2003.1232454.

Breiman, L. (2001). Random forests. Machine learning, 45:5-32. DOI: 10.1023/A:1010933404324.

Brown, W., Malveau, R., Brown, W., McCormick, H. I., and Mowbray, T. (1999). AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis. Addison-Wesley. Book.

Brownlee, J. (2020). Data Preparation for Machine Learning: Data Cleaning, Feature Selection, and Data Transforms in Python. Machine Learning Mastery. Available online [link].

Caprio, F., Casazza, G., Penta, M., and Villano, U. (2001). Measuring and predicting the linux kernel evolution. In Proceedings of the International Workshop of Empirical Studies on Software Maintenance, pages 77-83. Available online [link].

Catolino, G. and Ferrucci, F. (2018). Ensemble techniques for software change prediction: A preliminary investigation. In 2018 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE), pages 25-30. IEEE. DOI: 10.1109/MALTESQUE.2018.8368455.

Catolino, G. and Ferrucci, F. (2019). An extensive evaluation of ensemble techniques for software change prediction. Journal of Software: Evolution and Process, 31(9):e2156. DOI: 10.1002/smr.2156.

Catolino, G., Palomba, F., De Lucia, A., Ferrucci, F., and Zaidman, A. (2017). Developer-related factors in change prediction: An empirical assessment. In 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC), pages 186-195. DOI: 10.1109/ICPC.2017.19.

Catolino, G., Palomba, F., De Lucia, A., Ferrucci, F., and Zaidman, A. (2018). Enhancing change prediction models using developer-related factors. Journal of Systems and Software, 143:14-28. DOI: 10.1016/j.jss.2018.05.003.

Catolino, G., Palomba, F., Fontana, F. A., De Lucia, A., Andy, Z., and Ferrucci, F. (2020). Improving change prediction models with code smell-related information. Empirical Software Engineering, 25:49–95. DOI: 10.1007/s10664-019-09739-0.

Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321-357. DOI: 10.1613/jair.953.

Chidamber, S. and Kemerer, C. (1994). A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6):476–493. DOI: 10.1109/32.295895.

Dietterich, T. G. (2002). Machine learning for sequential data: A review. In Structural, Syntactic, and Statistical Pattern Recognition, pages 15-30, Berlin, Heidelberg. Springer Berlin Heidelberg. DOI: 10.1007/3-540-70659-3_2.

Elish, M. O. and Al-Khiaty, M. A. (2013). A suite of metrics for quantifying historical changes to predict future change-prone classes in object-oriented software. Journal of Software: Evolution and Process, 25(5):407-437. DOI: 10.1002/smr.1549.

Eski, S. and Buzluca, F. (2011). An empirical study on object-oriented metrics and software evolution in order to reduce testing costs by predicting change-prone classes. In 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, pages 566-571. DOI: 10.1109/ICSTW.2011.43.

Fluri, B., Wursch, M., Pinzger, M., and Gall, H. (2007). Change Distilling:Tree differencing for fine-grained source code change extraction. IEEE Transactions on Software Engineering, 33(11):725-743. DOI: 10.1109/TSE.2007.70731.

Fowler, M. (1999). Refactoring – Improving the Design of Existing Code. Addison-Wesley. Available online [link].

Gall, H. C., Fluri, B., and Pinzger, M. (2009). Change analysis with Evolizer and ChangeDistiller. IEEE software, 26(1):26-33. DOI: 10.1109/MS.2009.6.

Giger, E., Pinzger, M., and Gall, H. C. (2012). Can we predict types of code changes? an empirical analysis. In 2012 9th IEEE working conference on Mining Software Repositories (MSR), pages 217-226. IEEE. DOI: 10.1109/MSR.2012.6224284.

Godara, D. and Singh, R. (2014). A review of studies on change proneness prediction in object oriented software. International Journal of Computer Applications, 105(3):0975–8887. Available online [link].

Han, J., Pei, J., and Tong, H. (2022). Data mining: concepts and techniques. Morgan kaufmann. DOI: 10.1016/C2009-0-61819-5.

He, H., Bai, Y., Garcia, E. A., and Li, S. (2008). Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pages 1322-1328. DOI: 10.1109/IJCNN.2008.4633969.

Ilyas, I. and Chu, X. (2019). Data Cleaning. ACM Collection II Series. Association for Computing Machinery. DOI: 10.1145/3310205.

Kaur, K. and Jain, S. (2017). Evaluation of machine learning approaches for change-proneness prediction using code smells. Advances in Intelligent Systems and Computing, 515. DOI: 10.1007/978-981-10-3153-3_56.

Khanna, M., Priya, S., and Mehra, D. (2021). Software change prediction with homogeneous ensemble learners on large scale open-source systems. In 17th IFIP International Conference on Open Source Systems (OSS), pages 68-86. Springer International Publishing. DOI: 10.1007/978-3-030-75251-4_7.

Khomh, F., Di Penta, M., and Gueheneuc, Y.-G. (2009). An exploratory study of the impact of code smells on software change-proneness. In 2009 16th Working Conference on Reverse Engineering, pages 75-84. DOI: 10.1109/WCRE.2009.28.

Khomh, F., Di Penta, M., Guéhéneuc, Y.-G., and Antonio, G. (2011). An exploratory study of the impact of antipatterns on class change- and fault-proneness. Empirical Software Engineering, 17:243-275. DOI: 10.1007/s10664-011-9171-y.

Koru, A. and Tian, J. (2005). Comparing high-change modules and modules with the highest measurement values in two large-scale open-source products. IEEE Transactions on Software Engineering, 31(8):625-642. DOI: 10.1109/TSE.2005.89.

Krüger, J., Li, Y., Lossev, K., Zhu, C., Chechik, M., Berger, T., and Rubin, J. (2024). A meta-study of software-change intentions. ACM Comput. Surv.. DOI: 10.1145/3661484.

Kruskal, W. H. and Wallis, W. A. (1952). Use of Ranks in One-Criterion Variance Analysis. Journal of the American Statistical Association, 47(260):583-621. DOI: 10.2307/2280779.

Lanza, M., Marinescu, R., and Ducasse, S. (2010). Object-Oriented Metrics in Practice: Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems. Springer. DOI: 10.1007/3-540-39538-5.

Lindvall, M. (1998). Are large C++ classes change-prone? An empirical investigation. Journal of Software: Practice and Experience, 28(15):1551-1558. DOI: 10.1002/(SICI)1097-024X(19981225)28:15<1551::AID-SPE212>3.0.CO;2-0.

Lu, H., Zhou, Y., X, B., Leung, H., and Chen, L. (2012). The ability of object-oriented metrics to predict change-proneness: a meta-analysis. Empirical Software Engineering, 17:200–242. DOI: 10.1007/s10664-011-9170-z.

Malhotra, R. and Bansal, A. (2015). Predicting change using software metrics: A review. In IEEE International Conference on Reliability, Infocom Technologies and Optimization (ICRITO), pages 1-6. DOI: 10.1109/ICRITO.2015.7359253.

Malhotra, R., Kapoor, R., Aggarwal, D., and Garg, P. (2021a). Comparative study of feature reduction techniques in software change prediction. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pages 18-28. DOI: 10.1109/MSR52588.2021.00015.

Malhotra, R., Kapoor, R., Aggarwal, D., and Garg, P. (2021b). Comparative study of feature reduction techniques in software change prediction. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pages 18-28. DOI: 10.1109/MSR52588.2021.00015.

Malhotra, R. and Khanna, M. (2013). Investigation of relationship between object-oriented metrics and change proneness. International Journal of Machine Learning and Cybernetics, 4:273–286. DOI: 10.1007/s13042-012-0095-7.

Malhotra, R. and Khanna, M. (2018a). Particle swarm optimization-based ensemble learning for software change prediction. Information and Software Technology, 102:65-84. DOI: 10.1016/j.infsof.2018.05.007.

Malhotra, R. and Khanna, M. (2018b). Prediction of change prone classes using evolution-based and object-oriented metrics. Journal of Intelligent & Fuzzy Systems, 34:1755-1766. DOI: 10.3233/JIFS-169468.

Malhotra, R. and Khanna, M. (2019). Software change prediction: A systematic review and future guidelines. e-Informatica Software Engineering Journal, 13(1):227-259. DOI: 10.37190/e-inf.

Malhotra, R. and Khanna, M. (2021). On the applicability of search-based algorithms for software change prediction. International Journal of Systems Assurance Engineering and Management. DOI: 10.1007/s13198-021-01099-7.

Malhotra, R. and Lata, K. (2020). An empirical study on predictability of software maintainability using imbalanced data. Software Quality Journal, 28. DOI: 10.1007/s11219-020-09525-y.

Mann, H. B. and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1):50-60. DOI: 10.1214/aoms/1177730491.

Martins, A. D. F., Melo, C. S., Monteiro, J. M., and de Castro Machado, J. (2020). Empirical study about class change proneness prediction using software metrics and code smells. In International Conference on Enterprise Information Systems (ICEIS), pages 140-147. Available online [link].

Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253):68-78. DOI: doi/10.2307/2280095.

Massoudi, M., Jain, N. K., and Bansal, P. (2021). Software defect prediction using dimensionality reduction and deep learning. In 2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV), pages 884-893. DOI: 10.1109/ICICV50876.2021.9388622.

Melo, C. S., da Cruz, M. M. L., Martins, A. D. F., da Silva Monteiro Filho, J. M., and de Castro Machado, J. (2020). Time-series approaches to change-prone class prediction problem. In International Conference on Enterprise Information Systems (ICEIS), pages 122-132. Available online [link].

Metz, C. E. (1978). Basic principles of roc analysis. Seminars in Nuclear Medicine, 8(4):283-298. DOI: 10.1016/S0001-2998(78)80014-2.

Mitchell, T. M. (1997). Machine learning, volume 1. McGraw-hill New York. Available online [link].

Nielsen, A. (2019). Practical Time Series Analysis: Prediction with Statistics and Machine Learning. O’Reilly Media, 1 edition. Available at: Book.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830. Available online [link].

Pritam, N., Khari, M., Hoang Son, L., Kumar, R., Jha, S., Priyadarshini, I., Abdel-Basset, M., and Viet Long, H. (2019). Assessment of code smell for predicting class change proneness using machine learning. IEEE Access, 7:37414-37425. DOI: 10.1109/ACCESS.2019.2905133.

Romano, D. and Pinzger, M. (2011). Using source code metrics to predict change-prone Java interfaces. In 2011 27th IEEE International Conference on Software Maintenance (ICSM), pages 303-312. DOI: 10.1109/ICSM.2011.6080797.

Silva, R. d. C., Farah, P. R., and Vergilio, S. R. (2022). Machine learning for change-prone class prediction: A history-based approach. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering, SBES '22, page 289–298, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3555228.3555249.

Silva, R. d. C., Farah, P. R., and Vergilio, S. R. (2024). Supplementary Material - On the use of Change History Data to Enhance Class Change-Proneness Prediction Models. Available online [link].

Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society: Series B (Methodological), 36(2):111-133. DOI: 10.1111/j.2517-6161.1974.tb00994.x.

Sultana, K. Z., Anu, V., and Chong, T.-Y. (2021). Using software metrics for predicting vulnerable classes and methods in java projects: A machine learning approach. Journal of Software: Evolution and Process, 33(3):e2303. DOI: 10.1002/smr.2303.

Tsantalis, N., Chatzigeorgiou, A., and Stephanides, G. (2005). Predicting the probability of change in object-oriented systems. IEEE Transactions on Software Engineering, 31(7):601-614. DOI: 10.1109/TSE.2005.83.

Tsoukalas, D., Kehagias, D., Siavvas, M., and Chatzigeorgiou, A. (2020). Technical debt forecasting: An empirical study on open-source repositories. Journal of Systems and Software, 170:110777. DOI: 10.1016/j.jss.2020.110777.

Vargha, A. and Delaney, H. D. (2000). A critique and improvement of the cl common language effect size statistics of mcgraw and wong. Journal of Educational and Behavioral Statistics, 25(2):101-132. DOI: 10.3102/10769986025002101.

Witten, I. H., Frank, E., Hall, M. A., Pal, C. J., and Data, M. (2005). Practical machine learning tools and techniques, volume 2. Available online [link].

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2000). Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers. DOI: 10.1007/978-3-642-29044-2.

Zhou, Y., Leung, H., and Xu, B. (2009). Examining the potentially confounding effect of class size on the associations between object-oriented metrics and change-proneness. IEEE Transactions on Software Engineering, 35(5):607-623. DOI: 10.1109/TSE.2009.32.

Zhu, X., He, Y., Cheng, L., Jia, X., and Zhu, L. (2018). Software change-proneness prediction through combination of bagging and resampling methods. Journal of Software: Evolution and Process, 30(12):e2111. DOI: 10.1002/smr.2111.