Evaluating Thresholds for Object-Oriented Software Metrics
DOI:
https://doi.org/10.5753/jbcs.2024.3373Keywords:
Object-Oriented Software Metrics, Measurement, Thresholds Catalog, Internal Software Quality, Software EngineeringAbstract
Software metrics measure quantifiable or countable software characteristics. Researchers may apply them to provide better product understanding, evaluate the process effectiveness, and improve the software quality. A threshold is a value that aids the proper interpretation of software measurements; it indicates whether or not a given value represents a quality risk. Thresholds are unknown for most software metrics, inhibiting their use in a software quality assessment process. In a previous paper, we proposed a catalog with 18 object-oriented software metrics thresholds, providing a preliminary case study in proprietary software to validate them. This article evaluates these thresholds more deeply, considering significant aspects. We show a new example of threshold derivation, discussing it qualitatively. We explain these software metrics and discuss their threshold values, presenting each one’s application level, definition, formula, and implications for the software design. We conduct a study with two software systems to evaluate the capacity of our thresholds to identify software quality enhancement after a restructuring process. We assess these thresholds using two case studies, comparing the evaluation provided by the thresholds with the qualitative analysis given by manual inspections. The study results indicate that the thresholds may lead to few false-positive and false-negative occurrences, i.e., the thresholds provide a proper quantitative assessment of software quality. This study contributes with empirical evidence that the metrics’ thresholds proposed in our previous work provide a proper interpretation of software metrics and, hence, may aid the application of software metrics in practice.
Downloads
References
Abílio, R., Padilha, J., Figueiredo, E., and Costa, H. (2015). Detecting Code Smells in Software Product Lines - An Exploratory Study. In 12th International Conference on Information Technology - New Generations, pages 433-438, New York, NY, USA. DOI: 10.1109/ITNG.2015.76.
Al Dallal, J. (2011). Improving the applicability of object-oriented class cohesion metrics. Information and Software Technology, 53:914-928. DOI: 10.1016/j.infsof.2011.03.004.
Al Dallal, J. (2012). Constructing models for predicting extract subclass refactoring opportunities using object-oriented quality metrics. Information and Software Technology, 54:1125–1141. DOI: 10.1016/j.infsof.2012.04.004.
Al Dallal, J. and Briand, L. (2010). An object-oriented high-level design-based class cohesion metric. Information and Software Technology, 52:1346-1361. DOI: 10.1016/j.infsof.2010.08.006.
Alshayeb, M. (2009). Refactoring effect on cohesion metrics. In Computing, Engineering and Information, ICC, pages 3-7. DOI: 10.1109/ICC.2009.12.
Alves, T., Ypma, C., and Visser, M. (2010). Deriving metric thresholds from benchmark data. In Proc. of the IEEE Int. Conference on Software Maintenance, pages 1-10. DOI: 10.1109/ICSM.2010.5609747.
Aniche, M., Treude, C., Zaidman, A., Van Deursen, A., and Gerosa, M. A. (2016). SATT: Tailoring Code Metric Thresholds for Different Software Architectures. In IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM), pages 41-50. DOI: 10.1109/SCAM.2016.19.
Anquetil, N. and Laval, J. (2011). Legacy software restructuring: Analyzing a concrete case. In 15th European Conf. on Soft. Maint. and Reengineering, pages 279-286. DOI: 10.1109/CSMR.2011.34.
Arar, O. and Ayan, K. (2016). Deriving Thresholds of Software Metrics to Predict Faults on Open Source Software: Replicated Case Studies. Expert Systems with Applications, 61:106-121. DOI: 10.1016/j.eswa.2016.05.018.
Baxter, G., Frean, M., Noble, J., Rickerby, M., Smith, H., Visser, M., Melton, H., and Tempero, E. (2006). Understanding the shape of java software. In OOPSLA, pages 397-412. ACM. DOI: 10.1145/1167473.1167507.
Bender, R. (1999). Quantitative risk assessment in epidemiological studies investigating threshold effects. Biometrical Journal, 41:305-319. DOI: 10.1002/(SICI)1521-4036(199906)41:3<305::AID-BIMJ305>3.0.CO;2-Y.
Benlarbi, S., Khaled, E., Nishith, G., and Shesh, R. (2000). Thresholds for Object-Oriented Measures. In Proc. 11th International Symposium on Software Reliability Engineering, pages 24-38. IEEE. DOI: 10.1109/ISSRE.2000.885858.
Beranič, T. and Heričko, M. (2017). Approaches for Software Metrics Threshold Derivation: A Preliminary Review. In Proceedings of the Sixth Workshop on Software Quality Analysis, Monitoring, Improvement, and Application, pages 1-8. Available online [link].
Beranič, T. and Heričko, M. (2019). Comparison of systematically derived software metrics thresholds for object-oriented programming languages. Computer Science and Information Systems, 17(1):181-203. Available online [link].
Bigonha, M., Ferreira, K., Souza, P., Sousa, B., Januário, M., and Lima, D. (2019). The Usefulness of Software Metric Thresholds for Detection of Bad Smells and Fault Prediction. Information and Software Technology, 115:79-92. DOI: 10.1016/j.infsof.2019.08.005.
Bloch, J. (2008). Effective Java. 2nd Ed.Prentice Hall. Book.
Boucher, A. and Badri, M. (2016). Using Software Metrics Thresholds to Predict Fault-Prone Classes in Object-Oriented Software. In 4th Intl Conf on Applied Computing and Info. Technology/3rd Int. Conf on Computational Science/Int. and Applied Inf./1st Intl Conf on Big Data, Cloud Computing, Data Sc. & Eng., pages 169-176. DOI: 10.1109/ACIT-CSII-BCD.2016.042.
Boucher, A. and Mourad, B. (2018). Software metrics thresholds calculation techniques to predict faultproneness: An empirical comparison. Inf. and Soft. Technology, 96:38-67. DOI: 10.1016/j.infsof.2017.11.005.
Bouwers, E., V. J. and Van, D. A. (2012). Getting what you measure. Queue ACM, 10(5):54-59. DOI: 10.1145/2209249.2209266.
Chhikara, A., Chhillar, R. S., and Khatri, S. (2011). Evaluating the impact of different types of inheritance on the object oriented software metrics. Int. Journal of Enterprise Comp. and Business Syst., 1(2). Available online [link].
Chidamber, S. R. and Kemerer, C. F. (1994). A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6):476-493. DOI: 10.1109/32.295895.
Christensen and Henrik, B. (2004). Frameworks: Putting design patterns into perspective. In Proc. SIGCSE Conf. on Innovation and technology in Comp. Science Education, pages 142-145. DOI: 10.1145/1007996.1008035.
E. Tempero, C. Anslow, J. D. T. H. J. L. M. L. H. M. and Noble, J. (2010). The qualitas corpus: A curated collection of java code for empirical studies. In Proceedings of the 17th Asia Pacific Software Engineering Conference, pages 336-345, Sidney, Australia. DOI: 10.1109/APSEC.2010.46.
Fan, Z., Feng, Z., Xue, X., Chen, S., and Wu, H. (2020). Ecosystem evolution analysis and trend prediction of projects in Android application framework. In Proc. 35th IEEE/ACM Int. Conf. on Automated Soft.Eng. Workshops, pages 67-72. DOI: 10.1145/3417113.3422185.
Feng, Q., Cai, Y., Kazman, R., Cui, D., Liu, T., and Fang, H. (2019). Active Hotspot: An Issue-Oriented Model to Monitor Software Evolution and Degradation. In 34th Int. Conf. on Automated Soft. Engineering, page 986–997, New York, NY, USA. IEEE/ACM. DOI: 10.1109/ASE.2019.00095.
Fenton, N. (1994). Software Measurement: A Necessary Scientific Basis. Soft. Eng., IEEE Transactions on, 20:199 - 206. DOI: 10.1109/32.268921.
Ferreira, K., Bigonha, M., Bigonha, R., Mendes, L., and Almeida, H. (2012). Identifying thresholds for object-oriented software metrics. Journal of Systems and Software, 85(2):244-257. DOI: 10.1016/j.jss.2011.05.044.
Filó, T., Bigonha, M., and Ferreira, K. (2014). Raftool-ferramenta de filtragem de métodos, classes e pacotes com medições incomuns de métricas de software. In Proc. WAMPS-SOFTEX, pages 42-48. written in portuguese.
Filó, T., Bigonha, M., and Ferreira, K. (2014). Statistical dataset on software metrics in object-oriented systems. ACM SIGSOFT Software Engineering Notes, 39(5):1-6. DOI: 10.1145/2659118.2659130.
Filó, T., Bigonha, M., and Ferreira, K. (2015). A Catalog of Thresholds for Object-Oriented Software Metrics. In Proc. of the 1st SOFTENG, pages 48-55. Available online [link].
Fontana, F. A., Ferme, V., Zanoni, M., and Yamashita, A. (2015). Automatic Metric Thresholds Derivation for Code Smell Detection. In IEEE/ACM 6th International Workshop on Emerging Trends in Software Metrics, pages 44-53, Florence, Italy. DOI: 10.1109/WETSoM.2015.14.
Fowler, M., Beck, K., Brant, J., Opdyke, W., and Roberts, D. (1999). Refactoring: Improving the Design of Existing Code. Addison-Wesley. Book.
Gil, J. A. and Lalouche, G. (2016). When do software complexity metrics mean nothing? – when examined out of context. The Journal of Object Technology, 15:2-1. Available online [link].
Hegedüs, P., Bán, D., Ferenc, R., and Gyimóthy, T. (2012). Myth or Reality? Analyzing the Effect of Design Patterns on Software Maintainability. In Computer Applications for Software Engineering, Disaster Recovery, and Business Continuity Communications in Computer and Information Science, volume 340 Springer, pages 138-145. DOI: 10.1007/978-3-642-35267-6_18.
Herbold, S., Grabowski, J., and Waack, S. (2011). Calculation and optimization of thresholds for sets of software metrics. Empirical Software Engineering, 16:812-841. DOI: 10.1007/s10664-011-9162-z.
Hevery, M. (2008). it Google testing: Static Methods are death to testability. Available online [link].
Hussain, S., Keung, J., Khan, A. A., and Bennin, K. E. (2016). Detection of Fault-Prone Classes Using Logistic Regression Based Object-Oriented Metrics Thresholds. In IEEE Int. Conf. on Soft. Quality, Reliability and Security Companion (QRS-C), pages 93-100. DOI: 10.1109/QRS-C.2016.16.
Kaur, S., Singh, S., and Kaur, H. (2013). A Quantitative Investigation Of Software Metrics Threshold Values At Acceptable Risk Level. Int. Journal of Engineering Research & Technology, 2(3):1-7. Available online [link].
Kitchenham, B. (2010). Whats up with software metrics? A preliminary mapping study. JSS, 83(1):37-51. DOI: 10.1016/j.jss.2009.06.041.
Lanza, M. and Marinescu, R. (2010). Object-Oriented Metrics in Practice: using software metrics to characterize, evaluate, and improve the design of object-oriented systems. Springer Publishing Company. Book.
Lavazza, L. and Morasca, S. (2016). An empirical evaluation of distribution-based thresholds for internal software measures. In Proceedings of the 12th International Conference on Predictive Models and Data Analytics in Software Engineering, page 6. IEEE. DOI: 10.1145/2972958.2972965.
Li, W. (1998). Another Metric Suite for Object Oriented Programming. J. of System and Software, 44:155-162. DOI: 10.1016/S0164-1212(98)10052-3.
Li, W. and Henry, S. (1993). Object-oriented that predict maintainability. J. of System and Software, 23:111-122. DOI: 10.1016/0164-1212(93)90077-B.
Lochmann, K. (2012). A benchmarking-inspired approach to determine threshold values for metrics. SIGSOFT Softw. Eng. Notes, 37(6). DOI: 10.1145/2382756.2382782.
Lopez, M. and Habra, N. (2005). Relevance of the cyclomatic complexity threshold for the java programming language. In Proceedings of the 2nd Software Measurement European Forum, pages 195-202. Available online [link].
Lorenz, M. and Kidd, J. (1994). Object-Oriented Software Metrics: A Practical Guide. Prentice-Hall, USA. Book.
Malhotra, R. and Bansal, A. (2015). Fault prediction considering threshold effects of object-oriented metrics. Expert Systems, 32:203-219. DOI: 10.1111/exsy.12078.
Malhotra, R. and Bansal, A. (2017). Identifying threshold values of an open source software using Receiver Operating Characteristics curve (ROC). Journal of Information and Optimization Sciences, 38:39-69. DOI: 10.1080/02522667.2015.1135592.
Martin, R. (1994). OO Design Quality Metrics - An Analysis of Dependencies. In Proc. of Works. on Pragmatic and Theoretical Directions in Object-Oriented Soft. Metrics. Available online [link].
Martin, R. C. (2003). Agile Software Development: Principles, Patterns, and Practice. Prentice Hall, Upper Saddle River, New Jersy, USA. Book.
McCabe, T. (1976). A complexity measure. Software Engineering IEEE Transactions, 2(4):308-320. DOI: 10.1109/TSE.1976.233837.
Mei, Y., Rong, Y., Liu, S., Guo, Z., Yang, Y., Lu, H., Tang, Y., and Zhou, Y. (2023). Deriving Thresholds of Object-Oriented Metrics to Predict Defect-Proneness of Classes: A Large-Scale Meta-Analysis. International Journal of Software Engineering and Knowledge Engineering, 33:651-695. DOI: 10.1142/S0218194023500110.
Metrics (2014). Eclipse Metrics plugin 1.3.8. URL:. Available online [link]. Accessed in: 2014 12-30.
Mishra Alok, S. R., Cagatay, C., and Akhan, A. (2021). Techniques for Calculating Software Product Metrics Threshold Values: A Systematic Mapping Study. Applied Sciences, 11(23). DOI: 10.3390/app112311377.
Mohammed, A., Mohammed, A., and Lahouari, G. (2019). Threshold Extraction Framework for Software Metrics. J. of Comp. Sci. and Technology, 34(5):1063-1078. DOI: 10.1007/s11390-019-1960-6.
Mohović, M., Mausa, G., and Galinac Grbac, T. (2018). Using Threshold Derivation of Software Metrics for Building Classifiers in Defect Prediction. In Proc. of the SQAMIA 2018: 7th Works. of Soft. Quality, Analysis, Monitoring, Improvement, and Applications. Also published online by CEUR Works. Proc., pages 11-1 - 11-9, Novi Sad, Serbia. Available online [link].
Morasca, S. and Lavazza, L. (2016). Slope-based fault-proneness thresholds for software engineering measures. In Proc. of the 20th Int. Conf. on Evaluation and Assessment in Soft. Engineering, pages 1-10. DOI: 10.1145/2915970.2915997.
Morasca, S. and Lavazza, L. (2017). Risk-averse slope-based thresholds: Definition and empirical evaluation. Information and Software Technology, 89:37 - 63. DOI: 10.1016/j.infsof.2017.03.005.
Mori, A. (2018). Design and Evaluation of a Method to Derive Domain Metric Thresholds. Master's thesis, UFMG, Belo Horizonte, Minas Gerais. Available online [link].
Oliveira, P., Lima, F. P., Valente, M. T. O., and Serebrenik, A. (2014a). RTTool: A Tool for Extracting Relative Thresholds for Source Code Metrics. In 2014 IEEE Int. Conf. on Soft. Maint. and Evolution, pages 629-632. DOI: 10.1109/ICSME.2014.112.
Oliveira, P., Valente, M., and Lima, F. (2014b). Extracting relative thresholds for source code metrics. In Conf. on Soft. Evolution Week, pages 254-263, Antwerp, Belgium. CSMR-WCRE IEEE. DOI: 10.1109/CSMR-WCRE.2014.6747177.
Oracle (2014). Javase technical documentation. Available online [link]. Accessed: 2014-12-30.
Padhy, N., Panigrahi, R., and Neeraja, K. (2021). Threshold estimation from software metrics by using evolutionary techniques and its proposed algorithms, models. Evolutionary Intelligence - Special Issue, 14:315-329. DOI: 10.1007/s12065-019-00201-0.
Paige, R. and Meyer, B. (2008). Objects, Components, Models, and Patterns. In Proc. 46th Int. Conference, TOOLS EUROPE. DOI: 10.1007/978-3-540-69824-1.
Radjenović, D., Hericko, M., Torkar, R., and Živkovič, A. (2013). Software fault prediction metrics: A systematic literature review. Information and Software Technology, 55:1397–1418. DOI: 10.1016/j.infsof.2013.02.009.
Riehle, D. (2000). Framework design: A role modeling approach. Software Technik-Trends, 20(4). Available online [link].
Rosenberg, L., Ruth, S., and Gallo, A. (1999). Risk-based Object Oriented Testing. In Proc.24th S.E. Workshop, pages 1-6. NASA, S.E.Lab. Available online [link].
Shatnawi, R. (2010). A Quantitative Investigation of the Acceptable Risk levels of Object-Oriented Metrics in Open-Source Systems. IEEE Transactions on Software Engineering, 36:216-225. DOI: 10.1109/TSE.2010.9.
Shatnawi, R. (2015). Deriving metrics threshold using log transformation. J. of Soft. and Evol. Process, 27:95-113. DOI: 10.1002/smr.1702.
Shatnawi, R. (2018). Identifying Threshold Values of Change-Prone Modules. In International Conference on E-business and Mobile Commerce, pages 39-43, Chengdu, China. ACM. DOI: 10.1145/3230467.3230477.
Shatnawi, R. and Althebyan, Q. (2013). An Empirical Study of the Effect of Power Law Distribution on the Interpretation of OO Metrics. ISRN Software Engineering, 2013:18. DOI: 10.1155/2013/198937.
Shatnawi, R., Li, W., Swain, J., and T., N. (2009). Finding software metrics threshold values using ROC curves. Journal of Software Maintenance and Evolution: Research and Practice, 22(1):1-16. DOI: 10.1002/smr.404.
Singh, S. and Kaur, K. S. (2014). Object oriented software metrics threshold values at acceptable risk level. Int. Journal CSIT, 2:191-205. DOI: 10.1007/s40012-014-0057-1.
Sodiya, A. S., Aborisade, O., and Ikuomola, A. (2012). A survivability model for object-oriented software systems. In Fourth International Conference on Computational Aspects of Social Networks (CASoN), pages 283-290. DOI: 10.1109/CASoN.2012.6412416.
Sommerville, I. (2012). Software Engineering. Pearson, 9th edition. Book.
Sousa, B. L., Souza, P. P., Fernandes, E., Ferreira, K., and Bigonha, M. A. S. (2017). Findsmells: flexible composition of bad smell detection strategies. In ICPC, editor, Proceedings of the 25th International Conference on Program Comprehension, pages 360-363. DOI: 10.1109/ICPC.2017.8.
Souza, P. P., Sousa, B. L., Ferreira, K. A. M., and Bigonha, M. A. S. (2017). Applying software metric thresholds for detection of bad smells. In Proc. of the 11th SBCARS, pages 6-1-6-10. ACM. DOI: 10.1145/3132498.3134268.
Stojkovski, M. (2017). Thresholds for Software Quality Metrics in Open Source Android Projects. Norwegian Univ. of Science and Technology Comp. Sc. Department. Available online [link].
Sultan, A. (2021). Predicting Relative Thresholds for Object Oriented Metrics. In IEEE/ACM International Conf. on Technical Debt (TechDebt), pages 55-63. IEEE. DOI: 10.1109/TechDebt52882.2021.00015.
Terra, R., Miranda, L. F., Valente, M. T., and Bigonha, R. (2013). Qualitas.Class corpus: A compiled version of the qualitas corpus. Soft. Eng. Notes, 38(5):1-4. DOI: 10.1145/2507288.2507314.
Vale, G., A. D., Figueiredo, E., and A., G. (2015). Defining metric thresholds for software product lines: A comparative study. In Proc. of the 19th Int. Conf. on Soft. Product Line, pages 175-185. DOI: 10.1145/2791060.2791078.
Vale, G., Fernandes, E., and Figueiredo, E. (2019). On the proposal and evaluation of a benchmark-based threshold derivation method. Software Quality Journal, 27(1):275-306. DOI: 10.1007/s11219-018-9405-y.
Vale, G. and Figueiredo, G. (2015). A method to derive metric thresholds for software product lines. In 29th Brazilian Symp. on Soft. Eng.(CBSOFT), pages 110-119. DOI: 10.1109/SBES.2015.9.
Veado, L., Vale, G., Fernandes, E., and Figueiredo, E. (2016). TDTool: threshold derivation tool. In Proc. 20th Inter. Conference on Evaluation and Assessment in Software Engineering, pages 1-5. DOI: 10.1145/2915970.2916014.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Tarcísio G. S. Filó, Mariza A. S. Bigonha, Kecia A. M. Ferreira
This work is licensed under a Creative Commons Attribution 4.0 International License.
