LLMs on code smell detection and refactoring: a systematic literature review

Authors

DOI:

https://doi.org/10.5753/jserd.2026.7984

Keywords:

Large Language Models, Code Smells, Refactoring, SLR

Abstract

Large Language Models (LLMs) are increasingly being applied to software maintenance tasks such as code smell detection and code refactoring, yet the evidence on their use remains fragmented across heterogeneous tasks, evaluation settings, and tool architectures. In particular, there is still limited consolidated knowledge about the prompt-based use of frozen-parameter LLMs for these activities. This paper presents a systematic literature review of how such models have been applied and evaluated in code smell detection and refactoring. Following a predefined protocol, we combined database searches and snowballing and used source-grounded AI assistance during screening under human oversight, yielding a final corpus of 65 studies. The review shows that this is a recent and rapidly expanding research area, concentrated around a small number of model families, programming languages, and canonical smells and refactorings. Across the reviewed studies, standalone prompting was generally less reliable in structurally demanding scenarios, whereas hybrid and repository-aware approaches were more often associated with stronger outcomes. Retrieval-augmented generation, validation layers, and iterative workflows repeatedly emerged as key mechanisms for improving reliability. Beyond technical performance, the evidence also indicates that practical usefulness depends on transparency, iterative interaction, and visible developer control. Overall, this review consolidates a still-dispersed body of work and provides a focused basis for future research and for the design of context-aware, validated, and human-supervised LLM-based maintenance tools.

Downloads

Download data is not yet available.

References

Abid, C., Alizadeh, V., Kessentini, M., Ferreira, T. N., and Dig, D. (2020). 30 years of software refactoring research: a systematic literature review. ArXiv, abs/2007.02194:1–23.

Akalanka, I., Silva, S. D., Ganeshalingam, M., Abeykoon, A., Wijendra, D., and Krishara, J. (2025). AI powered integrated code repository analyzer for efficient developer workflow. In 8th International Research Conference on Smart Computing and Systems Engineering (SCSE). Institute of Electrical and Electronics Engineers Inc.

Alomar, E. A., Venkatakrishnan, A., Mkaouer, M. W., Newman, C. D., and Ouni, A. (2024). How to refactor this code? An exploratory study on developer-ChatGPT refactoring conversations. In 21st IEEE/ACM International Conference on Mining Software Repositories (MSR), pages 202–206. Institute of Electrical and Electronics Engineers Inc.

Alomari, N., Redah, M., Ashraf, A., and Alshayeb, M. (2026). Using LLMs to enhance code quality: A systematic literature review. Information and Software Technology, 190:107960.

Bellur, A. and Batole, F. (2025). IDE native, foundation model based agents for software refactoring. In 2nd IEEE/ACM IDE Workshop (IDE), pages 42–45. Institute of Electrical and Electronics Engineers Inc.

Bellur, A., Batole, F., Ullah, M. R., Dilhara, M., Zharov, Y., Bryksin, T., Ishikawa, K., Chen, H., Morimoto, M., Hosomi, T., Nguyen, T. N., Rajan, H., Tsantalis, N., and Dig, D. (2025). Together we are better: LLM, IDE and semantic embedding to assist move method refactoring. In 41st IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 430–442. Institute of Electrical and Electronics Engineers Inc.

Borg, M. (2025). Trust calibration in IDEs: Paving the way for widespread adoption of AI refactoring. In 2nd IEEE/ACM IDE Workshop (IDE), pages 37–41. Institute of Electrical and Electronics Engineers Inc.

Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., and Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80(4):571–583.

Cordeiro, J., Noei, S., and Zou, Y. (2024). An empirical study on the code refactoring capability of large language models. arXiv preprint arXiv:2411.02320.

Cordeiro, J., Noei, S., and Zou, Y. (2026). An empirical study on the code refactoring capability of large language models. ACM Trans. Softw. Eng. Methodol.

Das, J. K., Mondal, S., and Roy, C. K. (2025). Why do developers engage with ChatGPT in issue-tracker? Investigating usage and reliance on ChatGPT-generated code. In IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 68–79. Institute of Electrical and Electronics Engineers Inc.

Deo, S., Hinge, D., Chavan, O. S., Wang, Y. O., and Mkaouer, M. W. (2024). Analyzing developer-ChatGPT conversations for software refactoring: An exploratory study. In 21st IEEE/ACM International Conference on Mining Software Repositories (MSR), pages 207–211. Institute of Electrical and Electronics Engineers Inc.

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics.

Dilhara, M., Bellur, A., Bryksin, T., and Dig, D. (2024). Unprecedented code change automation: The fusion of LLMs and transformation by example. Proceedings of the ACM on Software Engineering, 1:631–653.

Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., and Zhang, J. M. (2023). Large language models for software engineering: Survey and open problems. In 45th IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), pages 31–53. Institute of Electrical and Electronics Engineers Inc.

Felizardo, K. R., Deizepe, A., Coutinho, D., Gomes, G., Meireles, M., Gerosa, M., and Steinmacher, I. (2025). On the difficulties of conducting and replicating systematic literature reviews studies using LLMs in software engineering. In 2nd IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE), pages 20–23.

Felizardo, K. R., Lima, M. S., Deizepe, A., Conte, T. U., and Steinmacher, I. (2024). ChatGPT application in systematic literature reviews in software engineering: an evaluation of its accuracy to support the selection activity. In 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 25–36.

Fowler, M. and Beck, K. (1999). Refactoring: improving the design of existing code. Addison-Wesley, 1 edition.

Guo, Q., Xie, X., Liu, S., Hu, M., Li, X., and Bu, L. (2025). Intention is all you need: Refining your code from your intention.

Habib, M. B., Jannat, A. T., and Azhar, T. (2025). Transformer-based multi-label classification for code smell detection in software engineering. In 2025 6th International Conference for Emerging Technology (INCET), pages 1–6.

Jin, H., Sun, Z., and Chen, H. (2024). RGD: Multi-LLM based agent debugger via refinement and generation guidance. In 2024 IEEE International Conference on Agents (ICA), pages 136–141.

Just, R., Jalali, D., and Ernst, M. D. (2014). Defects4J: a database of existing faults to enable controlled testing studies for Java programs. pages 437–440.

Kitchenham, B. and Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical report, Keele University.

Li, J., Gao, Y., Yang, Y., Bai, Y., Zhou, X., Li, Y., Sun, H., Liu, Y., Si, X., Ye, Y., Wu, Y., Lin, Y., Xu, B., Ren, B., Feng, C., and Huang, H. (2025). Fundamental capabilities and applications of large language models: A survey. ACM Comput. Surv., 58(2).

Martinez, S., Xu, L., Elnaggar, M., and Alomar, E. A. (2026). Software refactoring research with large language models: A systematic literature review. Journal of Systems and Software, 235:112762.

Melo, R., Simões, P., Gheyi, R., d’Amorim, M., Ribeiro, M., Soares, G., Almeida, E., and Soares, E. (2026). Agentic LMs: Hunting down test smells. IEEE Software, 43(1):32–40.

Menolli, A., Strik, B., and Rodrigues, L. (2024). Teaching refactoring to improve code quality with ChatGPT: An experience report in undergraduate lessons. In ACM International Conference Proceeding Series, pages 563–574. Association for Computing Machinery.

Mesbah, D., El Madhoun, N., Agha, K., and Chalouati, H. (2025). Leveraging Prompt-Based Large Language Models for Code Smell Detection: A Comparative Study on the MLCQ Dataset, pages 444–454.

Midolo, A. and Di Penta, M. (2025). Automated refactoring of non-idiomatic Python code: A differentiated replication with LLMs. In 33rd IEEE International Conference on Program Comprehension (ICPC), pages 113–123. IEEE Computer Society.

Midolo, A. and Tramontana, E. (2025). Refactoring loops in the era of LLMs: A comprehensive study. Future Internet, 17(9).

Miño, J., Andrade, R., Torres, J., and Chicaiza, K. (2024). Leveraging generative artificial intelligence for software antipattern detection. In Communications in Computer and Information Science, volume 2102 CCIS, pages 138–149. Springer Science and Business Media Deutschland GmbH.

Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., and Mian, A. (2025). A comprehensive overview of large language models. ACM Trans. Intell. Syst. Technol., 16(5).

Oliveira, E. C., Keuning, H., and Jeuring, J. (2025). Uncovering behavioral patterns in student–LLM conversations during code refactoring tasks. In 25th Koli Calling International Conference on Computing Education Research, Koli Calling ’25, New York, NY, USA. Association for Computing Machinery.

Pandini, G., Martini, A., Videsjorden, A. N., and Fontana, F. A. (2025). An exploratory study on architectural smell refactoring using large languages models. In 22nd IEEE International Conference on Software Architecture (ICSA-C), pages 462–471. Institute of Electrical and Electronics Engineers Inc.

Pereira, G., Vegi, L. F. M., and Di-Iorio, V. (2025). Building refactoring tools: Insights from RefactorEx. In 1st Workshop on Software Engineering for Functional Programming (SE4FP), pages 9–14.

Pomian, D., Bellur, A., Dilhara, M., Kurbatova, Z., Bogomolov, E., Sokolov, A., Bryksin, T., and Dig, D. (2024). EM-Assist: Safe automated ExtractMethod refactoring with LLMs. In 32nd ACM International Conference on the Foundations of Software Engineering (FSE), pages 582–586. Association for Computing Machinery, Inc.

Radford, A. and Narasimhan, K. (2018). Improving language understanding by generative pre-training.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language models are unsupervised multitask learners.

Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., and Chadha, A. (2025). A systematic survey of prompt engineering in large language models: Techniques and applications.

Shor, R., Greene, E. A., Sumberg, L., and Weingrad, A. B. (2025). AI tools in academia: Evaluating NotebookLM as a tool for conducting literature reviews. Psychiatry, 0(0):1–10. PMID: 40875632.

Silva, L. L., Silva, J. R. D., Montandon, J. E., Andrade, M., and Valente, M. T. (2024). Detecting code smells using ChatGPT: Initial insights. In 18th International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 400–406. IEEE Computer Society.

Sobrinho, E., De Lucia, A., and Maia, M. (2021). A systematic literature review on bad smells—5 W’s: Which, When, What, Who, Where. IEEE Transactions on Software Engineering, 47(1):17–66.

Sonkin, V. and Tudose, C. (2025). Beyond snippet assistance: A workflow-centric framework for end-to-end AI-driven code generation. Computers, 14(3).

Sun, S. (2024). Enhancing software design and developer experience via LLMs. In 39th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 2498–2501.

Tsantalis, N., Ketkar, A., and Dig, D. (2022). RefactoringMiner 2.0. IEEE Transactions on Software Engineering, 48(3):930–950.

van Deursen, A., Klint, P., and Visser, J. (2000). Domain-specific languages: an annotated bibliography. SIGPLAN Not., 35(6):26–36.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In 31st International Conference on Neural Information Processing Systems (NIPS), NIPS’17, pages 6000–6010, Red Hook, NY, USA. Curran Associates Inc.

Vo, Q.-H., Dao, H., and Fukuda, K. (2025). Harnessing the power of LLMs for code smell detection in Terraform infrastructure as code. In IEEE 49th Annual Computers, Software, and Applications Conference (COMPSAC), pages 533–542.

Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In 18th International Conference on Evaluation and Assessment in Software Engineering (EASE), pages 1–10.

Wu, D., Mu, F., Shi, L., Guo, Z., Liu, K., Zhuang, W., Zhong, Y., and Zhang, L. (2024). iSMELL: Assembling LLMs with expert toolsets for code smell detection and refactoring. In 39th ACM/IEEE International Conference on Automated Software Engineering (ASE), pages 1345–1357. Association for Computing Machinery, Inc.

Xu, Y. (2025). MUARF: Leveraging multi-agent workflows for automated code refactoring. In 47th International Conference on Software Engineering (ICSE), pages 226–227. IEEE Computer Society.

Zhu, X., Dong, X., Yu, W., Liang, H., and Kong, B. (2025). Refactored Maskformer: Refactor localization and classification for improved universal image segmentation. Displays, 87:102981.

Downloads

Published

2026-11-10

How to Cite

Dias, M., & Vegi, L. (2026). LLMs on code smell detection and refactoring: a systematic literature review. Journal of Software Engineering Research and Development, 14(1), 633–659. https://doi.org/10.5753/jserd.2026.7984

Issue

Section

Review