Model interpretation using improved local regression with variable importance
DOI:
https://doi.org/10.5753/jbcs.2026.6073Keywords:
Explainable AI (XAI), Interpretable Machine Learning, Local interpretation, Model AgnosticAbstract
One of the main limitations for the trust in the use of machine learning models is the understanding of how they produce their predictions. This limitation is related to an increasing demand for transparency in decision-making. Interpretability refers to how well a user can understand the model’s decision-making process without necessarily knowing its internal mechanisms. Several interpretability methods have emerged in the last years, such as LIME and Shapley values, which are valued for their flexibility, intuitive appeal, and strong theoretical foundations. While these methods have significantly contributed to better model transparency, they face challenges in several model deployment scenarios, such as handling irrelevant features or maintaining stability under small data perturbations. This article introduces two new agnostic interpretability methods, namely VarImp and SupClus, which overcome these issues by using local regressions fits with a weighted distance that takes into account variable importance. Whereas VarImp generates interpretations for each instance and can be applied to datasets with more complex relationships, SupClus interprets data clusters of instances with similar interpretations and can be applied to simpler datasets where data clusters can be found. In this paper, we compare these proposed methods with state-of-the-art methods and show that the proposed methods generate either equal or better interpretations, according to several proposed quantitative metrics (mean square error of coefficients, effect correlation, prediction correlation and ICE effect correlation), particularly in high-dimensional problems with irrelevant features and when the relationship between features and target is non-linear.
Downloads
References
Agarwal, R., Melnick, L., Frosst, N., Zhang, X., Lengerich, B., Caruana, R., and Hinton, G. E. (2021). Neural additive models: interpretable machine learning with neural nets. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA. Curran Associates Inc.. DOI: 10.5555/3540261.3540620.
Alvarez-Melis, D. and Jaakkola, T. S. (2018). On the robustness of interpretability methods. arXiv preprint arXiv:1806.08049. DOI: 10.48550/arXiv.1806.08049.
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., and Herrera, F. (2020). Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82-115. DOI: 10.1016/j.inffus.2019.12.012.
Botari, T., Hvilshøj, F., Izbicki, R., and de Carvalho, A. C. (2020). Melime: meaningful local explanation for machine learning models. arXiv preprint arXiv:2009.05818. DOI: 10.48550/arXiv.2009.05818.
Breiman, L. (2001). Random forests. Machine learning, 45(1):5-32. DOI: 10.1023/A:1010933404324.
Coscrato, V., Inácio, M. H., Botari, T., and Izbicki, R. (2023). Nls: An accurate and yet easy-to-interpret prediction method. Neural Networks, 162:117-130. DOI: 10.1016/j.neunet.2023.02.043.
Goldstein, A., Kapelner, A., Bleich, J., and Pitkin, E. (2015). Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics, 24(1):44-65. DOI: 10.1080/10618600.2014.907095.
Hakkoum, H., Idri, A., and Abnane, I. (2024). Global and local interpretability techniques of supervised machine learning black box models for numerical medical data. Engineering Applications of Artificial Intelligence, 131:107829. DOI: 10.1016/j.engappai.2023.107829.
Hall, P., Gill, N., Kurka, M., and Phan, W. (2017). Machine learning interpretability with h2o driverless ai. H2O. ai. Available at: [link].
Hechtlinger, Y. (2016). Interpretation of prediction models using the input gradient. arXiv preprint arXiv:1611.07634. DOI: 10.48550/arXiv.1611.07634.
Hu, L., Chen, J., Nair, V. N., and Sudjianto, A. (2018). Locally interpretable models and effects based on supervised partitioning (lime-sup). arXiv preprint arXiv:1806.00663. DOI: 10.48550/arXiv.1806.00663.
James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning: with Applications in R. Springer. DOI: 10.1007/978-1-0716-1418-1.
Kim, B., Khanna, R., and Koyejo, O. (2016). Examples are not enough, learn to criticize! criticism for interpretability. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, page 2288–2296, Red Hook, NY, USA. Curran Associates Inc. Available at: [link].
Koh, P. W. and Liang, P. (2017). Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 1885–1894. JMLR.org. Available at: [link].
Lee, J. D., Sun, D. L., Sun, Y., and Taylor, J. E. (2016). Exact post-selection inference, with application to the lasso. The Annals of Statistics, 44(3):907 - 927. DOI: 10.1214/15-AOS1371.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 4768–4777, Red Hook, NY, USA. Curran Associates Inc. Available at: [link].
Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267:1-38. DOI: 10.1016/j.artint.2018.07.007.
Molnar, C. (2020). Interpretable machine learning. Lulu.com. DOI: 10.21105/joss.00786.
Molnar, C., Casalicchio, G., and Bischl, B. (2018). iml: An r package for interpretable machine learning. Journal of Open Source Software, 3(26):786. DOI: 10.21105/joss.00786.
Ng, A. Y., Jordan, M. I., and Weiss, Y. (2001). On spectral clustering: analysis and an algorithm. In Proceedings of the 15th International Conference on Neural Information Processing Systems: Natural and Synthetic, NIPS'01, page 849–856, Cambridge, MA, USA. MIT Press. Available at:[link].
Pedersen, T. L. and Benesty, M. (2021). lime: Local interpretable model-agnostic explanations. R package version 0.5.2. DOI: 10.32614/CRAN.package.lime.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). "Why Should I Trust You?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, page 1135–1144, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/2939672.2939778.
Saeed, W. and Omlin, C. (2023). Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Systems, 263:110273. DOI: 10.1016/j.knosys.2023.110273.
Shapley, L. S. (2016). 17. A Value for n-Person Games, pages 307-318. Princeton University Press, Princeton. DOI: 10.1515/9781400881970-018.
Slack, D., Hilgard, S., Jia, E., Singh, S., and Lakkaraju, H. (2020). Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES '20, page 180–186, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3375627.3375830.
Štrumbelj, E. and Kononenko, I. (2014). Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3):647-665. DOI: 10.1007/s10115-013-0679-x.
Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 3319–3328. JMLR.org. Available at:[link].
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267-288. DOI: 10.1111/j.2517-6161.1996.tb02080.x.
Zafar, M. R. and Khan, N. (2021). Deterministic local interpretable model-agnostic explanations for stable explainability. Machine Learning and Knowledge Extraction, 3(3):525-541. DOI: 10.3390/make3030027.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Gilson Yuuji Shimizu, Rafael Izbicki, Fernando Rezende Zagatti, Filipe Loyola Lopes, André Gomes Regino, Rodrigo Bonacin, Andre C. P. L. F. de Carvalho
This work is licensed under a Creative Commons Attribution 4.0 International License.
