Dynamic Mechanistic Interpretability for Tabular Foundation Models in Healthcare
DOI:
https://doi.org/10.5753/reic.2026.8498Keywords:
Mechanistic interpretability, tabular foundation models, healthcare, artificial intelligence, explainabilityAbstract
Tabular Foundation Models (TFMs) achieve high predictive performance but remain opaque and vulnerable to temporal drift in healthcare. We propose a Dynamic Mechanistic Interpretability framework that makes them auditable and time-aware. Our pipeline processes internal representations in batches and uses Sparse Autoencoders (SAEs) to disentangle overlapping activation patterns. By adapting Testing with Concept Activation Vectors (TCAV) together with surrogate decision trees, we identify clinical risk patterns without manual annotations. Evaluated on a longitudinal renal dialysis registry (1997–2015), the approach shows that predictions rely on stable latent concepts, distinguishing population drift from changes in the model’s internal reasoning.
Downloads
References
Bricken, T. et al. (2023). Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread.
Campos, J. M., Gomes, R. M., Chaves, D. T., Meira Jr., W., Cherchiglia, M. L., Rocha, H. A., Rocha, L., and Gonçalves, M. A. (2026). Mechanistic dynamic interpretability for tabular foundation models in healthcare. In XAI World Conference 2026. Accepted paper.
Cunningham, H. et al. (2023). Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600.
Elhadri, H. et al. (2024). XNNTab: Interpretable neural networks for tabular data using sparse autoencoders. arXiv preprint arXiv:2512.13442.
Ghorbani, A. et al. (2019). Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems (NeurIPS), volume 32.
Guerra Junior, A. A. et al. (2018). Building the national database of health centred on the individual: administrative and epidemiological record linkage—brazil, 2000–2015. International Journal of Population Data Science, 3.
Helli, K. et al. (2024). Drift-resilient TabPFN: In-context learning temporal distribution shifts on tabular data. In Advances in Neural Information Processing Systems (NeurIPS).
Hollmann, N. et al. (2023). TabPFN: A transformer that solves small tabular classification problems in a second. In International Conference on Learning Representations (ICLR).
Kim, B. et al. (2018). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International Conference on Machine Learning (ICML), pages 2668–2677.
Marcolino, M. S. et al. (2025). Explainable artificial intelligence for predicting cardiovascular events in hospitalised COVID-19 patients. BMC Infectious Diseases, 25:1569.
Molnar, C. (2022). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. 2 edition.
Paiva, B. et al. (2024). A new natural language processing–inspired methodology to investigate temporal drifts in health care data. JMIR Medical Informatics, 12:e54246.
Pendyala, S. et al. (2022). Concept-based explanations for tabular data. arXiv preprint arXiv:2209.05690.
Vaswani, A. et al. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 The authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
