Drug Usage Instructions Supported by RAG in Large Language Models
DOI:
https://doi.org/10.5753/reic.2025.6039Keywords:
Artificial Intelligence, Large Language Models, Retrieval-Augmented Generation, Eletronic Prescribing, Medication SafetyAbstract
Prescription systems improve medication treatment by standardizing content and enhancing legibility, but challenges remain in personalizing instructions. This study proposes and evaluates a new approach based on Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) using patient information leaflets (PIL). Three models were tested on 119 outpatient cases, with instructions assessed by physicians for adequacy, clarity, and personalization. Our proposal significantly improved adequacy (100 vs. 93.0) and clarity (95.0 vs. 90.0), reduced errors while minimizing hallucinations. Our proposal can enhance medication safety by integrating authoritative information, but human validation remains essential for safe implementation.
Downloads
References
Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. (2024). M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the ACL. DOI: 10.18653/v1/2024.findings-acl.137.
Gosselin, L., Thibault, M., Lebel, D., and Bussières, J.- F. (2021). Utilisation de l’intelligence artificielle en pharmacie : une revue narrative. CJHP, 74. DOI: 10.4212/cjhp.v74i2.3100.
Goyal, S., Rastogi, E., Rajagopal, S. P., Yuan, D., Zhao, F., Chintagunta, J., Naik, G., and Ward, J. (2024). HealAI: A healthcare LLM for effective medical documentation. In Proceedings of the 17th WSDM. ACM. DOI: 10.1145/3616855.3635739.
Klepser, D., Lanham, A., and Cochran, G. (2016). Electronic prescriptions: opportunities and challenges for the patient and pharmacist. AHCT. DOI: 10.2147/AHCT.S64477.
Lu, Z., Tian, J., Wei, W., Qu, X., Cheng, Y., Xie, W., and Chen, D. (2024). Mitigating boundary ambiguity and inherent bias for text classification in the era of large language models. In Findings of ACL. DOI: 10.18653/v1/2024.findings-acl.467.
Reis, Z. S. N., Pagano, A. S., Ramos De Oliveira, I. J., Dias, C. D. S., Lage, E. M., Mineiro, E. F., Varella Pereira, G. M., De Carvalho Gomes, I., Basilio, V. A., Cruz-Correia, R. J., De Jesus, D. D. R., De Souza Júnior, A. P., and Rocha, L. (2024). Evaluating large language model–supported instructions for medication use: First steps toward a comprehensive model. Mayo Clinic Proceedings: Digital Health, 2. DOI: 10.1016/j.mcpdig.2024.09.006.
Rezende, L. H. O., Gehrke, F. D. S., Silva, M. A., Carneiro, A. M. F., Abreu, R. M., Monteiro, C. N., Leão, K. A., and Takei, K. (2020). Prescrição de medicamentos: uma análise para a implantação da prescrição eletrônica ambulatorial. Acervo Saúde, 12. DOI: 10.25248/reas.e3638.2020.
Roy, S., Khatua, A., Ghoochani, F., Hadler, U., Nejdl, W., and Ganguly, N. (2024). Beyond accuracy: Investigating error types in GPT-4 responses to USMLE questions. In Proceedings of the 47th ACM SIGIR. ACM. DOI: 10.1145/3626772.3657882.
Shiffman, S., Gerlach, K. K., Sembower, M. A., and Rohay, J. M. (2011). Consumer understanding of prescription drug information: An illustration using an antidepressant medication. Ann Pharmacother, 45. DOI: 10.1345/aph.1P477.
Tam, T. Y. C., Sivarajkumar, S., Kapoor, S., Stolyar, A. V., Polanska, K., McCarthy, K. R., Osterhoudt, H., Wu, X., Visweswaran, S., Fu, S., Mathur, P., Cacciamani, G. E., Sun, C., Peng, Y., and Wang, Y. (2024). A framework for human evaluation of large language models in healthcare derived from literature review. npj Digit. Med., 7. DOI: 10.1038/s41746-024-01258-7.
Xiong, G., Jin, Q., Lu, Z., and Zhang, A. (2024). Benchmarking retrieval-augmented generation for medicine. In Findings of ACL. Association for Computational Linguistics. DOI: 10.18653/v1/2024.findings-acl.372.
Yang, B., Jiang, S., Xu, L., Liu, K., Li, H., Xing, G., Chen, H., Jiang, X., and Yan, Z. (2024). DrHouse: An LLM-empowered diagnostic reasoning system through harnessing outcomes from sensor data and expert knowledge. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 8. DOI: 10.1145/3699765.
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., Boyd, J., Boyd, K., Hirsch, S., Vogelsong, M. A., Cunningham, J. P., and Hiesinger, W. (2024). Almanac — retrieval-augmented language models for clinical medicine. NEJM AI, 1. DOI: 10.1056/AIoa2300068.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 The authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
