BioNestedNER: A Hybrid Language Model Approach for Recognizing Nested, Discontinuous, and Multi-Type Named Entities

Elisa Terumi Rubel Schneider; Yohan Bonescki Gumiel; Paloma Martínez; Claudia Moro; Emerson Cabrera Paraiso

doi:10.5753/jbcs.2026.5790

Authors

Elisa Terumi Rubel Schneider Pontifícia Universidade Católica do Paraná https://orcid.org/0000-0002-8921-5598
Yohan Bonescki Gumiel Instituto do Coração - InCor/HC FMUSP https://orcid.org/0000-0001-8239-2930
Paloma Martínez Computer Science and Engineering Department, Universidad Carlos III de Madrid https://orcid.org/0000-0003-3013-3771
Claudia Moro Pontifícia Universidade Católica do Paraná https://orcid.org/0000-0003-2637-3086
Emerson Cabrera Paraiso Pontifícia Universidade Católica do Paraná https://orcid.org/0000-0002-6740-7855

DOI:

https://doi.org/10.5753/jbcs.2026.5790

Keywords:

Natural language processing, Named entity recognition, Language models, Clinical corpus, Transformer architecture, Machine learning, Question Answering

Abstract

Named Entity Recognition (NER) is essential in Natural Language Processing (NLP) for extracting pertinent information from unstructured data. Traditional NER approaches assume continuous and non-overlapping entities, which can be limiting in real-world scenarios. This research introduces BioNestedNER, a hybrid method for nested, discontinuous, and multi-type entity recognition, with a focus on clinical and biomedical domains. Our approach employs a language model (encoder-only Transformer-based model) using a machine reading comprehension strategy, treating NER as a question-answering-like task. A Conditional Random Field also addresses multi-label sequence labeling for handling nested entities as multi-type entities. Evaluation in Portuguese demonstrated state-of-the-art performance in micro F1-Scores across two clinical corpora. In NestedClinBr, featuring nested and discontinuous entities, our method achieved an F1-Score of 0.863, surpassing the second-place result by 2.1%. In SemClinBr, with multi-type entities, an F1-Score of 0.782 was achieved, surpassing the second-place result by 11.5%. This paper also presents a new clinical corpus in Brazilian Portuguese annotated with nested and discontinuous entities, offering a valuable resource for developing and evaluating models handling these complex entities. In conclusion, BioNestedNER presents an adaptable and effective NER solution for nested, discontinuous, and multi-type entities, with the potential to benefit various clinical applications.

Downloads

Download data is not yet available.

References

Alex, B., Haddow, B., and Grover, C. (2007). Recognising nested named entities in biomedical text. In Biological, translational, and clinical language processing, pages 65-72, Prague, Czech Republic. Association for Computational Linguistics. DOI: 10.3115/1572392.1572404.

Alhassan, A., Schlegel, V., Aloud, M., Batista-Navarro, R., and Nenadic, G. (2025). Discontinuous named entities in clinical text: A systematic literature review. Journal of Biomedical Informatics, 162. DOI: 10.1016/j.jbi.2025.104783.

Báez, P., Villena, F., Rojas, M., Durán, M., and Dunstan, J. (2020). The Chilean waiting list corpus: a new resource for clinical named entity recognition in Spanish. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 291-300, Online. Association for Computational Linguistics. DOI: 10.18653/v1/2020.clinicalnlp-1.32.

Banerjee, P., Pal, K. K., Devarakonda, M., and Baral, C. (2021). Biomedical named entity recognition via knowledge guidance and question answering. ACM Trans. Comput. Healthcare, 2(4). DOI: 10.1145/3465221.

Byrne, K. (2007). Nested named entity recognition in historical archive text. In International Conference on Semantic Computing (ICSC 2007), pages 589-596. DOI: 10.1109/ICSC.2007.107.

Campillos-Llanos L., Valverde-Mateos A., C.-C. A. (2021). A clinical trials corpus annotated with umls entities to enhance the access to evidence-based medicine.s. BMC Med Inform Decis Mak 21, 69. DOI: 10.1186/s12911-021-01395-z.

Campos, D., Matos, S., and Oliveira, J. L. (2012). Biomedical named entity recognition: A survey of machine-learning tools. In Sakurai, S., editor, Theory and Applications for Advanced Text Mining, chapter 8. IntechOpen, Rijeka. DOI: 10.5772/51066.

Chen, Y., Hu, Y., Li, Y., Huang, R., Qin, Y., Wu, Y., Zheng, Q., and Chen, P. (2020). A boundary assembling method for nested biomedical named entity recognition. IEEE Access, 8:214141-214152. DOI: 10.1109/ACCESS.2020.3040182.

Dai, X. (2018). Recognizing complex entity mentions: A review and future directions. In Proceedings of ACL 2018, Student Research Workshop, pages 37-44, Melbourne, Australia. Association for Computational Linguistics. DOI: 10.18653/v1/P18-3006.

Deleger, L., Li, Q., Lingren, T., Kaiser, M., Molnar, K., Stoutenborough, L., Kouril, M., Marsolo, K., Solti, I., et al. (2012). Building gold standard corpora for medical natural language processing tasks. In AMIA Annual Symposium Proceedings, volume 2012, page 144. American Medical Informatics Association. Available at:[link].

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics. DOI: 10.18653/v1/N19-1423.

Dogan, R., Leaman, R., and lu, Z. (2014). Ncbi disease corpus: A resource for disease name recognition and concept normalization. Journal of biomedical informatics, 47. DOI: 10.1016/j.jbi.2013.12.006.

e Oliveira, L. E. S., Peters, A. C., da Silva, A. M. P., Gebeluca, C. P., Gumiel, Y. B., Cintho, L. M. M., Carvalho, D. R., Hasan, S. A., and Moro, C. M. C. (2022). SemClinBr - a multi-institutional and multi-specialty semantically annotated corpus for portuguese clinical NLP tasks. Journal of Biomedical Semantics, 13(1). DOI: 10.1186/s13326-022-00269-1.

Fei, H., Ji, D., Li, B., Liu, Y., Ren, Y., and Li, F. (2021). Rethinking boundaries: End-to-end recognition of discontinuous mentions with pointer networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 12785-12793. DOI: 10.1609/aaai.v35i14.17513.

Finkel, J. R. and Manning, C. D. (2009). Nested named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 141-150, Singapore. Association for Computational Linguistics. DOI: 10.3115/1699510.1699529.

Gumiel, Y. B., Oliveira, L. E., de Souza, J. V., Schneider, E. T., Furlan, L. H., Paraiso, E. C., Moro, C., and Carvalho, D. R. (2023). Novel annotation schema for improved temporal reasoning over cardiology notes. Available at:[link].

Ji, L., Dang, Y., Du, Y., Gao, W., and Zhang, H. (2025). Nested named entity recognition: A survey of latest research. Expert Systems, 42(7):e70052. e70052 EXSY-Jan-25-037.R1. DOI: 10.1111/exsy.70052.

Ju, M., Miwa, M., and Ananiadou, S. (2018). A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1446-1459, New Orleans, Louisiana. Association for Computational Linguistics. DOI: 10.18653/v1/N18-1131.

Katiyar, A. and Cardie, C. (2018). Nested named entity recognition revisited. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 861-871, New Orleans, Louisiana. Association for Computational Linguistics. DOI: 10.18653/v1/N18-1079.

Kim, J.-D., Ohta, T., Tateisi, Y., and Tsujii, J. (2003). Genia corpus—a semantically annotated corpus for bio-textmining. Bioinformatics (Oxford, England), 19 Suppl 1:i180-2. DOI: 10.1093/bioinformatics/btg1023.

Lafferty, J. D., McCallum, A., and Pereira, F. C. N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML '01, page 282–289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. Available at:[link].

Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., and Kang, J. (2019). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234-1240. DOI: 10.1093/bioinformatics/btz682.

Li, J., Fei, H., Liu, J., Wu, S., Zhang, M., Teng, C., Ji, D., and Li, F. (2022). Unified named entity recognition as word-word relation classification. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):10965-10973. DOI: 10.1609/aaai.v36i10.21344.

Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., and Li, J. (2020). A unified MRC framework for named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849-5859, Online. Association for Computational Linguistics. DOI: 10.18653/v1/2020.acl-main.519.

Lin, H., Lu, Y., Han, X., Sun, L., Dong, B., and Jiang, S. (2019). Gazetteer-enhanced attentive neural networks for named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6232-6237, Hong Kong, China. Association for Computational Linguistics. DOI: 10.18653/v1/D19-1646.

Liu, J., Ji, D., Li, J., Xie, D., Teng, C., Zhao, L., and Li, F. (2023). Toe: A grid-tagging discontinuous ner model enhanced by embedding tag/word relations and more fine-grained tags. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:177-187. DOI: 10.1109/TASLP.2022.3221009.

Lopes, F., Gonçalo Oliveira, H., and Teixeira, C. (2020). Comparing different methods for named entity recognition in portuguese neurology text. Journal of Medical Systems. DOI: 10.1007/s10916-020-1542-8.

Lu, W. and Roth, D. (2015). Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 857-867, Lisbon, Portugal. Association for Computational Linguistics. DOI: 10.18653/v1/D15-1102.

Luo, Y. and Zhao, H. (2020). Bipartite flat-graph network for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. DOI: 10.18653/v1/2020.acl-main.571.

Mady, L., afify, y., and Badr, N. (2022). Nested biomedical named entity recognition. International Journal of Intelligent Computing and Information Sciences, 22(1):98-107. DOI: 10.21608/ijicis.2022.104170.1134.

Marinho, Z., Mendes, A., Miranda, S., and Nogueira, D. (2019). Hierarchical nested named entity recognition. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 28-34, Minneapolis, Minnesota, USA. Association for Computational Linguistics. DOI: 10.18653/v1/W19-1904.

Martínez-deMiguel, C., Segura-Bedmar, I., Chacón-Solano, E., and Guerrero-Aspizua, S. (2022). The raredis corpus: A corpus annotated with rare diseases, their signs and symptoms. Journal of Biomedical Informatics, 125:103961. DOI: 10.1016/j.jbi.2021.103961.

Naguib, M., Tannier, X., and Névéol, A. (2024). Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N., editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6829-6852, Miami, Florida, USA. Association for Computational Linguistics. DOI: 10.18653/v1/2024.findings-emnlp.400.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830. Available at:[link].

Rivera-Zavala, R. M. and Martínez, P. (2021). Analyzing transfer learning impact in biomedical cross-lingual named entity recognition and normalization. BMC bioinformatics, 22(1):1-23. DOI: 10.1186/s12859-021-04247-9.

Schneider, E. T. R., de Souza, J. V. A., Knafou, J., Oliveira, L. E. S. e., Copara, J., Gumiel, Y. B., Oliveira, L. F. A. d., Paraiso, E. C., Teodoro, D., and Barra, C. M. C. M. (2020). BioBERTpt - a Portuguese neural language model for clinical named entity recognition. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 65-72, Online. Association for Computational Linguistics. Available at:[link].

Shen, D., Zhang, J., Zhou, G., Su, J., and Tan, C.-L. (2003). Effective adaptation of hidden Markov model-based named entity recognizer for biomedical domain. In Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine, pages 49-56, Sapporo, Japan. Association for Computational Linguistics. DOI: 10.3115/1118958.1118965.

Shen, Y., Wang, X., Tan, Z., Xu, G., Xie, P., Huang, F., Lu, W., and Zhuang, Y. (2022). Parallel instance query network for named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 947-961, Dublin, Ireland. Association for Computational Linguistics. DOI: 10.18653/v1/2022.acl-long.67.

Sohrab, M. G. and Miwa, M. (2018). Deep exhaustive model for nested named entity recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2843-2849, Brussels, Belgium. Association for Computational Linguistics. DOI: 10.18653/v1/D18-1309.

Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: Pretrained bert models for brazilian portuguese. In Cerri, R. and Prati, R. C., editors, Intelligent Systems, pages 403-417, Cham. Springer International Publishing. DOI: 10.1007/978-3-030-61377-8_28.

Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., and Tsujii, J. (2012). Brat: a web-based tool for nlp-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 102-107. DOI: 10.1016/0020-7519(95)00001-i.

Tan, C., Qiu, W., Chen, M., Wang, R., and Huang, F. (2020). Boundary enhanced neural span classification for nested named entity recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):9016-9023. DOI: 10.1609/aaai.v34i05.6434.

Tsoumakas, G. and Katakis, I. (2009). Multi-label classification: An overview. International Journal of Data Warehousing and Mining, 3:1-13. DOI: 10.4018/jdwm.2007070101.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. NIPS'17, page 6000–6010, Red Hook, NY, USA. Curran Associates Inc.. DOI: 10.65215/nxvz2v36.

Wang, B. and Lu, W. (2018). Neural segmental hypergraphs for overlapping mention recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 204-214, Brussels, Belgium. Association for Computational Linguistics. DOI: 10.18653/v1/D18-1019.

Wang, J., Shou, L., Chen, K., and Chen, G. (2020). Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5918-5928, Online. Association for Computational Linguistics. DOI: 10.18653/v1/2020.acl-main.525.

Wang, L., Li, R., Yan, Y., Yan, Y., Wang, S., Wu, W., and Xu, W. (2022). Instructionner: A multi-task instruction-based generative framework for few-shot ner. DOI: 10.48550/arxiv.2203.03903.

Wang, S., Sun, X., Li, X., Ouyang, R., Wu, F., Zhang, T., Li, J., Wang, G., and Guo, C. (2025). GPT-NER: Named entity recognition via large language models. In Chiruzzo, L., Ritter, A., and Wang, L., editors, Findings of the Association for Computational Linguistics: NAACL 2025, pages 4257-4275, Albuquerque, New Mexico. Association for Computational Linguistics. DOI: 10.18653/v1/2025.findings-naacl.239.

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics. DOI: 10.18653/v1/2020.emnlp-demos.6.

Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., and Qiu, X. (2021). A unified generative framework for various ner subtasks. pages 5808-5822. DOI: 10.18653/v1/2021.acl-long.451.

Yuan, C., Wang, Y., Shang, N., Li, Z., Zhao, R., and Weng, C. (2020). A graph-based method for reconstructing entities from coordination ellipsis in medical text. Journal of the American Medical Informatics Association, 27(9):1364-1373. DOI: 10.1093/jamia/ocaa109.

Zhang, J., Liu, X., Lai, X., Gao, Y., Wang, S., Hu, Y., and Lin, Y. (2023). 2INER: Instructive and in-context learning on few-shot named entity recognition. In Bouamor, H., Pino, J., and Bali, K., editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3940-3951, Singapore. Association for Computational Linguistics. DOI: 10.18653/v1/2023.findings-emnlp.259.

Zhang, J., Shen, D., Zhou, G., Su, J., and Tan, C.-L. (2004). Enhancing hmm-based biomedical named entity recognition by studying special phenomena. Journal of Biomedical Informatics, 37(6):411-422. Named Entity Recognition in Biomedicine. DOI: 10.1016/j.jbi.2004.08.005.

Zhang, Y., Xu, G., Wang, Y., Lin, D., Li, F., Wu, C., Zhang, J., and Huang, T. (2020). A question answering-based framework for one-step event argument extraction. IEEE Access, 8:65420-65431. DOI: 10.1109/ACCESS.2020.2985126.

Zhou, G. (2006). Recognizing names in biomedical texts using mutual information independence model and svm plus sigmoid. International Journal of Medical Informatics, 75(6):456-467. Recent Advances in Natural Language Processing for Biomedical Applications Special Issue. DOI: 10.1016/j.ijmedinf.2005.06.012.

Zhou, G., Zhang, J., Jian, S., Shen, D., and Tan, C. L. (2004). Recognizing names in biomedical texts: a machine learning approach. Bioinformatics (Oxford, England), 20:1178-90. DOI: 10.1093/bioinformatics/bth060.