Enhancing Brazilian Legal Information Retrieval: An Automated Keyphrase Generation

Breno O. Funicheli; Kenzo Sakiyama; Rodrigo Nogueira; Roseli A. F. Romero

doi:10.5753/jbcs.2025.5711

Authors

Breno O. Funicheli University of São Paulo, São Carlos, Brazil https://orcid.org/0000-0002-4240-4578
Kenzo Sakiyama University of São Paulo, São Carlos, Brazil https://orcid.org/0000-0002-5284-4745
Rodrigo Nogueira University of Campinas, Campinas, Brazil https://orcid.org/0000-0002-2600-6035
Roseli A. F. Romero University of São Paulo, São Carlos, Brazil https://orcid.org/0000-0001-9366-2780

DOI:

https://doi.org/10.5753/jbcs.2025.5711

Keywords:

Text Generation, Information Retrieval, Legal Texts

Abstract

The volume of legal proceedings in Brazil has grown significantly in recent years, highlighting the potential for leveraging advances in Natural Language Processing (NLP) to automate tasks in the legal domain. This study explores text decoding methods for automating keyphrase generation—sequences of key terms that summarize legal documents. A sequence-to-sequence Transformer-based framework generates keyphrases using three decoding techniques: greedy, top-K, and top-p sampling. To evaluate the effectiveness of the generated keyphrases, we integrate them into legal document retrieval tasks using traditional Information Retrieval (IR) methods, such as TF-IDF and BM25. Our results, validated through IR metrics, demonstrate that greedy decoding produces high-quality keyphrases that closely align with those written by human specialists, achieving statistically significant improvements in retrieval performance. As part of this study, we introduce a new data set of Brazilian legal documents, including dates and pre-processed keyphrases, which allows reproducibility and supports further research on keyphrase generation and legal document retrieval tasks.

Downloads

Download data is not yet available.

References

Althammer, S., Askari, A., Verberne, S., and Hanbury, A. (2021). Dossier@ coliee 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval. arXiv preprint arXiv:2108.03937. DOI: 10.48550/arXiv.2108.03937.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901. DOI: 10.48550/arxiv.2005.14165.

Carmo, D., Piau, M., Campiotti, I., Nogueira, R., and Lotufo, R. (2020). Ptt5: Pretraining and validating the t5 model on brazilian portuguese data. arXiv preprint arXiv:2008.09144. DOI: 10.48550/arxiv.2008.09144.

Clauset, A., Shalizi, C. R., and Newman, M. E. J. (2009). Power-law distributions in empirical data. SIAM Review, 51(4):661-703. DOI: 10.1137/070710111.

Conselho Nacional de Justiça - CNJ (23). Conselho nacional de justiça — justiça em números. Available online [link]. Accessed on 08/05/2023.

Fan, A., Lewis, M., and Dauphin, Y. N. (2018). Hierarchical neural story generation. CoRR, abs/1805.04833. DOI: 10.18653/v1/p18-1082.

Feijo, D. and Moreira, V. (2019). Summarizing legal rulings: Comparative experiments. In proceedings of the international conference on recent advances in natural language processing (RANLP 2019), pages 313-322. DOI: 10.26615/978-954-452-056-4_036.

Floridi, L. and Chiriatti, M. (2020). Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30(4):681-694. DOI: 10.1007/s11023-020-09548-1.

Guimarães, J. A. C. and Santos, J. C. G. (2016). A ementa jurisprudencial como resumo informativo em um domínio especializado: aspectos estruturais. Brazilian Journal of Information Science: research trends, 10(3). Avilable at:[link].

Holtzman, A., Buys, J., Forbes, M., and Choi, Y. (2019). The curious case of neural text degeneration. CoRR, abs/1904.09751. DOI: 10.48550/arxiv.1904.09751.

Huang, Y., Shen, X., Li, C., Ge, J., and Luo, B. (2021). Dependency learning for legal judgment prediction with a unified text-to-text transformer. arXiv preprint arXiv:2112.06370. DOI: 10.48550/arxiv.2112.06370.

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., and Fung, P. (2022). Survey of hallucination in natural language generation. ACM Computing Surveys. DOI: 10.1145/3571730.

Li, W. (2002). Zipf's law everywhere. Glottometrics, 5:14-21. Available online [link].

Lima, J. P., Costa, J. A., and Araújo, D. C. (2021). Comparison of feature extraction methods for brazilian legal documents clustering. In 2021 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pages 1-5. IEEE. DOI: 10.1109/la-cci48322.2021.9769839.

Lin, C.-Y. (2004). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81. Available online [link].

Mandal, A., Ghosh, K., Ghosh, S., and Mandal, S. (2021). Unsupervised approaches for measuring textual similarity between legal court case reports. Artificial Intelligence and Law, 29(3):417-451. DOI: 10.1007/s10506-020-09280-2.

Nogueira, R., Yang, W., Lin, J., and Cho, K. (2019). Document expansion by query prediction. arXiv preprint arXiv:1904.08375. DOI: 10.48550/arxiv.1904.08375.

Ostendorff, M., Ash, E., Ruas, T., Gipp, B., Moreno-Schneider, J., and Rehm, G. (2021). Evaluating document representations for content-based legal literature recommendations. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law, pages 109-118. DOI: 10.1145/3462757.3466073.

Pacheco, C. C. (2008). Os estudos sobre judiciário e política no brasil pós 1988: uma revisão da literatura. Pensar-Revista de Ciências Jurídicas, 13(1):75-86. DOI: 10.5020/23172150.2012.75-86.

Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318. Available online [link].

Pedroso, D. d. S. C., Ladeira, M., and de Paulo Faleiros, T. (2019). Does semantic search performs better than lexical search in the task of assisting legal opinion writing? In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pages 680-685. IEEE. DOI: 10.1109/icmla.2019.00123.

Peric, L., Mijic, S., Stammbach, D., and Ash, E. (2020). Legal language modeling with transformers. In Proceedings of the Fourth Workshop on Automated Semantic Analysis of Information in Legal Text (ASAIL 2020) held online in conjunction with te 33rd International Conference on Legal Knowledge and Information Systems (JURIX 2020) December 9, 2020, volume 2764. CEUR-WS. Available online [link].

Perlingeiro, R. and Schmidt, L. S. (2023). An overview of environmental justice in brazil. British Journal of American Legal Studies, 12(1):27-50. DOI: 10.2478/bjals-2023-0003.

Piantadosi, S. T. (2014). Zipf's word frequency law in natural language: A critical review and future directions. Psychonomic Bulletin & Review, 21(5):1112-1130. DOI: 10.3758/s13423-014-0585-6.

Post, M. (2018). A call for clarity in reporting bleu scores. arXiv preprint arXiv:1804.08771. DOI: 10.18653/v1/w18-6319.

Pradeep, R., Ma, X., Zhang, X., Cui, H., Xu, R., Nogueira, R., and Lin, J. (2020). H2oloo at trec 2020: When all you got is a hammer... deep learning, health misinformation, and precision medicine. Corpus, 5(d3):d2. DOI: 10.6028/nist.sp.1266.misinfo-h2oloo.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. Available online [link].

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P. J., et al. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1-67. Available online [link].

Robertson, S. E. and Walker, S. (1999). Okapi/keenbow at trec-8. In TREC, volume 8, pages 151-162. Citeseer. DOI: 10.6028/nist.sp.500-246.adhoc-microsoft.

Rosa, G. M., Bonifacio, L. H., de Souza, L. R., Lotufo, R., and Nogueira, R. (2021a). A cost-benefit analysis of cross-lingual transfer methods. arXiv preprint arXiv:2105.06813. DOI: 10.48550/arxiv.2105.06813.

Rosa, G. M., Rodrigues, R. C., Lotufo, R., and Nogueira, R. (2021b). Yes, bm25 is a strong baseline for legal case retrieval. arXiv preprint arXiv:2105.05686. DOI: 10.48550/arXiv.2105.05686.

Russell-Rose, T., Chamberlain, J., and Azzopardi, L. (2018). Information retrieval in the workplace: A comparison of professional search practices. Information Processing & Management, 54(6):1042-1057. DOI: 10.1016/j.ipm.2018.07.003.

Sakiyama, K., Montanari, R., Malaquias Junior, R., Nogueira, R., and Romero, R. A. (2023a). Exploring text decoding methods for portuguese legal text generation. In Brazilian Conference on Intelligent Systems, pages 63-77. Springer. DOI: 10.1007/978-3-031-45368-7_5.

Sakiyama, K., Nogueira, R., and Romero, R. A. F. (2023b). Automated keyphrase generation for brazilian legal information retrieval. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1-8. DOI: 10.1109/IJCNN54540.2023.10191598.

Sanderson, M. (2010). Test collection based evaluation of information retrieval systems. Now Publishers Inc. DOI: 10.1561/1500000009.

Saravanan, M., Ravindran, B., and Raman, S. (2009a). Automatic identification of rhetorical roles using conditional random fields for legal document summarization. In International Conference on Artificial Intelligence and Law, pages 231-232. Avilable at:[link].

Saravanan, M., Ravindran, B., and Raman, S. (2009b). Improving legal information retrieval using an ontological framework. Artificial Intelligence and Law, 17(2):101-124. DOI: 10.1007/s10506-009-9075-y.

Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić,, S., Hesslow, D., Castagné, R., Luccioni, A. S., Yvon, F., Gallé, M., et al. (2022). Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. DOI: 10.48550/arxiv.2211.05100.

Schweighofer, E. (2010). Semantic indexing of legal documents. In Semantic Processing of Legal Texts: Where the Language of Law Meets the Law of Language, pages 157-169. Springer. DOI: 10.1007/978-3-642-12837-0.

Souza, E., Moriyama, G., Vitório, D., de Carvalho, A. C., Félix, N., Albuquerque, H. O., and Oliveira, A. L. (2021). Assessing the impact of stemming algorithms applied to brazilian legislative documents retrieval. In Anais do XIII Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 227-236. SBC. DOI: 10.5753/stil.2021.17802.

Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: pretrained bert models for brazilian portuguese. In Brazilian conference on intelligent systems, pages 403-417. Springer. DOI: 10.1007/978-3-030-61377-8_28.

Su, Y., Lan, T., Wang, Y., Yogatama, D., Kong, L., and Collier, N. (2022). A contrastive framework for neural text generation. arXiv preprint arXiv:2202.06417. DOI: 10.48550/arxiv.2202.06417.

Superior Tribunal de Justiça (2024). Stj registra crescimento de 5% no número de processos recebidos. Available online [link].Acessado em: 5 nov. 2024.

Tinarrage, R., Ennes, H., Resck, L. E., Gomes, L. T., Ponciano, J. R., and Poco, J. (2024). Empirical analysis of biding precedent efficiency in the brazilian supreme court via similar case retrieval. arXiv preprint arXiv:2407.07004. DOI: 10.48550/arxiv.2407.07004.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. DOI: 10.48550/arxiv.1706.03762.

Wagner Filho, J. A., Wilkens, R., Idiart, M., and Villavicencio, A. (2018). The brwac corpus: a new open resource for brazilian portuguese. In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018). Avilable at:[link].

Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2020). mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934. DOI: 10.18653/v1/2021.naacl-main.41.

Yoon, J., Junaid, M., Ali, S., and Lee, J. (2022). Abstractive summarization of korean legal cases using pre-trained language models. In 2022 16th International Conference on Ubiquitous Information Management and Communication (IMCOM), pages 1-7. IEEE. DOI: 10.1109/imcom53663.2022.9721808.

Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022). Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068. DOI: 10.48550/arxiv.2205.01068.

Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. (2019). Bertscore: Evaluating text generation with BERT. CoRR, abs/1904.09675. DOI: 10.48550/arxiv.1904.09675.

Zipf, G. K. (1949). Human Behavior and the Principle of Least Effort. Addison-Wesley. Book.