Um Estudo Comparativo de Estratégias de Seleção de Exemplos para In-Context Learning Aplicado à Classificação Automática de Texto com LLMs

Gabriel Prenassi; Guilherme Fonseca; Marcos André Gonçalves; Leonardo Rocha

doi:10.5753/reic.2026.8461

Authors

Gabriel Prenassi Universidade Federal de São João del-Rei https://orcid.org/0009-0006-8748-6745
Guilherme Fonseca Universidade Federal de Minas Gerais https://orcid.org/0009-0000-7862-8701
Marcos André Gonçalves Universidade Federal de Minas Gerais https://orcid.org/0000-0002-2075-3363
Leonardo Rocha https://orcid.org/0000-0002-4913-4902

DOI:

https://doi.org/10.5753/reic.2026.8461

Keywords:

Text Classification, Large Language Models, In-Context Learning, Example Selection, Vector Representations

Abstract

Text Classification with LLMs can be performed via zero-shot (ZS) (lower cost and lower performance) or through fine-tuning (FT) (higher cost and higher performance). This work investigates in-context learning (ICL) as an intermediate alternative, analyzing the trade-off between effectiveness and efficiency. We evaluate example selection strategies, comparing a random approach with methods based on vector representations (TF-IDF, RoBERTa, SBERT, and LLM2Vec), as well as varying the number of examples included in the prompts. The results indicate that well-designed selection strategies make ICL promising, outperforming ZS in effectiveness while incurring lower cost than FT. Nevertheless, the best cost-benefit ratio remains with fine-tuning SLMs such as RoBERTa.

Downloads

Download data is not yet available.

References

BehnamGhader, P., Adlakha, V., Mosbach, M., Bahdanau, D., Chapados, N., and Reddy, S. (2024). Llm2vec: Large language models are secretly powerful text encoders.

Chandra, M., Ganguly, D., and Ounis, I. (2025). One size doesn’t fit all: Predicting the number of examples for in-context learning. In Advances in Information Retrieval, pages 67–84, Cham.

Cunha, W., França, C., Fonseca, G., Rocha, L., and Gonçalves, M. A. (2023). An effective, efficient, and scalable confidence-based instance selection framework for transformer-based text classification. In Proceedings ACM SIGIR, page 665–674. DOI: 10.1145/3539618.3591638.

Cunha, W., Moreo Fernández, A., Esuli, A., Sebastiani, F., Rocha, L., and Gonçalves, M. A. (2025a). A noise-oriented and redundancy-aware instance selection framework. ACM Trans. Inf. Syst., 43(2). DOI: 10.1145/3705000.

Cunha, W., Rocha, L., and Gonçalves, M. A. (2025b). A thorough benchmark of automatic text classification: From traditional approaches to large language models.

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding.

Edwards, A. and Camacho-Collados, J. (2024). Language models for text classification: Is in-context learning enough? In Proceedings of the 2024 LREC-COLING 2024, pages 10058–10072, Torino, Italia.

Fonseca, G., Cunha, W., Prenassi, G., Gonçalves, M. A., and Rocha, L. C. D. D. (2025). Instance-selection-inspired undersampling strategies for bias reduction in small and large language models for binary text classification. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 9323–9340.

Grattafiori, A. et al. (2024). The llama 3 herd of models.

Hochberg, Y. (1988). A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75(4):800–802.

Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models.

Kumar, S. and Talukdar, P. (2021). Reordering examples helps during priming-based few-shot learning. In Findings of ACL-IJCNLP 2021, pages 4507–4518, Online. Association for Computational Linguistics. DOI: 10.18653/v1/2021.findings-acl.395.

Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W. (2022). What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114. DOI: 10.18653/v1/2022.deelio-1.10.

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach.

Lu, Z., Tian, J., Wei, W., Qu, X., Cheng, Y., Xie, W., and Chen, D. (2024). Mitigating boundary ambiguity and inherent bias for text classification in the era of large language models. In Findings of ACL 2024, pages 7841–7864. DOI: 10.18653/v1/2024.findings-acl.467.

Luhn, H. P. (1957). A statistical approach to mechanized encoding and searching of literary information. IBM Journal of Research and Development, 1(4):309–317. DOI: 10.1147/rd.14.0309.

OpenAI et al. (2024). Gpt-4 technical report.

Prenassi, G. S., Fonseca, G., Reis, D., Cunha, W., Gonçalves, M. A., and Rocha, L. (2025). Um estudo comparativo de estratégias de seleção de exemplos para in-context learning aplicado à classificação automática de texto com grandes modelos de linguagem. In Simpósio Brasileiro de Banco de Dados (SBBD), pages 921–927. SBC.

Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 EMNLP.

Xu, C., Xu, Y., Wang, S., Liu, Y., Zhu, C., and McAuley, J. (2024). Small models are valuable plug-ins for large language models. In Findings of ACL 2024, pages 283–294. DOI: 10.18653/v1/2024.findings-acl.18.

A Comparative Study of Example Selection Strategies for In-Context Learning Applied to Automatic Text Classification with LLMs

Authors

DOI:

Keywords:

Abstract

Downloads

References

Downloads

Published

How to Cite

Issue

Section

License

Make a Submission

Language