Rating Prediction in Brazilian Portuguese: From Classical Features to Large Language Models
DOI:
https://doi.org/10.5753/reic.2026.8437Keywords:
Rating Prediction, Natural Language Processing, E-commerceAbstract
Online reviews play a crucial role in e-commerce, yet research on rating prediction for Brazilian Portuguese remains limited. This paper consolidates results from six interconnected studies investigating rating prediction and rating-text inconsistency detection. We evaluate approaches spanning classical machine learning with 58 textual features, BERT-based models, and ten large language models in zero-shot settings. Results show that BERTimbau achieves the best performance among fine-tuned models (MAE 0.56, RMSE 0.91), while DeepSeek and ChatGPT-4o lead among Large Language Models (LLMs) (RMSE 0.93). We also extend the analysis to a multilingual context with emoji signals across 13 European languages. For inconsistency detection, we find that LLM reliability varies substantially: ChatGPT-o3 shows low consistency across runs (κ = 0.18), while DeepSeek-3.2 achieves near-perfect agreement (κ > 0.95) with F1-score above 97%. Our findings provide practical guidelines for model selection based on accuracy requirements, training data availability, and cost constraints.
Downloads
References
Ahmed, B. H. and Ghabayen, A. S. (2022). Review rating prediction framework using deep learning. Journal of Ambient Intelligence and Humanized Computing, 13(7):3423–3432. DOI: 10.1007/s12652-020-01807-4.
Almansour, A., Alotaibi, R., and Alharbi, H. (2022). Text-rating review discrepancy (trrd): an integrative review and implications for research. Future Business Journal, 8(1):3. DOI: 10.1186/s43093-022-00114-y.
Barman, K. D., Bordoloi, B., Kumar, A., and Halder, A. (2024). Review rating predictions using improved deep learning architecture. In 2024 IEEE 16th International Conference on Computational Intelligence and Communication Networks (CICN), pages 468–472. IEEE. DOI: 10.1109/CICN63059.2024.10847509.
Chambua, J. and Niu, Z. (2021). Review text based rating prediction approaches: preference knowledge learning, representation and utilization. Artificial Intelligence Review, 54:1171–1200. DOI: 10.1007/s10462-020-09873-y.
de Melo, T., da Silva, A. S., de Moura, E. S., and Calado, P. (2019). Opinionlink: Leveraging user opinions for product catalog enrichment. Information Processing & Management, 56(3):823–843. DOI: 10.1016/j.ipm.2019.01.004.
Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the NAACL, pages 4171–4186. DOI: 10.18653/v1/N19-1423.
Gardazi, N. M., Daud, A., Malik, M. K., Bukhari, A., Alsahfi, T., and Alshemaimri, B. (2025). Bert applications in natural language processing: a review. Artificial Intelligence Review, 58(6):1–49. DOI: 10.1007/s10462-025-11162-5.
Kang, W.-C., Ni, J., Mehta, N., Sathiamoorthy, M., Hong, L., Chi, E., and Cheng, D. Z. (2023). Do llms understand user preferences? evaluating llms on user rating prediction. arXiv preprint arXiv:2305.06474. DOI: 10.48550/arXiv.2305.06474.
Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, pages 159–174. DOI: 10.2307/2529310.
Li, S., Liu, F., Zhang, Y., Zhu, B., Zhu, H., and Yu, Z. (2022). Text mining of user-generated content (UGC) for business applications in e-commerce: A systematic review. Mathematics, 10(19):3554. DOI: 10.3390/math10193554.
Liu, B. and Zhang, L. (2012). A survey of opinion mining and sentiment analysis. In Mining Text Data, pages 415–463. Springer. DOI: 10.1007/978-1-4614-3223-4_13.
Marreira, E. and de Melo, T. (2025). Predição numérica de avaliações em português: Comparando bertimbau e modelos multilíngues. In Simpósio Brasileiro de Banco de Dados (SBBD), pages 57–63. SBC. DOI: 10.5753/sbbd_estendido.2025.247518.
Marreira, E., de Melo, T., de Oliveira, M., and Figueiredo, C. M. (2025a). Rating prediction in brazilian portuguese: A benchmark of large language models. Journal of the Brazilian Computer Society, 31(1):827–838. DOI: 10.5753/jbcs.2025.5667.
Marreira, E., de Melo, T., de Oliveira, M., and Figueiredo, C. M. S. (2026a). Emoji signals across europe: Rating prediction and cross-lingual analysis. Natural Language Processing. Under review.
Marreira, E., de Melo, T., Oliveira, M., and Maurício, C. (2025b). Detectando incoerências avaliativas em e-commerce com llms: um estudo de caso na Amazon Brasil. In Brazilian Symposium on Multimedia and the Web (WebMedia), pages 535–539. SBC. DOI: 10.5753/webmedia.2025.15952.
Marreira, E., de Oliveira, M., and de Melo, T. (2025c). Rating prediction in brazilian portuguese reviews: An approach based on textual features. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 1352–1363. SBC. DOI: 10.5753/eniac.2025.11794.
Marreira, E., Figueiredo, C. M., and de Melo, T. (2026b). Rating–text mismatch in brazilian portuguese reviews: How reliable are zero-shot llms? In Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026), Vol. 1, pages 959–967.
Mudambi, S. M., Schuff, D., and Zhang, Z. (2014). Why aren’t the stars aligned? an analysis of online review content and star ratings. In 2014 47th Hawaii International Conference on System Sciences, pages 3139–3147. IEEE. DOI: 10.1109/HICSS.2014.389.
Pereira, D. A. (2021). A survey of sentiment analysis in the portuguese language. Artificial Intelligence Review, 54(2):1087–1115. DOI: 10.1007/s10462-020-09870-1.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 The authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
