Transformers para previsão de desempenho acadêmico no ensino Fundamental e Médio
DOI:
https://doi.org/10.5753/rbie.2024.3661Keywords:
Desempenho acadêmico, Transformers, EDMAbstract
A previsão de desempenho acadêmico apresenta um potencial grande no trabalho pró-ativo das escolas na identificação de alunos em risco de reprovação. de duas redes distintas, permitindo a comparação entre diferentes anos escolares, anos letivos e redes de ensino. Contrastaram-se os desempenhos de modelos baseados na arquitetura Transformers com modelos mais estabelecidos, como o XGBoost e um modelo de rede neural mais simples. Os resultados mostraram que os Transformers tiveram um desempenho interessante na tarefa de previsão de desempenho acadêmico, especialmente com um número maior de avaliações. No entanto, o XGBoost conseguiu alcançar um alto desempenho mais cedo no período letivo. Uma vantagem dos Transformers é sua flexibilidade no treinamento, permitindo lidar com conjuntos de dados semi-estruturados sem a necessidade de pré-processamento. Em última análise, esta pesquisa contribui para o desenvolvimento de métodos que podem identificar precocemente alunos em risco de reprovação, oferecendo a oportunidade de intervenção e apoio adequados. Isso pode ter um impacto positivo na formação dos alunos e na sociedade como um todo, mitigando prejuízos e promovendo a educação de qualidade.
Downloads
Referências
Almayan, H., & Al Mayyan, W. (2016). Improving accuracy of students’ final grade prediction model using PSO (2ª ed.). Decision Analytics, 35–39.https://doi.org/10.1109/INFOCOMAN.2016.7784211. [GS Search]
Amra, I., & Maghari, A. (2017). Students performance prediction using KNN and Naïve Bayesian (8ª ed.). International Conference on Information Technology (ICIT), 909–913. https://doi.org/10.1109/ICITECH.2017.8079967. [GS Search]
Athani, S. S., Kodli, S. A., Banavasi, M. N., & Hiremath, P. G. S. (2017). Student academic performance and social behavior predictor using data mining techniques (s.n). 2017 International Conference on Computing, Communication and Automation (ICCCA), 170–174. https://doi.org/10.1109/CCAA.2017.8229794. [GS Search]
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate.https://doi.org/10.48550/ARXIV.1409.0473. [GS Search]
Barros, T. M., Souza Neto, P. A., Silva, I., & Guedes, L. A. (2019). Predictive Models for Imbalanced Data: A School Dropout Perspective. Education Sciences, 9(44), 275.https://doi.org/10.3390/educsci9040275. [GS Search]
Blasi, A. (2017). Performance increment of high school students using ANN model and sa algorithm. Journal of Theoretical and Applied Information Technology, 95(11), 2417–2425.[GS Search]
Bonaccorso, G. (2017). Machine learning algorithms. Packt Publishing Ltd. [GS Search]
Chen, Q., Zhao, H., Li, W., Huang, P., & Ou, W. (2019). Behavior Sequence Transformer for ECommerce Recommendation in Alibaba. Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. https://doi.org/10.1145/3326937.3341261. [GS Search]
Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. https://doi.org/10.1145/2939672.2939785. [GS Search]
Chen, W., Brinton, C. G., Cao, D., Mason-Singh, A., Lu, C., & Chiang, M. (2019). Early Detection Prediction of Learning Outcomes in Online Short-Courses via Learning Behaviors. IEEE Transactions on Learning Technologies, 12(1), 44–58. https://doi.org/10.1109/TLT.2018.2793193. [GS Search]
Chollet, F., et al. (2015). Keras.[GS Search]
Cornell-Farrow, S., & Garrard, R. (2020). Machine learning classifiers do not improve the prediction of academic risk: Evidence from Australia. Communications in Statistics Case Studies Data Analysis and Applications, 6(2), 228–246. https://doi.org/10.1080/23737484.2020.1752849. [GS Search]
Cortez, P., & Silva, A. M. G. (2008). Using data mining to predict secondary school student performance (s.n) [Accepted: 2008-08-20T18:31:05Z]. Proceedings of 5th Annual Future Business Technology Conference. https://repositorium.sdum.uminho.pt/handle/1822/8024. [GS Search]
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. https://doi.org/10.48550/ARXIV.1810.04805. [GS Search]
Elsayed, S., Thyssens, D., Rashed, A., Schmidt-Thieme, L., & Jomaa, H. S. (2021). Do We Really Need Deep Learning Models for Time Series Forecasting? CoRR, abs/2101.02118. https://arxiv.org/abs/2101.02118. [GS Search]
Fávero, L. P., & Belfiore, P. (2017). Manual de análise de dados: estatística e modelagem multivariada com Excel®, SPSS® e Stata®. Elsevier Brasil. [GS Search]
Fernandes, E., Holanda, M., Victorino, M., Borges, V., Carvalho, R., & Erven, G. (2019). Educational data mining: Predictive analysis of academic performance of public school students in the capital of Brazil. Journal of Business Research, 94, 335–343. https://doi.org/10.1016/j.jbusres.2018.02.012. [GS Search]
García-González, J., & Skrita, A. (2019). Predicting academic performance based on students’ family environment: Evidence for Colombia using classification trees. Psychology, Society and Education, 11(3), 299–311. https://doi.org/10.25115/psye.v11i3.2056. [GS Search]
Gil, J., Delima, A., & Vilchez, R. (2020). Predicting students’ dropout indicators in public school using data mining approaches. International Journal of Advanced Trends in Computer Science and Engineering, 9(1), 774–778. https://doi.org/10.30534/ijatcse/2020/110912020. [GS Search]
Goldschmidt, R., Passos, E., & Bezerra, E. (2015). Data mining. Elsevier Brasil. [GS Search]
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press. [GS Search]
Gottardo, E., Kaestner, C. A. A., & Noronha, R. V. (2014). Estimativa de Desempenho Acadêmico de Estudantes: Análise da Aplicação de Técnicas de Mineração de Dados em Cursos a Distância. Revista Brasileira de Informática na Educacação, 22(1). https://doi.org/10.5753/rbie.2014.22.01.45. [GS Search]
Guo, B., Zhang, R., Xu, G., Shi, C., & Yang, L. (2016). Predicting Students Performance in Educational Data Mining (s.n). 2015 International Symposium on Educational Technology (ISET), 125–128. https://doi.org/10.1109/ISET.2015.33. [GS Search]
Guyon, I. (1991). Neural networks and applications tutorial. Physics Reports, 207(3-5), 215–259. [GS Search]
H. Alamri, L., S. Almuslim, R., S. Alotibi, M., K. Alkadi, D., Ullah Khan, I., & Aslam, N. (2020). Predicting Student Academic Performance using Support Vector Machine and Random Forest (3ª ed.) [01/08/2021]. 2020 3rd International Conference on Education Technology Management, (s.n), 100–107. https://doi.org/10.1145/3446590.3446607. [GS Search]
Hellas, A., Ihantola, P., Petersen, A., Ajanovski, V. V., Gutica, M., Hynninen, T., Knutas, A., Leinonen, J., Messom, C., & Liao, S. N. (2018). Predicting academic performance: a systematic literature review (23ª ed.) [01/08/2021]. Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, (s.n), 175–199. https://doi.org/10.1145/3293881.3295783. [GS Search]
Hernández-Blanco, A., Herrera-Flores, B., Tomás, D., & Navarro-Colorado, B. (2019). A Systematic Review of Deep Learning Approaches to Educational Data Mining. Complexity, 2019, e1306039. https://doi.org/10.1155/2019/1306039. [GS Search]
Hussain, S., Muhsin, Z., Salal, Y., Theodorou, P., Kurtoglu, F., & Hazarika, G. (2019). Prediction model on student performance based on internal assessment using deep learning. International Journal of Emerging Technologies in Learning, 14(8), 4–22. https://doi.org/10.3991/ijet.v14i08.10001. [GS Search]
Hussain, S., & Khan, M. Q. (2021). Student-Performulator: Predicting Students’ Academic Performance at Secondary and Intermediate Level Using Machine Learning. Annals of Data Science, s.n, 1–19.[GS Search].
Imran, M., Latif, S., Mehmood, D., & Shah, M. (2019). Student academic performance prediction using supervised learning techniques. International Journal of Emerging Technologies in Learning, 14(14), 92–104. https://doi.org/10.3991/ijet.v14i14.10310. [GS Search]
Kim, B.-H., Vizitei, E., & Ganapathi, V. (2019). Domain Adaptation for Real-Time Student Performance Prediction. https://doi.org/10.48550/arXiv.1809.06686. [GS Search]
Kim, B., Vizitei, E., & Ganapathi, V. (2018). GritNet: Student Performance Prediction with Deep Learning [01/08/2021]. CoRR, abs/1804.07405. https://doi.org/10.48550/arXiv.1804.07405. [GS Search]
Lee, S., & Chung, J. Y. (2019). The Machine Learning-Based Dropout Early Warning System for Improving the Performance of Dropout Prediction. Applied Sciences, 9(1515), 3093. https://doi.org/10.3390/app9153093. [GS Search]
Livingstone, D. J. (2008). Artificial neural networks: methods and applications. Springer. [GS Search]
Liz-Domínguez, M., Caeiro-Rodríguez, M., Llamas-Nistal, M., & Mikic-Fonte, F. A. (2019). Systematic Literature Review of Predictive Analysis Tools in Higher Education. Applied Sciences, 9(2424), 5569.https://doi.org/10.3390/app9245569. [GS Search]
Lu, H., & Yuan, J. (2018). Student performance prediction model based on discriminative feature selection. International Journal of Emerging Technologies in Learning, 13(10), 55–68.https://doi.org/10.3991/ijet.v13i10.9451. [GS Search]
Márquez-Vera, C., Cano, A., Romero, C., & Ventura, S. (2013). Predicting student failure at school using genetic programming and different data mining approaches with high dimensional and imbalanced data. Applied Intelligence, 38(3), 315–330. https://doi.org/10.1007/s10489-012-0374-8. [GS Search]
Mitchell, T. M., et al. (2007). Machine learning (Vol. 1). McGraw-hill New York. [GS Search]
Namoun, A., & Alshanqiti, A. (2021). Predicting Student Performance Using Data Mining and Learning Analytics Techniques: A Systematic Literature Review. Applied Sciences, 11(11), 237. https://doi.org/10.3390/app11010237. [GS Search]
Niu, Z., Zhong, G., & Yu, H. (2021). A review on the attention mechanism of deep learning. Neurocomputing, 452, 48–62. https://doi.org/10.1016/j.neucom.2021.03.091. [GS Search]
Oliphant, T. E. (2007). Python for Scientific Computing. Computing in Science Engineering, 9(3),10–20. https://doi.org/10.1109/MCSE.2007.58. [GS Search]
Orooji, M., & Chen, J. (2019). Predicting Louisiana Public High School Dropout through Imbalanced Learning Techniques [01/08/2021]. CoRR, abs/1910.13018. http://arxiv.org/abs/1910.13018. [GS Search]
Pandey, M., Fernandez, M., Gentile, F., Isayev, O., Tropsha, A., Stern, A. C., & Cherkasov, A. (2022). The transformational role of GPU computing and deep learning in drug discovery. Nature Machine Intelligence, 4(3), 211–221. https://doi.org/10.1038/s42256-022-00463-x. [GS Search]
Qazdar, A., Er-Raha, B., Cherkaoui, C., & Mammass, D. (2019). A machine learning algorithm framework for predicting students performance: A case study of baccalaureate students in Morocco. Education and Information Technologies, 24(6), 3577–3589. https://doi.org/10.1007/s10639-019-09946-8. [GS Search]
Razaque, A., & Alajlan, A. (2020). Supervised machine learning model-based approach for performance prediction of students. Journal of Computer Science, 16(8), 1150–1162. https://doi.org/10.3844/jcssp.2020.1150.1162. [GS Search]
Rodrigues, L. S., dos Santos, M., Costa, I., & Moreira, M. A. L. (2022). Student Performance Prediction on Primary and Secondary Schools-A Systematic Literature Review. Procedia Computer Science, 214, 680–687. https://doi.org/10.1016/j.procs.2022.11.229. [GS Search]
Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (2010). Handbook of educational data mining. CRC press. [GS Search]
Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6), 386.https://doi.org/10.1037/h0042519. [GS Search]
Rovira, S., Puertas, E., & Igual, L. (2017). Data-driven system to predict academic grades and dropout [01/08/2021]. PLOS ONE, 12(2), 1–21. https://doi.org/10.1371/journal.pone.0171207. [GS Search]
Roy, S., & Garg, A. (2017). Predicting academic performance of student using classification techniques (4ª ed.). 2017 4th IEEE Uttar Pradesh Section International Conference on Electrical, Computer and Electronics (UPCON), 568–572. https://doi.org/10.1109/UPCON.2017.8251112. [GS Search]
Sharma, S., Sharma, S., & Athaiya, A. (2017). Activation functions in neural networks. towards data science, 6(12), 310–316. [GS Search].
Sokkhey, P., & Okazaki, T. (2020). Study on dominant factor for academic performance prediction using feature selection methods. International Journal of Advanced Computer Science and Applications, 11(8), 492–502. https://doi.org/10.14569/IJACSA.2020.0110862. [GS Search]
Souza, V. F. d., & Santos, T. C. B. d. (2021). Processo de Mineração de Dados Educacionais aplicado na Previsão do Desempenho de Alunos: Uma comparação entre as Técnicas de Aprendizagem de Máquina e Aprendizagem Profunda. Revista Brasileira de Informática na Educação, 29, 519–546. https://doi.org/10.5753/rbie.2021.29.0.519. [GS Search]
Tatar, A. E., & Dü¸stegör, D. (2020). Prediction of Academic Performance at Undergraduate Graduation: Course Grades or Grade Point Average? Applied Sciences, 10(1414), 4967.https://doi.org/10.3390/app10144967. [GS Search]
Turabieh, H. (2019). Hybrid machine learning classifiers to predict student performance (2ª ed.). 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), 1–6.https://doi.org/10.1109/ICTCS.2019.8923093. [GS Search]
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need. https://doi.org/10.48550/ARXIV.1706.03762. [GS Search]
Xiao, J., & Zhou, Z. (2020). Research progress of RNN language model. 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), 1285–1288. https://doi.org/10.1109/ICAICA50127.2020.9182390. [GS Search]
Xu, P., Kumar, D., Yang, W., Zi, W., Tang, K., Huang, C., Cheung, J. C. K., Prince, S. J., & Cao, Y. (2021). Optimizing Deeper Transformers on Small Datasets. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2089–2102.https://doi.org/10.18653/v1/2021.acl-long.163. [GS Search]
Yang, F., & Li, F. (2018). Study on student performance estimation, student progress analysis, and student potential prediction based on data mining. Computers and Education, 123, 97–108.https://doi.org/10.1016/j.compedu.2018.04.006. [GS Search]
Zaffar, M., Hashmani, M., Savita, K., & Rizvi, S. (2018). A study of feature selection algorithms for predicting students academic performance. International Journal of Advanced Computer Science and Applications, 9(5), 541–549.https://doi.org/10.14569/IJACSA.2018.090569. [GS Search]
Zhang, L., & Li, K. F. (2018). Education Analytics: Challenges and Approaches (1ª ed.). 2018 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA), 193–198. https://doi.org/10.1109/WAINA.2018.00086. [GS Search]
Arquivos adicionais
Published
Como Citar
Issue
Section
Licença
Copyright (c) 2024 Lorran Santos Rodrigues, Marcos Santos, Carlos Francisco Simoes Gomes, Ricardo Choren, Ronaldo Goldschmidt, Saulo Barbará
Este trabalho está licenciado sob uma licença Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
