Transformadores para predecir el rendimiento académico en educación Primaria y Secundaria
DOI:
https://doi.org/10.5753/rbie.2024.3661Keywords:
Desempeño académico, Transformers, EDMAbstract
La predicción del rendimiento académico tiene un gran potencial en los esfuerzos proactivos de las escuelas para identificar a los estudiantes en riesgo de fracaso. Este estudio fue motivado por la brecha en los estudios de predicción del rendimiento académico centrados en la educación media y primaria que exploran modelos de aprendizaje profundo. Esto llevó a la recopilación de un conjunto de datos de estudiantes de educación básica en Matemáticas y Portugués de dos redes educativas distintas, lo que permitió comparaciones entre diferentes años escolares, años académicos y redes educativas. Se contrastaron los desempeños de modelos basados en arquitecturas Transformer con modelos más establecidos como XGBoost y un modelo de red neuronal más simple. Los resultados mostraron que los Transformers tuvieron un desempeño interesante en la tarea de predicción del rendimiento académico, especialmente con un mayor número de evaluaciones. Sin embargo, XGBoost logró alcanzar un alto rendimiento más temprano en el período escolar. Una ventaja de los Transformers es su flexibilidad en el entrenamiento, lo que les permite manejar conjuntos de datos semi-estructurados sin necesidad de preprocesamiento. En última instancia, esta investigación contribuye al desarrollo de métodos que pueden identificar tempranamente a los estudiantes en riesgo de fracaso, ofreciendo la oportunidad de intervención y apoyo oportunos. Esto puede tener un impacto positivo en la educación de los estudiantes y en la sociedad en su conjunto, mitigando pérdidas y promoviendo una educación de calidad.
Descargas
Citas
Almayan, H., & Al Mayyan, W. (2016). Improving accuracy of students’ final grade prediction model using PSO (2ª ed.). Decision Analytics, 35–39.https://doi.org/10.1109/INFOCOMAN.2016.7784211. [GS Search]
Amra, I., & Maghari, A. (2017). Students performance prediction using KNN and Naïve Bayesian
(8ª ed.). International Conference on Information Technology (ICIT), 909–913. https://doi.org/10.1109/ICITECH.2017.8079967. [GS Search]
Athani, S. S., Kodli, S. A., Banavasi, M. N., & Hiremath, P. G. S. (2017). Student academic
performance and social behavior predictor using data mining techniques (s.n). 2017 International Conference on Computing, Communication and Automation (ICCCA), 170–174. https://doi.org/10.1109/CCAA.2017.8229794. [GS Search]
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to
Align and Translate.https://doi.org/10.48550/ARXIV.1409.0473. [GS Search]
Barros, T. M., Souza Neto, P. A., Silva, I., & Guedes, L. A. (2019). Predictive Models for Imbalanced Data: A School Dropout Perspective. Education Sciences, 9(44), 275.https://doi.org/10.3390/educsci9040275. [GS Search]
Blasi, A. (2017). Performance increment of high school students using ANN model and sa algorithm. Journal of Theoretical and Applied Information Technology, 95(11), 2417–2425.[GS Search]
Bonaccorso, G. (2017). Machine learning algorithms. Packt Publishing Ltd. [GS Search]
Chen, Q., Zhao, H., Li, W., Huang, P., & Ou, W. (2019). Behavior Sequence Transformer for ECommerce Recommendation in Alibaba. Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. https://doi.org/10.1145/3326937.3341261. [GS Search]
Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. https://doi.org/10.1145/2939672.2939785. [GS Search]
Chen, W., Brinton, C. G., Cao, D., Mason-Singh, A., Lu, C., & Chiang, M. (2019). Early Detection Prediction of Learning Outcomes in Online Short-Courses via Learning Behaviors. IEEE Transactions on Learning Technologies, 12(1), 44–58. https://doi.org/10.1109/TLT.2018.2793193. [GS Search]
Chollet, F., et al. (2015). Keras.[GS Search]
Cornell-Farrow, S., & Garrard, R. (2020). Machine learning classifiers do not improve the prediction of academic risk: Evidence from Australia. Communications in Statistics Case Studies Data Analysis and Applications, 6(2), 228–246. https://doi.org/10.1080/23737484.2020.1752849. [GS Search]
Cortez, P., & Silva, A. M. G. (2008). Using data mining to predict secondary school student performance (s.n) [Accepted: 2008-08-20T18:31:05Z]. Proceedings of 5th Annual Future Business Technology Conference. https://repositorium.sdum.uminho.pt/handle/1822/8024. [GS Search]
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. https://doi.org/10.48550/ARXIV.1810.04805. [GS Search]
Elsayed, S., Thyssens, D., Rashed, A., Schmidt-Thieme, L., & Jomaa, H. S. (2021). Do We Really Need Deep Learning Models for Time Series Forecasting? CoRR, abs/2101.02118. https://arxiv.org/abs/2101.02118. [GS Search]
Fávero, L. P., & Belfiore, P. (2017). Manual de análise de dados: estatística e modelagem multivariada com Excel®, SPSS® e Stata®. Elsevier Brasil. [GS Search]
Fernandes, E., Holanda, M., Victorino, M., Borges, V., Carvalho, R., & Erven, G. (2019). Educational data mining: Predictive analysis of academic performance of public school students in the capital of Brazil. Journal of Business Research, 94, 335–343. https://doi.org/10.1016/j.jbusres.2018.02.012. [GS Search]
García-González, J., & Skrita, A. (2019). Predicting academic performance based on students’
family environment: Evidence for Colombia using classification trees. Psychology, Society
and Education, 11(3), 299–311. https://doi.org/10.25115/psye.v11i3.2056. [GS Search]
Gil, J., Delima, A., & Vilchez, R. (2020). Predicting students’ dropout indicators in public school using data mining approaches. International Journal of Advanced Trends in Computer Science and Engineering, 9(1), 774–778. https://doi.org/10.30534/ijatcse/2020/110912020. [GS Search]
Goldschmidt, R., Passos, E., & Bezerra, E. (2015). Data mining. Elsevier Brasil. [GS Search]
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press. [GS Search]
Gottardo, E., Kaestner, C. A. A., & Noronha, R. V. (2014). Estimativa de Desempenho Acadêmico de Estudantes: Análise da Aplicação de Técnicas de Mineração de Dados em Cursos a Distância. Revista Brasileira de Informática na Educacação, 22(1). https://doi.org/10.5753/rbie.2014.22.01.45. [GS Search]
Guo, B., Zhang, R., Xu, G., Shi, C., & Yang, L. (2016). Predicting Students Performance in Educational Data Mining (s.n). 2015 International Symposium on Educational Technology (ISET), 125–128. https://doi.org/10.1109/ISET.2015.33. [GS Search]
Guyon, I. (1991). Neural networks and applications tutorial. Physics Reports, 207(3-5), 215–259. [GS Search]
H. Alamri, L., S. Almuslim, R., S. Alotibi, M., K. Alkadi, D., Ullah Khan, I., & Aslam, N. (2020).
Predicting Student Academic Performance using Support Vector Machine and Random Forest (3ª ed.) [01/08/2021]. 2020 3rd International Conference on Education Technology Management, (s.n), 100–107. https://doi.org/10.1145/3446590.3446607. [GS Search]
Hellas, A., Ihantola, P., Petersen, A., Ajanovski, V. V., Gutica, M., Hynninen, T., Knutas, A., Leinonen, J., Messom, C., & Liao, S. N. (2018). Predicting academic performance: a systematic literature review (23ª ed.) [01/08/2021]. Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, (s.n), 175–199. https://doi.org/10.1145/3293881.3295783. [GS Search]
Hernández-Blanco, A., Herrera-Flores, B., Tomás, D., & Navarro-Colorado, B. (2019). A Systematic Review of Deep Learning Approaches to Educational Data Mining. Complexity, 2019, e1306039. https://doi.org/10.1155/2019/1306039. [GS Search]
Hussain, S., Muhsin, Z., Salal, Y., Theodorou, P., Kurtoglu, F., & Hazarika, G. (2019). Prediction model on student performance based on internal assessment using deep learning. International Journal of Emerging Technologies in Learning, 14(8), 4–22. https://doi.org/10.3991/ijet.v14i08.10001. [GS Search]
Hussain, S., & Khan, M. Q. (2021). Student-Performulator: Predicting Students’ Academic Performance at Secondary and Intermediate Level Using Machine Learning. Annals of Data Science, s.n, 1–19.[GS Search].
Imran, M., Latif, S., Mehmood, D., & Shah, M. (2019). Student academic performance prediction using supervised learning techniques. International Journal of Emerging Technologies in Learning, 14(14), 92–104. https://doi.org/10.3991/ijet.v14i14.10310. [GS Search]
Kim, B.-H., Vizitei, E., & Ganapathi, V. (2019). Domain Adaptation for Real-Time Student Performance Prediction. https://doi.org/10.48550/arXiv.1809.06686. [GS Search]
Kim, B., Vizitei, E., & Ganapathi, V. (2018). GritNet: Student Performance Prediction with Deep
Learning [01/08/2021]. CoRR, abs/1804.07405.
https://doi.org/10.48550/arXiv.1804.07405”>
https://doi.org/10.48550/arXiv.1804.07405. [GS Search]
Lee, S., & Chung, J. Y. (2019). The Machine Learning-Based Dropout Early Warning System
for Improving the Performance of Dropout Prediction. Applied Sciences, 9(1515), 3093. https://doi.org/10.3390/app9153093. [GS Search]
Livingstone, D. J. (2008). Artificial neural networks: methods and applications. Springer. [GS Search]
Liz-Domínguez, M., Caeiro-Rodríguez, M., Llamas-Nistal, M., & Mikic-Fonte, F. A. (2019). Systematic Literature Review of Predictive Analysis Tools in Higher Education. Applied Sciences, 9(2424), 5569.https://doi.org/10.3390/app9245569. [GS Search]
Lu, H., & Yuan, J. (2018). Student performance prediction model based on discriminative feature selection. International Journal of Emerging Technologies in Learning, 13(10), 55–68.https://doi.org/10.3991/ijet.v13i10.9451. [GS Search]
Márquez-Vera, C., Cano, A., Romero, C., & Ventura, S. (2013). Predicting student failure at school using genetic programming and different data mining approaches with high dimensional and imbalanced data. Applied Intelligence, 38(3), 315–330. https://doi.org/10.1007/s10489-012-0374-8. [GS Search]
Mitchell, T. M., et al. (2007). Machine learning (Vol. 1). McGraw-hill New York. [GS Search]
Namoun, A., & Alshanqiti, A. (2021). Predicting Student Performance Using Data Mining and Learning Analytics Techniques: A Systematic Literature Review. Applied Sciences, 11(11), 237. https://doi.org/10.3390/app11010237. [GS Search]
Niu, Z., Zhong, G., & Yu, H. (2021). A review on the attention mechanism of deep learning. Neurocomputing, 452, 48–62. https://doi.org/10.1016/j.neucom.2021.03.091. [GS Search]
Oliphant, T. E. (2007). Python for Scientific Computing. Computing in Science Engineering, 9(3),10–20. https://doi.org/10.1109/MCSE.2007.58. [GS Search]
Orooji, M., & Chen, J. (2019). Predicting Louisiana Public High School Dropout through Imbalanced Learning Techniques [01/08/2021]. CoRR, abs/1910.13018. http://arxiv.org/abs/1910.13018. [GS Search]
Pandey, M., Fernandez, M., Gentile, F., Isayev, O., Tropsha, A., Stern, A. C., & Cherkasov, A. (2022). The transformational role of GPU computing and deep learning in drug discovery. Nature Machine Intelligence, 4(3), 211–221. https://doi.org/10.1038/s42256-022-00463-x. [GS Search]
Qazdar, A., Er-Raha, B., Cherkaoui, C., & Mammass, D. (2019). A machine learning algorithm framework for predicting students performance: A case study of baccalaureate students in Morocco. Education and Information Technologies, 24(6), 3577–3589. https://doi.org/10.1007/s10639-019-09946-8. [GS Search]
Razaque, A., & Alajlan, A. (2020). Supervised machine learning model-based approach for performance prediction of students. Journal of Computer Science, 16(8), 1150–1162. https://doi.org/10.3844/jcssp.2020.1150.1162. [GS Search]
Rodrigues, L. S., dos Santos, M., Costa, I., & Moreira, M. A. L. (2022). Student Performance Prediction on Primary and Secondary Schools-A Systematic Literature Review. Procedia Computer Science, 214, 680–687. https://doi.org/10.1016/j.procs.2022.11.229. [GS Search]
Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (2010). Handbook of educational data mining. CRC press. [GS Search]
Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6), 386.https://doi.org/10.1037/h0042519. [GS Search]
Rovira, S., Puertas, E., & Igual, L. (2017). Data-driven system to predict academic grades and dropout [01/08/2021]. PLOS ONE, 12(2), 1–21. https://doi.org/10.1371/journal.pone.0171207. [GS Search]
Roy, S., & Garg, A. (2017). Predicting academic performance of student using classification techniques (4ª ed.). 2017 4th IEEE Uttar Pradesh Section International Conference on Electrical, Computer and Electronics (UPCON), 568–572. https://doi.org/10.1109/UPCON.2017.8251112. [GS Search]
Sharma, S., Sharma, S., & Athaiya, A. (2017). Activation functions in neural networks. towards data science, 6(12), 310–316. [GS Search].
Sokkhey, P., & Okazaki, T. (2020). Study on dominant factor for academic performance prediction using feature selection methods. International Journal of Advanced Computer Science and Applications, 11(8), 492–502. https://doi.org/10.14569/IJACSA.2020.0110862. [GS Search]
Souza, V. F. d., & Santos, T. C. B. d. (2021). Processo de Mineração de Dados Educacionais aplicado na Previsão do Desempenho de Alunos: Uma comparação entre as Técnicas de Aprendizagem de Máquina e Aprendizagem Profunda. Revista Brasileira de Informática na Educação, 29, 519–546. https://doi.org/10.5753/rbie.2021.29.0.519. [GS Search]
Tatar, A. E., & Dü¸stegör, D. (2020). Prediction of Academic Performance at Undergraduate Graduation: Course Grades or Grade Point Average? Applied Sciences, 10(1414), 4967.https://doi.org/10.3390/app10144967. [GS Search]
Turabieh, H. (2019). Hybrid machine learning classifiers to predict student performance (2ª ed.). 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), 1–6.https://doi.org/10.1109/ICTCS.2019.8923093. [GS Search]
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need. https://doi.org/10.48550/ARXIV.1706.03762. [GS Search]
Xiao, J., & Zhou, Z. (2020). Research progress of RNN language model. 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), 1285–1288. https://doi.org/10.1109/ICAICA50127.2020.9182390. [GS Search]
Xu, P., Kumar, D., Yang, W., Zi, W., Tang, K., Huang, C., Cheung, J. C. K., Prince, S. J., & Cao, Y. (2021). Optimizing Deeper Transformers on Small Datasets. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2089–2102.https://doi.org/10.18653/v1/2021.acl-long.163. [GS Search]
Yang, F., & Li, F. (2018). Study on student performance estimation, student progress analysis, and student potential prediction based on data mining. Computers and Education, 123, 97–108.https://doi.org/10.1016/j.compedu.2018.04.006. [GS Search]
Zaffar, M., Hashmani, M., Savita, K., & Rizvi, S. (2018). A study of feature selection algorithms for predicting students academic performance. International Journal of Advanced Computer Science and Applications, 9(5), 541–549.https://doi.org/10.14569/IJACSA.2018.090569. [GS Search]
Zhang, L., & Li, K. F. (2018). Education Analytics: Challenges and Approaches (1ª ed.). 2018 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA), 193–198. https://doi.org/10.1109/WAINA.2018.00086. [GS Search]
Archivos adicionales
Published
Cómo citar
Issue
Section
Licencia
Derechos de autor 2024 Lorran Santos Rodrigues, Marcos Santos, Carlos Francisco Simoes Gomes, Ricardo Choren, Ronaldo Goldschmidt, Saulo Barbará
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-SinDerivadas 4.0.
