Detecção de Emoções na Aprendizagem de Programação: Os Efeitos de Usar Estimativas de Conhecimento em Modelos Livres de Sensores que Detectam a Confusão do Aluno
DOI:
https://doi.org/10.5753/rbie.2024.3437Keywords:
Detecção de confusão, Modelos livres de sensores, Aprendizado de máquina, Estimativas de conhecimento do aluno, Aprendizagem de programação de computadores, Regulação emocional na aprendizagem, Emoções na aprendizagemAbstract
A confusão é uma emoção provável de ocorrer em tarefas de aprendizagem de conteúdos complexos, como na aprendizagem de programação de computadores. Quando não regulada pelo aluno, a confusão pode afetar negativamente o aprendizado. Quando regulada, pode levar a aprendizagem a níveis mais profundos. O estudo descrito neste artigo buscou melhorar o desempenho de modelos livres de sensores que detectam a confusão do aluno enquanto envolvido em tarefas de aprendizagem de programação. Estes modelos são interessantes quando integrados a ferramentas de programação porque, ao detectar a confusão do aluno durante a aprendizagem, a ferramenta poderia intervir e auxiliar o aluno na regulação dessa emoção. Trabalhos relacionados treinaram modelos de detecção de confusão usando dados de interação do aluno com o ambiente de programação, como dados sobre movimentos de teclado e mouse. Nosso estudo levantou a hipótese que incorporar dados sobre estimativas de conhecimento do aluno aos dados de interação poderia melhorar o desempenho dos modelos. Nós comparamos o desempenho de modelos de aprendizado de máquina treinados com a abordagem da hipótese com modelos treinados com a abordagem dos trabalhos relacionados. Os modelos foram treinados com dados coletados de 62 alunos em aulas de programação ao longo de cinco meses. Os resultados apresentaram evidências positivas que apoiam nossa hipótese. Também discutimos cenários onde nossa abordagem é vantajosa, como o tamanho adequado dos segmentos de dados, os algoritmos com melhor desempenho e o poder de generalização dos modelos para alunos de diferentes níveis de ensino.
Downloads
Referências
Agarwal, D., Baker, R., & Muraleedharan, A. (2020). Dynamic knowledge tracing through data driven recency weights. Em A. N. Rafferty, J. Whitehill, V. Cavalli-Sforza & C. Romero (Ed.), Proceedings of The 13th International Conference on Educational Data Mining (EDM 2020) (pp. 725–729). [GS Search]
Alzoubi, O., D’Mello, S., & Calvo, R. A. (2012). Detecting Naturalistic Expressions of Nonbasic Affect Using Physiological Signals. IEEE Transactions on Affective Computing, 3(3), 298- 310. https://doi.org/10.1109/t-affc.2012.4. [GS Search]
Arguel, A., Lockyer, L., Kennedy, G., Lodge, J. M., & Pachman, M. (2019). Seeking optimal confusion: a review on epistemic emotion management in interactive digital learning environments. Interactive Learning Environments, 27(2), 200-210. https://doi.org/10.1080/10494820.2018.1457544. [GS Search]
Arroyo, I., Cooper, D., Burleson, W., Woolf, B., Muldner, K., & Christopherson, R. (2009). Emotion sensors go to school. Frontiers in Artificial Intelligence and Applications, (1), 17-24. https://doi.org/10.3233/978-1-60750-028-5-17. [GS Search]
Ausubel, D. P., Novak, J. D., & Hanesian, H. (1978). Educational Psychology: A Cognitive View. Holt, Rinehart; Winston. [GS Search]
Badrinath, A., Wang, F., & Pardos, Z. (2021). pybkt: An accessible python library of bayesian knowledge tracing models. Proceedings of the 14th International Conference on Educational Data Mining. https://doi.org/10.48550/arXiv.2105.00385. [GS Search]
Baker, R. S. d., Corbett, A., Gowda, S. M., Wagner, A. Z., MacLaren, B. A., Kauffman, L. R., Mitchell, A. P., & Giguere, S. (2010). Contextual slip and prediction of student performance after use of an intelligent tutor. User Modeling, Adaptation, and Personalization: 18th International Conference, UMAP 2010, 52-63. https://doi.org/10.1007/978-3-642-13470-8_7. [GS Search]
Baker, R. S. d., Pardos, Z., Gowda, S. M., Nooraei, B. B., & Heffernan, N. T. (2011). Ensembling predictions of student knowledge within intelligent tutoring systems. User Modeling, Adaption and Personalization: 19th International Conference, UMAP 2011, 13-24. https://doi.org/10.1007/978-3-642-22362-4_2. [GS Search]
Beck, J. E., & Chang, K.-m. (2007). Identifiability: A fundamental problem of student modeling. Em C. Conati, K. McCoy & G. Paliouras (Ed.), International Conference on User Modeling (pp. 137–146). https://doi.org/10.1007/978-3-540-73078-1_17. [GS Search]
Bosch, N., Chen, Y., & D’Mello, S. (2014). It’s written on your face: Detecting affective states from facial expressions while learning computer programming. Intelligent Tutoring Systems, 39-44. https://doi.org/10.1007/978-3-319-07221-0_5. [GS Search]
Bosch, N., & D’Mello, S. (2017). The affective experience of novice computer programmers. International Journal of Artificial Intelligence in Education, 27(1), 181-206. https://doi.org/10.1007/s40593-015-0069-5. [GS Search]
Bosch, N., D’Mello, S., & Mills, C. (2013). What emotions do novices experience during their first computer programming learning session? https://doi.org/10.1007/978-3-642-39112-5_2. [GS Search]
Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. Artificial Intelligence in Education, 40-51. https://doi.org/10.1007/978-3-319-61425-0_4. [GS Search]
Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32. https://doi.org/10.1023/A:1010933404324. [GS Search]
Calvo, R. A., & D’Mello, S. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing (TAC), 1(1), 18-37. https://doi.org/10.1109/T-AFFC.2010.1. [GS Search]
Chi, M. T., & Ohlsson, S. (2005). Complex Declarative Learning. Cambridge University Press. https://doi.org/10.1007/978-1-4419-1428-6_295. [GS Search]
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1), 37-46. https://doi.org/10.1177/001316446002000104. [GS Search]
Corbett, A., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4, 253-278. https://doi.org/10.1007/bf01099821. [GS Search]
Corbett, A., Kauffman, L., Maclaren, B., Wagner, A., & Jones, E. (2010). A Cognitive Tutor for genetics problem solving: Learning gains and student modeling. Journal of Educational Computing Research, 42(2), 219-239. https://doi.org/10.2190/EC.42.2.e. [GS Search]
Coto, M., Mora, S., Grass, B., & Murillo-Morera, J. (2021). Emotions and programming learning: systematic mapping. Computer Science Education, 32(1), 1-36. https://doi.org/10.1080/08993408.2021.1920816. [GS Search]
Craig, S. D., & et al. (2008a). Emote aloud during learning with AutoTutor: Applying the Facial Action Coding System to cognitive - Affective states during learning. Cognition and Emotion, 22(5), 777-788. https://doi.org/10.1080/02699930701516759. [GS Search]
de Oliveira Alves, M., Medeiros, F. P. A., & Melo, L. B. (2020). Levantamento do Estado da Arte sobre Aprendizagem baseada em Problemas na Educação a Distância e Híbrida. Anais do XXXI Simpósio Brasileiro de Informática na Educação, 61-71. https://doi.org/10.5753/cbie.sbie.2020.61. [GS Search]
Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation, 10(7), 1895-1923. https://doi.org/10.1162/089976698300017197. [GS Search]
D’Mello, S. (2020). Big data in the science of learning. Em Big data in psychological research (pp. 203-225). American Psychological Association. https://doi.org/10.1037/0000193-010. [GS Search]
D’Mello, S., & Calvo, R. A. (2013). Beyond the basic emotions: what should affective computing compute? CHI'13 Extended Abstracts on Human Factors in Computing Systems, 2287-2294. https://doi.org/10.1145/2468356.2468751. [GS Search]
D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145-157. https://doi.org/10.1016/j.learninstruc.2011.10.001. [GS Search]
D’Mello, S., & Graesser, A. C. (2014). Confusion. Em International Handbook of Emotions in Education (pp. 299-320). Routledge. https://doi.org/10.4324/9780203148211. [GS Search]
D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153-170. https://doi.org/10.1016/j.learninstruc.2012.05.003. [GS Search]
D’Mello, S., Person, N., & Lehman, B. (2009). Antecedent-consequent relationships and cyclical patterns between affective states and problem solving outcomes. Frontiers in Artificial Intelligence and Applications, 200, 57-64. https://doi.org/10.3233/978-1-60750-028-5-57. [GS Search]
Felipe, D. A. M., Gutierrez, K. I. N., Quiros, E. C. M., & Vea, L. A. (2012). Towards the development of intelligent agent for novice c/c++ programmers through affective analysis of event logs. Proc. Int. MultiConference Eng. Comput. Sci, 1, 2012. [GS Search]
Fino, C. N. (2001). Vygotsky e a Zona de Desenvolvimento Proximal (ZDP): três implicações pedagógicas. Revista Portuguesa de educação, 14, 273-291. [GS Search]
Gong, Y., Beck, J. E., & Heffernan, N. T. (2010). Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. Intelligent Tutoring Systems: 10th International Conference, ITS 2010, Pittsburgh, PA, USA, June 14-18, 2010, Proceedings, Part I 10, 35–44. https://doi.org/10.1007/978-3-642-13388-6_8. [GS Search]
Gowda, S. M., Rowe, J. P., de Baker, R. S. J., Chi, M., & Koedinger, K. R. (2011). Improving Models of Slipping, Guessing, and Moment-By-Moment Learning with Estimates of Skill Difficulty. EDM, 2011, 199-208. [GS Search]
Graesser, A., Chipman, P., King, B., McDaniel, B., & D’Mello, S. (2007). Emotions and learning with autotutor. Proceedings of the 2007 Conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, 569-571. [GS Search]
Grafsgaard, J. F., Boyer, K. E., & Lester, J. C. (2011). Predicting facial indicators of confusion with hidden markov models. Affective Computing and Intelligent Interaction, 6974 LNCS(PART 1), 97-106. https://doi.org/10.1007/978-3-642-24600-5_13. [GS Search]
Halgren, E., & et al. (2002). N400-like Magnetoencephalography Responses Modulated by Semantic Context, Word Frequency, and Lexical Class in Sentences. NeuroImage, 17(3), 1101-1116. https://doi.org/10.1006/nimg.2002.1268. [GS Search]
Hess, U. (2003). Now you see it, now you don't-the confusing case of confusion as an emotion: Commentary on Rozin and Cohen (2003). Emotion, 3(1), 76-80. https://doi.org/10.1037/1528-3542.3.1.76. [GS Search]
Izard, C. E. (2010). The many meanings/aspects of emotion: Definitions, functions, activation, and regulation. Emotion Review, 2(4), 363-370. https://doi.org/10.1177/1754073910374661. [GS Search]
Jófili, Z. (2002). Piaget, Vygotsky, Freire e a construção do conhecimento na escola. Educação: teorias e práticas, 2(2), 191-208. [GS Search]
Kasurinen, J., & Nikula, U. (2009). Estimating programming knowledge with Bayesian knowledge tracing. ACM SIGCSE Bulletin, 41(3), 313-317. https://doi.org/10.1145/1595496.1562972. [GS Search]
Kautzmann, T. R., Ramos, G. d. O., & Jaques, P. A. (2022). O uso de estimativas de conhecimento do aluno em programação de computadores em modelos de detecção da emoção confusão livres de sensores. Anais do XXXIII Simpósio Brasileiro de Informática na Educação, 1196-1208. https://doi.org/10.5753/sbie.2022.225768. [GS Search]
Keltner, D., & Shiota, M. N. (2003). New displays and new emotions: A commentary on Rozin and Cohen (2003). Emotion, 3(1), 86-91. https://doi.org/10.1037/1528-3542.3.1.86. [GS Search]
Khajah, M., Lindsey, R. V., & Mozer, M. C. (2016). How deep is knowledge tracing? arXiv preprint arXiv:1604.02416. https://doi.org/10.48550/arXiv.1604.02416. [GS Search]
Knottnerus, J. A., & Tugwell, P. (2010). Real world research. Journal of clinical epidemiology, 63(10), 1051-1052. https://doi.org/10.1016/j.jclinepi.2010.08.001. [GS Search]
Koedinger, K. R., Corbett, A., & Perfetti, C. (2012). The Knowledge-Learning-Instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive science, 36(5), 757-798. https://doi.org/10.1111/j.1551-6709.2012.01245.x. [GS Search]
Kubat, M. (2017). An introduction to machine learning (Vol. 2). Springer. https://doi.org/10.1007/978-3-319-63913-0. [GS Search]
Lee, D. M. C., Rodrigo, M. M. T., Baker, R. S., Sugay, J. O., & Coronel, A. (2011). Exploring the relationship between novice programmer confusion and achievement. International Conference on Affective Computing and Intelligent Interaction, 175-184. https://doi.org/10.1007/978-3-642-24600-5_21. [GS Search]
Lehman, B., D’Mello, S., Strain, A., Mills, C., Gross, M., Dobbins, A., Wallace, P., Millis, K., & Graesser, A. (2013). Inducing and tracking confusion with contradictions during complex learning. International Journal of Artificial Intelligence in Education, 22, 85-105. https://doi.org/10.3233/JAI-130025. [GS Search]
Lin, C., & Chi, M. (2016). Intervention-bkt: incorporating instructional interventions into bayesian knowledge tracing. Intelligent Tutoring Systems: 13th International Conference, ITS 2016, Zagreb, Croatia, June 7-10, 2016. Proceedings 13, 208–218. https://doi.org/10.1007/978-3-319-39583-8_20. [GS Search]
MacDowell, K. A., & Mandler, G. (1989). Constructions of emotion: Discrepancy, arousal, and mood. Motivation and Emotion, 13(2), 105-124. https://doi.org/10.1007/bf00992957. [GS Search]
Mao, Y. (2018). Deep Learning vs. Bayesian Knowledge Tracing: Student Models for Interventions. Journal of educational data mining, 10(2). https://doi.org/10.5281/zenodo.3554691. [GS Search]
Mayer, J. D., Salovey, P., & Caruso, D. R. (2004). Emotional intelligence: Theory, findings, and implications. Psychological inquiry, 15(3), 197-215. [GS Search]
McDaniel, B., & et al. (2007). Facial Features for Affective State Detection in Learning Environments. Proceedings of the Annual Meeting of the Cognitive Science Society. [GS Search]
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157. https://doi.org/10.1007/bf02295996. [GS Search]
Molnar, C., Freiesleben, T., König, G., Herbinger, J., Reisinger, T., Casalicchio, G., Wright, M. N., & Bischl, B. (2023). Relating the partial dependence plot and permutation feature importance to the data generating process. World Conference on Explainable Artificial Intelligence, 456-479. https://doi.org/10.1007/978-3-031-44064-9_24. [GS Search]
Moon, T. K. (1996). The expectation-maximization algorithm. IEEE Signal processing magazine, 13(6), 47-60. https://doi.org/10.1109/79.543975. [GS Search]
Moors, A., Ellsworth, P. C., Scherer, K. R., & Frijda, N. H. (2013). Appraisal theories of emotion: State of the art and future development. Emotion Review, 5(2), 119-124. https://doi.org/10.1177/1754073912468165. [GS Search]
Nagatani, K., Zhang, Q., Sato, M., Chen, Y.-Y., Chen, F., & Ohkuma, T. (2019). Augmenting knowledge tracing by considering forgetting behavior. The world wide web conference, 3101-3107. https://doi.org/10.1145/3308558.3313565. [GS Search]
Pardos, Z., & Heffernan, N. (2010). Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm. Educational Data Mining 2010. [GS Search]
Pardos, Z., & Heffernan, N. T. (2011). KT-IDEM: Introducing item difficulty to the knowledge tracing model. User Modeling, Adaption and Personalization: 19th International Conference, UMAP 2011, Girona, Spain, July 11-15, 2011. Proceedings 19, 243-254. https://doi.org/10.1007/978-3-642-22362-4_21. [GS Search]
Pekrun, R. (2011). Emotions as drivers of learning and cognitive development. New perspectives on affect and learning technologies, 3, 23-39. https://doi.org/10.1007/978-1-4419-9625-1_3. [GS Search]
Pekrun, R., & Stephens, E. J. (2012). Academic emotions. Em APA Educational Psychology Handbook, Vol 2: Individual Differences and Cultural and Contextual Factors (pp. 3-31). American Psychological Association. https://doi.org/10.1037/13274-001. [GS Search]
Pekrun, R. (2014). Emotions and learning. [GS Search]
Pelánek, R. (2017). Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Modeling and User-Adapted Interaction, 27, 313-350. https://doi.org/10.1007/s11257-017-9193-2. [GS Search]
Pelizzari, A., KriegL, M. d. L., Baron, M. P., Finck, N. T. L., & Dorocinski, S. I. (2002). Teoria da aprendizagem significativa segundo Ausubel. revista PEC, 2(1), 37-42. [GS Search]
Penmetsa, P. (2021). Investigate effectiveness of code features in knowledge tracing task on novice programming course. North Carolina State University. [GS Search]
Raposo, A. C., Maranhão, D., & Neto, C. S. (2019). Analise do modelo bkt na avaliacao da curva de aprendizagem de alunos de algoritmos. Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), 30(1), 479. https://doi.org/10.5753/cbie.sbie.2019.479. [GS Search]
Raschka, S. (2018). Model evaluation, model selection, and algorithm selection in machine learning. https://doi.org/10.48550/arXiv.1811.12808. [GS Search]
Rodrigo, M. M. T., Baker, R. S. J., & Nabos, J. Q. (2010). The relationships between sequences of affective states and learner achievement. Proceedings of the 18th International Conference on Computers in Education, 56-60. [GS Search]
Rozin, P., & Cohen, A. B. (2003). High frequency of facial expressions corresponding to confusion, concentration, and worry in an analysis of naturally occurring facial expressions of Americans. Emotion, 3(1), 68-75. https://doi.org/10.1037/1528-3542.3.1.68. [GS Search]
Scherer, K. R. (2005). What are emotions? and how can they be measured? Social science information, 44(4), 695-729. https://doi.org/10.1177/0539018405058216. [GS Search]
Shwartz-Ziv, R., & Armon, A. (2022). Tabular data: Deep learning is not all you need. Information Fusion, 81, 84-90. https://doi.org/10.1016/j.inffus.2021.11.011. [GS Search]
Silvia, P. J. (2010). Confusion and interest: The role of knowledge emotions in aesthetic experience. Psychology of Aesthetics, Creativity, and the Arts, 4(2), 75-80. https://doi.org/10.1037/a0017081. [GS Search]
Slater, S., & Baker, R. S. (2018). Degree of error in Bayesian knowledge tracing estimates from differences in sample sizes. Behaviormetrika, 45(2), 475-493. https://doi.org/10.1007/s41237-018-0072-x. [GS Search]
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1988). Cognitive Architecture and Instructional Design. Educational Psychology Review, 10(3), 251-296. https://doi.org/10.1023/A:1022193728205. [GS Search]
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive Architecture and Instructional Design: 20 Years Later. Educational Psychology Review, 10, 251-296. https://doi.org/10.1007/s10648-019-09465-5. [GS Search]
Taksic, V. (2000). Convergent and divergent validity of the Emotional Skills and Competence Questionnaire. Comunicación presentada en XII Days of Psychology, Zadar, Croacia. [GS Search]
Tiam-Lee, T. J., & Sumi, K. (2018). Adaptive feedback based on student emotion in a system for programming practice. Intelligent Tutoring Systems, 243-255. https://doi.org/10.1007/978-3-319-91464-0_24. [GS Search]
Tiam-Lee, T. J., & Sumi, K. (2019). Analysis and prediction of student emotions while doing programming exercises. International Conference on Intelligent Tutoring Systems, 24-33. https://doi.org/10.1007/978-3-030-22244-4_4. [GS Search]
VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 209-249. https://doi.org/10.1207/s1532690xci2103_01. [GS Search]
Vea, L., & Rodrigo, M. M. (2017). Modeling negative affect detector of novice programming students using keyboard dynamics and mouse behavior. Trends in Artificial Intelligence: PRICAI 2016 Workshops, 127-138. https://doi.org/10.1007/978-3-319-60675-0_11. [GS Search]
Wang, S., Han, Y., Wu, W., & Hu, Z. (2017). Modeling student learning outcomes in studying programming language course. 2017 Seventh International Conference on Information Science and Technology (ICIST), 263-270. https://doi.org/10.1109/icist.2017.7926768. [GS Search]
Wang, Y., Heffernan, N. T., & Heffernan, C. (2015). Towards better affect detectors: effect of missing skills, class features and common wrong answers. Proceedings of the fifth international conference on learning analytics and knowledge, 31-35. https://doi.org/10.1145/2723576.2723618. [GS Search]
Williams, R. J., & Zipser, D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2), 270-280. https://doi.org/10.1162/neco.1989.1.2.270. [GS Search]
Wong, T. T. (2015). Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recognition, 48, 2839-2846. https://doi.org/10.1016/j.patcog.2015.03.009. [GS Search]
Yang, T.-Y., Baker, R. S., Studer, C., Heffernan, N., & Lan, A. S. (2019). Active learning for student affect detection. Proceedings of the 12th International Conference on Educational Data Mining, 208-217. [GS Search]
Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013). Individualized bayesian knowledge tracing models. Artificial Intelligence in Education: 16th International Conference, AIED 2013, 171-180. https://doi.org/10.1007/978-3-642-39112-5_18. [GS Search]
Arquivos adicionais
Published
Como Citar
Issue
Section
Licença
Copyright (c) 2024 Tiago R. Kautzmann, Gabriel de O. Ramos, Patricia A. Jaques
Este trabalho está licenciado sob uma licença Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
