Considering personality and the transition of emotions to improve data mining-based emotion detection
DOI:
https://doi.org/10.5753/rbie.2020.28.0.749Keywords:
Sensor-free Affect Detection, Personality, Transition of Emotions, Educational Data Mining, Step-Based Intelligent Tutoring SystemsAbstract
This paper uses data mining, with data coming from the interaction of the students in a step-based ITS, to detect four learning emotions: confusion, engagement, frustration, and boredom. Unlike related works, our model aims to verify whether the students’ personality can improve the precision of the detection. Besides personality, we have also considered data from the transition of emotions. For that, the emotion labels were obtained through an annotation protocol which allows the capture of transitions of the students’ emotions. As results, we have identified that only the engagement detector, trained with personality data of the students, obtained a small precision improvement in the detection. However, with the use of a feature selection algorithm, it was possible to verify that among 348 available features, only ten were selected, including personality data. With the combination of personality data, emotion transitions, and logs captured from a step-based ITS, it was possible to reach an accuracy of K = .633 and A0 = .846 in detecting engagement. This being the minimum value required for human encoders in emotion detection protocols.
Downloads
References
Andres, J. M. A. L., Ocumpaugh, J., Baker, R. S., Slater, S., Paquette, L., Jiang, Y., . . . others (2019). Affect sequences and learning in betty’s brain. In International learning analytics and knowledge conference (pp. 383–390). [DOI: 10.1145/3303772.3303807] [GS Search]
Arroyo, I., Woolf, B. P., Burelson, W., Muldner, K., Rai, D., & Tai, M. (2014). A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. International Journal of Artificial Intelligence in Education, 24(4), 387–426. [DOI: 10.1007/s40593-014-0023-y] [GS Search]
Azevedo, O., de Morais, F., & Jaques, P. A. (2018). Exploring gamification to prevent gaming the system and help refusal in tutoring systems. In European conference on technology enhanced learning (pp. 231–244). [DOI: 10.1007/978-3-319-98572-518] [GS Search]
Azevedo, R., & Aleven, V. (2013). International handbook of metacognition and learning technologies (Vol. 26). Springer. [GS Search]
Baker, R. S., Gowda, S. M., Wixon, M., Kalka, J., Wagner, A. Z., Salvi, A., . . . Rossi, L. (2012). Towards sensor-free affect detection in cognitive tutor algebra. International Conference on Educational Data Mining. [GS Search]
Barbosa, A. A. G. (2009). Modelo hierárquico de fobias infanto-juvenis: testagem e relação com os estilos maternos. Unpublished doctoral dissertation, PhD thesis, Psicologia Social (UFRN), João Pessoa. [GS Search]
Bosch, N., & D’Mello, S. (2017). The affective experience of novice computer programmers. International Journal of Artificial Intelligence in Education, 27(1), 181–206. [DOI: 10.1007/s40593-015-0069-5] [GS Search]
Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. In International conference on artificial intelligence in education (pp. 40–51). [DOI: 10.1007/978-3-319-61425-0_4] [GS Search]
Botelho, A. F., Baker, R. S., Ocumpaugh, J., & Heffernan, N. T. (2018). Studying affect dynamics and chronometry using sensor-free detectors. In International conference on educational data mining. ERIC. [GS Search]
Calvo, R. A., & D’Mello, S. K. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1), 18-37. [DOI: 10.1109/T-AFFC.2010.1] [GS Search]
Canli, T., Zhao, Z., Desmond, J. E., Kang, E., Gross, J., & Gabrieli, J. D. (2001). An fmri study of personality influences on brain reactivity to emotional stimuli. Behavioral neuroscience, 115(1), 33. [DOI: 10.1037/0735-7044.115.1.33] [GS Search]
D’Mello, S. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105(4), 1082. [DOI: 10.1037/a0032674] [GS Search]
D’Mello, S., & Calvo, R. A. (2013). Beyond the basic emotions: what should affective computing compute? In Chi’13 extended abstracts on human factors in computing systems (pp. 2287– 2294). [DOI: 10.1145/2468356.2468751] [GS Search]
Douglas, H. E., Bore, M., & Munro, D. (2016). Coping with university education: The relationships of time management behaviour and work engagement with the five factor model aspects. Learning and individual Differences, 45, 268–274. [DOI: 10.1016/j.lindif.2015.12.004] [GS Search]
D’Mello, S. (2012). Monitoring affective trajectories during complex learning. In Encyclopedia of the sciences of learning (pp. 2325–2328). Springer. [GS Search]
D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. [DOI: 10.1016/j.learninstruc.2011.10.001] [GS Search]
D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153 - 170. [DOI: 10.1016/j.learninstruc.2012.05.003] [GS Search]
Fredrickson, B. L. (1998). What good are positive emotions? Review of General Psychology, 2(3), 300-319. [DOI: 10.1037/1089-2680.2.3.300] [GS Search]
Graesser, A., & D’Mello, S. K. (2011). Theoretical perspectives on affect and deep learning. In New perspectives on affect and learning technologies (pp. 11–21). Springer. [DOI: 10.1007/978-1-4419-9625-1_2] [GS Search]
Harley, J. M. (2016). Measuring emotions: a survey of cutting edge methodologies used in computer-based learning environment research. In Emotions, technology, design, and learning (pp. 89–114). Elsevier. [DOI: 10.1016/B978-0-12-801856- 9.00005-0] [GS Search]
Jaques, P. A., & Nunes, M. A. S. N. (2019). Computação afetiva aplicada à educação. Informática na Educação: técnicas e tecnologias computacionais, 3. Retrieved from https://ieducacao.ceie-br.org/computacaoafetiva/ [GS Search]
Jaques, P. A., Seffrin, H., Rubi, G., Morais, F., Ghilardi, C., Bittencourt, I. I., & Isotani, S. (2013). Rule-based expert systems to support step-by-step guidance in algebraic problem solving: The case of the tutor PAT2math. Expert Systems with Applications, 40(14), 5456–5465. [DOI: 10.1016/j.eswa.2013.04.004] [GS Search]
Jiang, Y., Bosch, N., Baker, R. S., Paquette, L., Ocumpaugh, J., Andres, J. M. A. L., . . . Biswas, G. (2018). Expert feature-engineering vs. deep neural networks: which is better for sensor-free affect detection? In International conference on artificial intelligence in education (pp. 198–211). [DOI: 10.1007/978-3-319-93843-1_15] [GS Search]
Komarraju, M., & Karau, S. J. (2005). The relationship between the big five personality traits and academic motivation. Personality and individual differences, 39(3), 557–567. [DOI: 10.1016/j.paid.2005.02.013] [GS Search]
Morais, F., da Silva, J., Reis, H., Isotani, S., & Jaques, P. (2017). Computação afetiva aplicada à educação: uma revisão sistemática das pesquisas publicadas no Brasil. In Brazilian symposium on computers in education (simpósio brasileiro de informática na educação- sbie)(Vol.28,p.163).[DOI: 10.5753/cbie.sbie.2017.163] [GS Search]
Morais, F., & Jaques, P. (2013). Avaliação de usabilidade do sistema tutor inteligente PAT2Math. RENOTE-Revista Novas Tecnologias na Educação, 11(3). [DOI: 10.22456/1679-1916.44929] [GS Search]
Morais, F., & Jaques, P. A. (2017). PAT2Math + Handwriting: Evoluindo Sistemas Tutores de Matemática com reconhecimento da escrita à mão. In Simpósio brasileiro de informática na educação (sbie) (Vol. 28). [DOI: 10.5753/cbie.sbie.2017.1237] [GS Search]
Morais, F., & Jaques, P. A. (2019). Predição de emoções baseada em mineração de dados: considerando a personalidade para melhorar a detecção. In Simpósio brasileiro de informática na educação (sbie) (Vol. 30). [DOI: 10.5753/cbie.sbie.2019.1521] [GS Search]
Morais, F., Kautzmann, T. R., Bittencourt, I. I., & Jaques, P. A. (2019). EMAP-ML: A protocol of emotions and behaviors annotation for machine learning labels. In European conference on technology enhanced learning (pp. 495–509). Netherlands: Springer. [DOI: 10.1007/978-3-030-29736-7_37] [GS Search]
Nunes, M., & Cazella, S. (2011). O que sua personalidade revela? fidelizando clientes web através de sistemas de recomendação e traços de personalidade. WebMedia, 1. [GS Search]
Ocumpaugh, J. (2015). Baker rodrigo ocumpaugh monitoring protocol (bromp) 2.0 technical and training manual. New York, NY and Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences. [GS Search]
Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501. [DOI: 10.1111/bjet.12156] [GS Search]
Paquette, L., Baker, R. S., Sao Pedro, M. A., Gobert, J. D., Rossi, L., Nakama, A., & Kauffman- Rogoff, Z. (2014). Sensor-free affect detection for a simulation-based science inquiry learning environment. In International conference on intelligent tutoring systems (pp. 1– 10). [DOI: 10.1007/978-3-319-07221-0_1] [GS Search]
Paquette, L., Rowe, J., Baker, R., Mott, B., Lester, J., DeFalco, J., ... Georgoulas, V. (2016). Sensor-free or sensor-full: A comparison of data modalities in multi-channel affect detection. In International conference on educational data mining. ERIC. [GS Search]
Pardos, Z. A., Baker, R. S., San Pedro, M. O., Gowda, S. M., & Gowda, S. M. (2014). Affective states and state tests: Investigating how affect and engagement during the school year predict end-of-year learning outcomes. Journal of Learning Analytics, 1(1), 107–128. [GS Search]
Pekrun, R. (2014). Emotions and learning. Educational practices series, 24. Retrieved from [Link] [GS Search]
Reis, H., Alvares, D., Jaques, P., & Isotani, S. (2018). Analysis of permanence time in emotional states: A case study using educational software. In International conference on intelligent tutoring systems (pp. 180–190). [DOI: 10.1007/978-3-319-91464-0_18] [GS Search]
Rusting, C. L. (1998). Personality, mood, and cognitive processing of emotional information: three conceptual frameworks. Psychological bulletin, 124(2), 165. [DOI: 10.1037/0033-2909.124.2.165] [GS Search]
Rusting, C. L., & Larsen, R. J. (1997). Extraversion, neuroticism, and susceptibility to positive and negative affect: A test of two theoretical models. Personality and Individual Differences, 22(5), 607 - 612. [DOI: 10.1016/S0191-8869(96)00246-2] [GS Search]
Salmeron-Majadas, S., Santos, O. C., & Boticario, J. G. (2014). An evaluation of mouse and keyboard interaction indicators towards non-intrusive and low cost affective modeling in an educational context. Procedia Computer Science, 35, 691–700. [DOI: 10.1016/j.procs.2014.08.151] [GS Search]
Scherer, K. R. (2005). What are emotions? and how can they be measured? Social Science Information, 44(4), 695–729. [DOI: 10.1177/0539018405058216] [GS Search]
Seffrin, H., Rubi, G., Ghilardi, C., Morais, F., Jaques, P., Isotani, S., & Bittencourt, I. I.
(2012). Dicas Inteligentes no Sistema Tutor Inteligente PAT2Math. In Brazilian symposium on computers in education (simpósio brasileiro de informática na educação-sbie) (Vol. 23). [DOI: 10.5753/cbie.sbie.2012.%25p] [GS Search]
Seffrin, H., Rubi, G., & Jaques, P. (2011). O modelo cognitivo do Sistema Tutor Inteligente PAT2Math. In Sbie (Vol. 1). [DOI: 10.5753/cbie.sbie.2011.%25p] [GS Search]
Shanabrook, D. H., Arroyo, I., & Woolf, B. P. (2012). Using touch as a predictor of effort: what the ipad can tell us about user affective state. In International conference on user modeling, adaptation, and personalization (pp. 322–327). [DOI: 10.1007/978-3-642-31454-4_29] [GS Search]
Sinclair, J., Jang, E. E., Azevedo, R., Lau, C., Taub, M., & Mudrick, N. V. (2018). Changes in emotion and their relationship with learning gains in the context of metatutor. In International conference on intelligent tutoring systems (pp. 202–211). [DOI: 10.1007/978-3-319-91464-0_20] [GS Search]
Tong, E. M. (2010). Personality influences in appraisal–emotion relationships: The role of neuroticism. Journal of Personality, 78(2), 393–417. [DOI: 10.1111/j.1467-6494.2010.00620.x] [GS Search]
Vanlehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265. [GS Search]
Wang, Y., Heffernan, N. T., & Heffernan, C. (2015). Towards better affect detectors: effect of missing skills, class features and common wrong answers. In International learning analytics and knowledge conference (pp. 31–35). [DOI: 10.1145/2723576.2723618] [GS Search]
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Felipe de Morais, Patrícia A. Jaques
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
