Assessing the Psychological Impact of AI on Computer and Data Science Education: An Exploratory Study

Pedro Henrique Ramos Pinto; Cleydson de Souza Ferreira Junior; Lutero Lima Goulart; João Vitor Cardoso Beltrão; Filipe de Lima Vaz Monteiro; Paloma Duarte de Lira; Samuel José Fernandes Mendes; Vitor Meneghetti Ugulino de Araujo

doi:10.5753/jbcs.2026.6085

Authors

Pedro Henrique Ramos Pinto Federal University of Paraíba https://orcid.org/0009-0003-0071-402X
Cleydson de Souza Ferreira Junior Federal University of Paraíba https://orcid.org/0009-0001-0198-3083
Lutero Lima Goulart Federal University of Paraíba https://orcid.org/0009-0006-7776-1424
João Vitor Cardoso Beltrão Federal University of Paraíba https://orcid.org/0009-0008-9734-4606
Filipe de Lima Vaz Monteiro Federal University of Paraíba https://orcid.org/0009-0000-6741-7132
Paloma Duarte de Lira Federal University of Paraíba https://orcid.org/0009-0003-3701-4115
Samuel José Fernandes Mendes Federal University of Paraíba https://orcid.org/0009-0004-3360-5038
Vitor Meneghetti Ugulino de Araujo Federal University of Paraíba https://orcid.org/0009-0005-9624-1515

DOI:

https://doi.org/10.5753/jbcs.2026.6085

Keywords:

Psychopedagogy, LLMs, GenAI, Computer Education, Validity Assessment, Psychometric Properties

Abstract

This study assesses the impact of Generative AI on the educational experiences of computer and data science students at the Center for Informatics, Federal University of Paraíba (CI/UFPB), Brazil. Through Exploratory Factor Analysis (EFA) of five psychometric scales, the research examines students’ acceptance of LLMs, their levels of academic burnout, technology-related anxiety, and the prevalence of both metacognitive and dysfunctional learning strategies associated with LLM use. Results revealed high adoption of LLMs, low levels of AI-related technology anxiety, and frequent use of metacognitive strategies. However, dysfunctional learning patterns were still present, particularly among students experiencing higher levels of academic burnout. This study contributes to the ongoing discourse on AI in education, emphasizing the need for pedagogical frameworks that support the effective and ethical adoption of AI while addressing the psychological demands placed on students. The validated instruments are made available for future research in educational and psychological contexts, along with their versions back-translated into English.

Downloads

Download data is not yet available.

References

Alnaim, M. S. (2024). Generative AI: A Case Study of ChatGPT’s Impact on University Students’ Learning Practices. Journal of Higher Education Theory and Practice, 24(3):44-54. DOI: 10.21203/rs.3.rs-4515034/v1.

Brown, L. (2023). factor_analyzer: A Python package for exploratory and confirmatory factor analysis. Read the Docs. Available at:[link].

Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., and Wienrich, C. (2023). MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies. Computers in Human Behavior: Artificial Humans, 1:100014. DOI: 10.1016/j.chbah.2023.100014.

Cooper, G. (2023). Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence. Journal of Science Education and Technology, 32:444-452. DOI: 10.1007/s10956-023-10039-y.

Chiu, T. K. F., Moorhouse, B. L., Chai, C. S., and Ismailov, M. (2023). Teacher support and student motivation to learn with Artificial Intelligence (AI) based chatbot. Interactive Learning Environments. DOI: 10.1080/10494820.2023.2172044.

Choi, J. H., Hickman, K. E., Monahan, A., and Schwarcs, D. B. (2023). ChatGPT Goes to Law School. Journal of Legal Education, 71:387 (2022), submitted:1-16. DOI: 10.2139/ssrn.4335905.

Crawford, A. V., Green, S. B., Levy, R., Lo, W., Scott, L., Svetina, D., and Thompson, M. S. (2010). Evaluation of parallel analysis methods for determinning the number of factors. Educational and Psychological Measurement, 79(6):885-901. DOI: 10.1177/0013164410379332.

Damasio, B. (2012). Uses of Exploratory Factorial Analysis in Psychology. Avaliação Psicológica, 11(2):213-228. DOI: 10.1590/S1677-04712012000200007.

Dunn, T. J., Baguley, T., and Brunsden, V. (2013). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3):399-412. DOI: 10.1111/bjop.12046.

Dwivedi, Y. K., Kshetri, N., Huges, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., (...) and Wright, R. (2023). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71:102642. DOI: 10.1016/j.ijinfomgt.2023.102642.

Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., and Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1):1-19. DOI: 10.1186/s41239-022-00362-6.

Gao, S. and Gao, A. K. (2023). On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models. ArXiv preprints, submitted:1-14. DOI: 10.48550/arXiv.2307.09793.

Gilbert, W., Bureau, J. S., Poellhuber, B., and Guay, F. (2022). Educational contexts that nurture students' psychological needs predict low distress and healthy lifestyle through facilitated self-control. Current Psychology, 42:29661-29618. DOI: 10.1007/s12144-022-04019-5.

Goretzko, D., Pham, T. T. H., and Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40:3510-3521. DOI: 10.1007/s12144-019-00300-2.

Goretzko D. and Bühner M. (2020). One model to rule them all? Using machine learning algorithms to determine the number of factors in exploratory factor analysis. Psychological Methods, 25(6):776-786. DOI: 10.1037/met0000262.

Hadi, M. U., Al-Tashi, G., Qureshi, R., Shah, A., Muneer, A., Shaikh, M. B., Al-Garadi, M. A., Wu, J., and Mirjalili, S. (2023). Large Language Models: A Comprehensive Survey of Applications, Challenges, Limitations, and Future Prospects. TechRxiv preprints, submitted:1-44. DOI: 10.36227/techrxiv.23589741.v4.

Hanum S. F., Hasmayni, B., and Lubis, A. H. (2023). The Analysis of Chat GPT Usage Impact on Learning Motivation among Scout Students. International Journal of Research and Review, 10(7):632-638. DOI: 10.52403/ijrr.20230774.

Hung, J. and Chen, J. (2023). The Benefits, Risks and Regulation of Using ChatGPT in Chinese Academia: A Content Analysis. Social Sciences, 12(9):380. DOI: 10.3390/socsci12070380.

Hwang, H. S., Zhu, L. C., and Cui, Q. (2023). Development and Validation of a Digital Literacy Scale in the Artificial Intelligence Era for College Students. KSII Transactions on Internet and Information Systems, 17(8):2241-2258. DOI: 10.3837/tiis.2023.08.016.

Kristensen, T. S., Borritz, M., Villadsen, E., and Christensen B. C. (2005). The Copenhagen Burnout Inventory: A new tool for the assessment of burnout. Work and Stress, 19(3):192-207. DOI: 10.1080/02678370500297720.

Laupichler, M. C., Aster, A., Haverkamp, N., and Raupach, T. (2023). Development of the “Scale for the assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human Behavior Reports, 12:100338. DOI: 10.1016/j.chbr.2023.100338.

Lee, Y. F., Hwang, G. J., and Chen, P. Y. (2023). Impacts of an AI-based chabot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation. Educational technology research and development, 70:1843-1865. DOI: 10.1007/s11423-022-10142-8.

Lintner, T. A. (2024). A systematic review of AI literacy scales. Science of Learning, 9(50). DOI: 10.1038/s41539-024-00264-4.

Mofatteh M. (2020). Risk factors associated with stress, anxiety, and depression among university undergraduate students. AIMS Public Health, 8(1):36-65. DOI: 10.3934/publichealth.2021004.

Neji, W., Boughattas, N., and Ziadi, F. (2023). Exploring New AI-Based Technologies to Enchance Students' Motivation. Informing Science Institute, 20. DOI: 10.28945/5149.

Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., and Chu, S. K. W. (2024). Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach. British Journal of Educational Technology, 55(3):1082-1104. DOI: 10.1111/bjet.13411.

Oliveira, A. F. and Caliatto, S. G. (2018). Exploratory Factor Analysis of a Scale of Learning Strategies. Educação: Teoria e Prática, 18(59):548-565. DOI: 10.18675/1981-8106.vol28.n59.p548-565.

Okonkwo, C. W. and Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2:100033. DOI: 10.1016/j.caeai.2021.100033.

Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., and Podsakoff, N. P. (2001). Common method biases in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5):879-903. DOI: 10.1037/0021-9010.88.5.879.

Pahune, S. and Chandrasekharan, M. (2023). Several Categories of Large Language Models (LLMs): A Short Survey. IJRASET, 11(7). DOI: 10.22214/ijraset.2023.54677.

Patil, V. H., Singh, S. N., Mishra, S., and Donavan, D. T. (2008). Efficient theory development and factor retention criteria: Abandon the 'eigenvalue greater than one' criterion. Journal of Business Research, 61(2):162-170. DOI: 10.1016/j.jbusres.2007.05.008.

Pereira, C. P. da S., Santos, A. A. A. dos, and Ferraz, A. S. (2020). Learning Strategies Assessment Scale for Vocational Education: Adaptation and psychometric studies. Revista Portuguesa de Educação, 33(1):75-93. DOI: 10.21814/rpe.19127.

Pinto, P. H. R. (2025). LLMs Dataset UFPB [Data set]. Kaggle. DOI: 10.34740/kaggle/ds/6906431.

Ramos, A. S. M. (2023). Generative Artificial Intelligence based on large language models - tools for use in academic research. Scielo preprints, submitted. DOI: 10.1590/SciELOPreprints.6105.

Rahman, M., Terano, H. J. R., Rahman, N., Salamzadeh, A., and Rahaman, S. (2023). ChatGPT and Academic Research: A Review and Recommendations Based on Practical Examples. Journal of Education, Management and Development Studies, 3(1):1-12. DOI: 10.52631/jemds.v3i1.175.

Revelle, W. (2023). psych: Procedures for Psychological, Psychometric, and Personality Research. R package version 2.3.9. Northwestern University. Available at:[link].

Sallam, M., Salim, N. A., Barakat, M., Al-Mahzoum, K., Al-Tammemi, A. B., Malaeb, D., Hallit, R., and Hallit, S. (2023). Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study. Journal of Medical Internet Research, 9. DOI: 10.2196/48254.

Sass, D. A. and Schmitt, T. A. (2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavioral Research, 45(1):73-103. DOI: 10.1080/00273170903504810.

Shanahan, M. (2022). Talking About Large Language Models. ArXiv preprint, submitted; 5:1-13. DOI: 10.48550/arXiv.2212.03551.

Taherdoost, H., Sahibuddin, S., and Jalaliyoon, N. (2022). Exploratory Factor Analysis: concepts and theory. In Advances in Applied and Pure Mathematics. 27, WSEAS:375-382. Available at:[link].

Theobald, M. (2021). Self-regulated learning training programs enhance university students' academic performance, self-regulated learning strategies, and motivation: A meta-analysis. Contemporary Educational Psychology, 66:101976. DOI: 10.1016/j.cedpsych.2021.101976.

Tu, X., Zou, J. Su, W. J., and Zhang, L. (2023). What Should Data Science Education Do with Large Language Models? arXiv preprints, submitted:1-18. DOI: 10.48550/arXiv.2307.02792.

Urban, M., Děchtěrenko, F., Lukavsky, J., Hrabalová, V., Švácha, F., Brom, C., and Urban, K. (2024). ChatGPT Improves Creative Problem-Solving Performance in University Students: An Experimental Study. Computers & Education. DOI: 10.1016/j.compedu.2024.105031.

Yin, J., Goh, T., Yang, B., and Xiaobin, Y. (2023). Conversation Technology With Micro-Learning: The Impact of Chatbot-Based Learning on Students' Learning Motivation and Performance. Journal of Educational Computing Research, 59(1). DOI: 10.1177/0735633120952067.

Zhai, X. (2023). ChatGPT User Experience: Implications for Education. SSRN Electronic Journal, submitted. DOI: 10.2139/ssrn.4312418.

Zhou, X., Teng, D., and Al-Samarraie, H. (2023). The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving. Education Sciences, 14(1302). DOI: 10.3390/educsci14121302.

Wilson, M. L., Huggins-Manley, A. C., Ritzhaupt, A. D., and Ruggles, K. (2023). Development of the Abbreviated Technology Anxiety Scale (ATAS). Behavior Research Methods, 55:185-199. DOI: 10.3758/s13428-022-01820-9.