Deep Learning for Automated Authoring of Domain Models for Step-Based Tutoring Systems
DOI:
https://doi.org/10.5753/rbie.2024.3702Keywords:
Intelligent Tutoring Systems, PAT2Math, Deep Learning, TransformersAbstract
Intelligent Tutoring Systems (ITS) are computer programs that assist students during the learning process by providing personalized assistance based on the students' characteristics. They have a domain module that represents expert knowledge in a specific area of study. In step-based ITS, the domain module is typically developed as a rule-based expert system, meaning a system that uses knowledge-based rules to solve and correct problems presented to the students. These rule-based expert systems entail greater complexity in their creation, maintenance, and knowledge updates, potentially leading to errors or increased processing time. Therefore, this work automates the domain module of a step-based intelligent tutoring system by creating a model that employs Deep Learning to correct first-degree equations. The correction is performed without the use of any mathematical knowledge, solely relying on Natural Language Processing applied to a database of approximately 115,000 mathematical expressions. Six different versions using GRU and transformer architectures are presented and compared. The final model achieves a 95.5% accuracy rate in evaluating/correcting these equations.
Downloads
References
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. [GS Search].
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems. [GS Search].
Charton, F., Hayat, A., & Lample, G. (2020). Learning advanced mathematical computations from examples. arXiv preprint arXiv:2006.06462. [GS Search].
Chawla, N. V. (2005). Data Mining for Imbalanced Datasets: An Overview. Data Mining and Knowledge Discovery Handbook, 853–867. [GS Search].
Chollet, F. (2021). Deep Learning with Python, Second Edition. Manning. [GS Search].
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. [GS Search].
Crabtree, A., M. Nehme. (2023). What is Data Analysis? An Expert Guide With Examples. [DataCamp]
DeepLearning.ai. (2020). Natural Language Processing with Attention Models. [Coursera]
Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., & Kaiser, L. (2018). Universal transformers. arXiv preprint arXiv:1807.03819. [GS Search].
Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs/1810.04805. [arXiv] [GS Search].
Escovedo, T., & Koshiyama, A. (2020). Introdução a Data Science: Algoritmos de Machine Learning e métodos de análise. Casa do Código. [GS Search].
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., & Steinhardt, J. (2021). Measuring Mathematical Problem Solving With the MATH Dataset. arXiv preprint arXiv:2103.03874. [GS Search].
Hochreiter, S., & Schmidhuber, J. (1997). Long Short-term Memory. Neural computation, 9, 1735–80. https://doi.org/10.1162/neco.1997.9.8.1735 [GS Search].
Jaques, P. A., Seffrin, H., Rubi, G., de Morais, F., Ghilardi, C., Bittencourt, I. I., & Isotani, S. (2013). Rule-based expert systems to support step-by-step guidance in algebraic problem solving: The case of the tutor PAT2Math. Expert Systems With Applications, 40(14), 5456–5465. https://doi.org/10.1016/j.eswa.2013.04.004 [GS Search].
Lample, G., & Charton, F. (2019). Deep Learning for Symbolic Mathematics. arXiv preprint arXiv:1912.01412. [GS Search].
Lane, H., Hapke, H., & Howard, C. (2019). Natural Language Processing in Action: Understanding, analyzing, and generating text with Python. Manning Publications. [GS Search].
Luong, M.-T., Pham, H., & Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. [GS Search].
Nagori, V., & Trivedi, B. (2012). Which type of Expert System – Rule Base, Fuzzy or Neural is Most Suited for Evaluating Motivational Strategies on Human Resources: An Analytical Case Study. International Journal of Business Research and Management (IJBRM), 3(5), 249–254. [Link] [GS Search].
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9. [GS Search].
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. CoRR, abs/1910.10683. [arXiv] [GS Search].
Russell, S., & Norvig, P. (2010). Artificial Intelligence: A Modern Approach (3ª ed.). Prentice Hall. [GS Search].
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2020). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [GS Search].
Santoro, A., Hill, F., Barrett, D. G. T., Morcos, A. S., & Lillicrap, T. (2018). Measuring abstract reasoning in neural networks. ArXiv, abs/1807.04225. [GS Search].
Saxton, D., Grefenstette, E., Hill, F., & Kohli, P. (2019). Analysing Mathematical Reasoning Abilities of Neural Models. arXiv preprint arXiv:1904.01557. [GS Search].
Schlag, I., Smolensky, P., Fernandez, R., Jojic, N., Schmidhuber, J., & Gao, J. (2019). Enhancing the transformer with explicit relational encoding for math problem solving. arXiv preprint arXiv:1910.06611. [GS Search].
TensorFlow. (2020). Transformer model for language understanding. [TensorFlow]
Trask, A. W. (2019). Grokking deep learning. Simon; Schuster. [GS Search].
VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369 [GS Search].
VanLehn, K. (2006). The Behavior of Tutoring Systems. Int. J. Artif. Intell. Ed., 16(3), 227–265. http://dl.acm.org/citation.cfm?id=1435351.1435353
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is All you Need. Em I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan & R. Garnett (Ed.), Advances in Neural Information Processing Systems 30 (pp. 5998–6008). Curran Associates, Inc. [Link] [GS Search].
Wangperawong, A. (2019). Attending to Mathematical Language with Transformers. arXiv eprints, jourarticle arXiv:1812.02825. [GS Search].
Woolf, B. P. (2007). Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing e-Learning. Morgan Kaufmann Publishers Inc. [GS Search].
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Fábio Manique de Castilhos, Patrícia A. Jaques
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
