Evaluación automática de la escritura: una revisión sistemática
DOI:
https://doi.org/10.5753/rbie.2023.2869Keywords:
Corrección de la Escritura, Análisis de Contenido, Procesamiento de Lenguaje NaturalAbstract
La calificación automática de ensayos (AES) ha sido un tema ampliamente explorado en la literatura. Permite prescindir del esfuerzo humano que supone corregir un gran número de redacciones en poco tiempo. La mayoría de los trabajos se centran en el esfuerzo por desarrollar algoritmos capaces de corregir automáticamente textos en inglés. Sin embargo, en el caso de la lengua portuguesa, se trata de un área que aún está en desarrollo. En este contexto, este trabajo presenta un Mapeo Sistemático de la Literatura que busca identificar los enfoques de Inteligencia Artificial que están siendo utilizados para apoyar la evaluación de ensayos escritos en lengua portuguesa. Los principales hallazgos de este artículo incluyen los siguientes hechos: (i) los enfoques de los trabajos seleccionados suelen centrarse en el uso de atributos extraídos del texto en lugar del uso de modelos preentrenados basados en Deep Learning; (ii) hay una prevalencia de las métricas tradicionales, como Accuracy, Coverage y F-Measure en la validación de los resultados; (iii) los feedbacks generados por los enfoques tienen un bajo nivel de detalle; y (iv) los artículos seleccionados no analizan el impacto práctico en aplicaciones del mundo real.
Descargas
Citas
Becheikh, N., Landry, R., & Amara, N. (2006). Lessons from innovation empirical studies in the manufacturing sector: A systematic review of the literature from 1993–2003. Technovation, 26(5-6), 644-664. doi: 10.1016/j.technovation.2005.06.016 [GS Search]
Camelo, R., Justino, S., & Mello, R. (2020). Coh-Metrix PT-BR: Uma API web de análise textual para a educação. Anais dos Workshops do IX Congresso Brasileiro de Informática na Educação, 179–186. doi: 10.5753/cbie.wcbie.2020.179 [GS Search]
Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., et al. (2015). Xgboost: extreme gradient boosting. R package version 0.4-2, 1(4), 1–4. doi: 10.1145/2939672.2939785 [GS Search]
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. doi: 10.1007/BF00994018 [GS Search]
Costa, L., Oliveira, E. H. T. d., & Castro Júnior, A. (2020). Corretor Automático de Redações em Língua Portuguesa: um mapeamento sistemático de literatura. Anais do XXXI Simpósio Brasileiro de Informática na Educação (SBIE 2020), 1403–1412. doi: 10.5753/cbie.sbie.2020.1403 [GS Search]
Ferreira Mello, R., Fiorentino, G., Oliveira, H., Miranda, P., Rakovic, M., & Gasevic, D. (2022). Towards automated content analysis of rhetorical structure of written essays using sequential content-independent features in Portuguese. LAK22: 12th International Learning Analytics and Knowledge Conference, 404–414. doi: 10.1145/3506860.3506977 [GS Search]
Filho, A. H., Concatto, F., Nau, J., Prado, H. A. d., Imhof, D. O., & Ferneda, E. (2019). Imbalanced Learning Techniques for Improving the Performance of Statistical Models in Automated Essay Scoring. Procedia Computer Science, 159, 764–773. doi: 10.1016/j.procs.2019.09.235 [GS Search]
Filho, A. H., do Prado, H. A., Ferneda, E., & Nau, J. (2018). An approach to evaluate adherence to the theme and the argumentative structure of essays. Procedia Computer Science, 126, 788–797. doi: 10.1016/j.procs.2018.08.013 [GS Search]
Fonseca, E., Medeiros, I., Kamikawachi, D., & Bokan, A. (2018). Automatically Grading Brazilian Student Essays [Series Title: Lecture Notes in Computer Science]. Em A. Villavicencio, V. Moreira, A. Abad, H. Caseli, P. Gamallo, C. Ramisch, H. Gonçalo Oliveira & G. H. Paetzold (Ed.), Computational Processing of the Portuguese Language (pp. 170–179). Springer International Publishing. doi: 10.1007/978-3-319-99722-3_18 [GS Search]
Guimarães, N. S., Ferreira, A. J., Silva, R. d. C. R., de Paula, A. A., Lisboa, C. S., Magno, L., Ichiara, M. Y., & Barreto, M. L. (2022). Deduplicating records in systematic reviews: there are free, accurate automated ways to do so. Journal of Clinical Epidemiology, 152, 110–115. doi: j.jclinepi.2022.10.009 [GS Search]
Haendchen Filho, A., do Prado, H. A., Ferneda, E., & Nau, J. (2018). An approach to evaluate adherence to the theme and the argumentative structure of essays. Procedia Computer Science, 126, 788–797. doi: 10.1016/j.procs.2018.08.013 [GS Search]
Johnson, N., & Phillips, M. (2018). Rayyan for systematic reviews. Journal of Electronic Resources Librarianship, 30(1), 46–48. doi: 10.1080/1941126X.2018.1444339 [GS Search]
Kahn, J. H., Tobin, R. M., Massey, A. E., & Anderson, J. A. (2007). Measuring emotional expression with the Linguistic Inquiry and Word Count. The American journal of psychology, 120(2), 263–286. [GS Search]
Kitchenham, B. A., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering (rel. técn. EBSE 2007-001). Keele University e Durham University Joint Report. [GS Search]
Marinho, J., Anchiêta, R., & Moura, R. (2021). Essay-BR: a Brazilian Corpus of Essays. Anais do III Dataset Showcase Workshop, 53–64. doi: 10.5753/dsw.2021.17414 [GS Search]
McKeown, S., & Mir, Z. M. (2021). Considerations for conducting systematic reviews: evaluating the performance of different methods for de-duplicating references. Systematic reviews, 10, 1–8. doi: 10.1186/s13643-021-01583-y [GS Search]
Mello, R. F., Fiorentino, G., Miranda, P., Oliveira, H., Rakovic, M., & Gašević, D. (2021). Towards Automatic Content Analysis of Rhetorical Structure in Brazilian College Entrance Essays [Series Title: Lecture Notes in Computer Science]. Em I. Roll, D. McNamara, S. Sosnovsky, R. Luckin & V. Dimitrova (Ed.), Artificial Intelligence in Education (pp. 162–167). Springer International Publishing. doi: 10.1007/978-3-030-78270-2_29 [GS Search]
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group*, t. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of internal medicine, 151(4), 264–269. doi: 10.7326/0003-4819-151-4-200908180-00135 [GS Search]
Nau, J., Haendchen Filho, A., & Dazzi, R. L. S. (2019). Identificação e Avaliação Automática da Proposta de Intervenção em Textos Dissertativos-Argumentativos: Uma Revisão Sistemática da Literatura. Anais do Computer on the Beach, 493–501. doi: 10.4013/cld.2017.153.08 [GS Search]
Nunes, A., Cordeiro, C., Limpo, T., & Castro, S. L. (2022). Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020. Journal of Computer Assisted Learning, 38(2), 599–620. doi: 10.1111/jcal.12635 [GS Search]
Palermo, C., & Thomson, M. M. (2018). Teacher implementation of self-regulated strategy development with an automated writing evaluation system: Effects on the argumentative writing performance of middle school students. Contemporary Educational Psychology, 54, 255–270. doi: 10.1016/j.cedpsych.2018.07.002 [GS Search]
Papadopoulos, I., Koulouglioti, C., Lazzarino, R., & Ali, S. (2020). Enablers and barriers to the implementation of socially assistive humanoid robots in health and social care: a systematic review. BMJ open, 10(1), e033096. doi: 10.1136/bmjopen-2019-033096 [GS Search]
Safavian, S. R., & Landgrebe, D. (1991). A survey of decision tree classifier methodology. IEEE transactions on systems, man, and cybernetics, 21(3), 660–674. doi: 10.1109/21.97458 [GS Search]
Santos, K. S., Soder, M., Marques, B. S. B., & Feltrim, V. D. (2018). Analyzing the rhetorical structure of opinion articles in the context of a Brazilian college entrance examination. Computational Processing of the Portuguese Language: 13th International Conference, PROPOR 2018, Canela, Brazil, September 24–26, 2018, Proceedings 13, 3–12. doi: 10.1007/978-3-319-99722-3_1 [GS Search]
Sousa, A., Leite, B., Rocha, G., & Lopes Cardoso, H. (2021). Cross-Lingual Annotation Projection for Argument Mining in Portuguese [Series Title: Lecture Notes in Computer Science]. Em G. Marreiros, F. S. Melo, N. Lau, H. Lopes Cardoso & L. P. Reis (Ed.), Progress in Artificial Intelligence (pp. 752–765). Springer International Publishing. doi: 10.1007/978-3-030-86230-5_59 [GS Search]
Tang, J., & Rich, C. S. (2017). Automated writing evaluation in an EFL setting Lessons from China. JALT CALL Journal, 13(2), 117--146. doi: 10.29140/jaltcall.v13n2.215 [GS Search]
Ware, P. (2014). Feedback for Adolescent Writers in the English Classroom. Writing & Pedagogy, 6(2). doi: 10.1558/wap.v6i2.223 [GS Search]
Tang, J., & Rich, C. S. (2017). Automated writing evaluation in an EFL setting Lessons from China. JALT CALL Journal, 13(2), 117--146. doi: 10.29140/jaltcall.v13n2.215 [GS Search]
Archivos adicionales
Published
Cómo citar
Issue
Section
Licencia
Derechos de autor 2023 Tiago Barbosa de Lima, Ingrid Luana Almeida da Silva, Elyda Laisa Soares Xavier Freitas, Rafael Ferreira Mello
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-SinDerivadas 4.0.
