Tutoria: a software platform to improve feedback in education
DOI:
https://doi.org/10.5753/jis.2023.3247Keywords:
Educational feedback, Software tools, Written assessmentAbstract
Educational feedback is essential to help students learn from their mistakes and self-regulate their learning strategies. However, work overload and lack of time are barriers for educators to give quality and timely feedback, particularly for written assessments. Software tools to support feedback processes typically focus on automatic messages, lacking personalization. We present Tutoria, a software tool that uses artificial intelligence techniques to correct assessments more efficiently while also ensuring that good practices of educational feedback are followed. Tutoria was developed through a user-centered design process, including interviews and prototype validation with undergraduate students and instructors from higher education institutions in different fields of knowledge. Results indicate that the software presents good usability and relevance for educators. We expect that Tutoria can help educators construct personalized written feedback efficiently, allowing them to give quality feedback to large groups within realistic time frames.
Downloads
References
Barbosa, S. and Silva, B. (2010). Interação humano-computador. Elsevier Brasil.
Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in continuing education, 22(2):151–167.
Boud, D. and Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in higher education, 38(6):698–712.
Carless, D., Salter, D., Yang, M., and Lam, J. (2011). Developing sustainable feedback practices. Studies in higher education, 36(4):395–407.
Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai,Y.-S., Gašević, D., and Mello, R. F. (2021). Automatic feedback in online learning environments: A systematic literature review. Computers and Education: Artificial Intelligence, 2:100027.
Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K.,Nascimento, A., Freitas, F., and Gašević, D. (2020). How good is my feedback? a content analysis of written feedback. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, pages 428–437.
Erickson, J. A. and Botelho, A. (2021). Is it fair? automated open response grading. In International Conference on Educational Data Mining.
Falcao, T. P., Ferreira, R., Rodrigues, R. L., Diniz, J., and Gasevic, D. (2019). Students’ perceptions about learning analytics in a brazilian higher education institution. In 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), volume 2161, pages 204–206. IEEE.
Falcão, T. P., Mello, R. F., Rodrigues, R. L., Diniz, J. R. B., Tsai, Y.-S., and Gašević, D. (2020). Perceptions and expectations about learning analytics from a brazilian higher education institution. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, pages 240–249.
Ferreira-Mello, R., André, M., Pinheiro, A., Costa, E., and Romero, C. (2019). Text mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(6):e1332.
Freeman, R. and Lewis, R. (2016). Planning and implementing assessment. Routledge.
Garcia, S., Marques, E., Mello, R. F., Gašević, D., and Falcão, T. P. (2021). Aligning expectations about the adoption of learning analytics in a brazilian higher education institution. In Proceedings of the Conference of Artificial Intelligence in Education, pages 1–6.
Gulwani, S., Radiček, I., and Zuleger, F. (2014). Feedback generation for performance problems in introductory programming assignments. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 41–51. ACM.
Hattie, J. and Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1):81–112.
Higgins, R., Hartley, P., and Skelton, A. (2001). Getting the message across: the problem of communicating assessment feedback. Teaching in higher education, 6(2):269–274.
Hounsell, D. (2004). Reinventing feedback for the contemporary scottish university. In Quality Enhancement Conference on Assessment, University of Glasgow, volume 4.
Ivanic, R., Clark, R., and Rimmershaw, R. (2000). What am i supposed to make of this?: the messages conveyed to students by tutors’ written comments.
Krusche, S. and Seitz, A. (2018). Artemis: An automatic assessment management system for interactive learning. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education, pages 284–289. ACM.
Marin, V. J., Pereira, T., Sridharan, S., and Rivero, C. R. (2017). Automated personalized feedback in introductory java programming moocs. In 2017 IEEE 33rd International Conference on Data Engineering (ICDE), pages 1259–1270. IEEE.
Mello, R. F., Neto, R., Fiorentino, G., Alves, G., Arêdes, V., Silva, J. V. G. F., Falcão, T. P., and Gašević, D. (2022). Enhancing instructors’ capability to assess open-response using natural language processing and learning analytics. In Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption: 17th European Conference on Technology Enhanced Learning, EC-TEL 2022, Toulouse, France, September 12–16, 2022, Proceedings, pages 102–115. Springer
Nicol, D. J. and Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2):199–218.
Pardo, A., Bartimote, K., Shum, S. B., Dawson, S., Gao, J., Gašević, D., Leichtweis, S., Liu, D., Martínez-Maldonado, R., Mirriahi, N., et al. (2018). Ontask: Delivering data-informed, personalized learning support actions. Journal of Learning Analytics, 5(3):235–249.
Pardo, A., Dawson, S., Gašević, D., and Siemens, G. (2019). Supporting feedback processes at scale with ontask a hands-on tutorial. In International Learning Analytics & Knowledge Conference 2019, pages 285–288. Association for Computing Machinery (ACM).
Pereira, F. D., Oliveira, E. H., Oliveira, D. B., Cristea, A. I., Carvalho, L. S., Fonseca, S. C., Toda, A., and Isotani, S. (2020). Using learning analytics in the amazonas: understanding students’ behaviour in introductory programming. British journal of educational technology, 51(4):955–972.
Ragupathi, K. and Lee, A. (2020). Beyond fairness and consistency in grading: The role of rubrics in higher education. In Diversity and inclusion in global higher education, pages 73–95. Palgrave Macmillan, Singapore.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2):119–144.
Santos, J. C. and Ribeiro, A. R. (2012). Jonline: proposta preliminar de um juiz online didático para o ensino de programação. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 1.
Tsai, Y.-S., Mello, R. F., Jovanović, J., and Gašević, D. (2021). Student appreciation of data-driven feedback: A pilot study on ontask. In LAK21: 11th International Learning Analytics and Knowledge Conference, pages 511–517.
Wiggins, G. (1998). Educative Assessment. Designing Assessments To Inform and Improve Student Performance. ERIC.
Winstone, N. E. and Carless, D. (2021). Who is feedback for? the influence of accountability and quality assurance agendas on the enactment of feedback processes. Assessment in Education: Principles, Policy & Practice, 0(0):1–18
Yujian, L. and Bo, L. (2007). A normalized levenshtein distance metric. IEEE transactions on pattern analysis and machine intelligence, 29(6):1091–1095.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Taciana Pontual Falcão, Verenna Arêdes, Samuel Barbosa Jatobá de Souza, Giuseppe Fiorentino, José Rodrigues Neto, Gabriel Alves, Rafael Ferreira Mello
This work is licensed under a Creative Commons Attribution 4.0 International License.
JIS is free of charge for authors and readers, and all papers published by JIS follow the Creative Commons Attribution 4.0 International (CC BY 4.0) license.
