Screen-Reader Based Contextual Exploration of Mathematical Formulas in Brazilian Portuguese: Design, User Evaluation and Teaching Scenario in the Context of Numerical Analysis

Hérlon Manollo C. Guedes; Paula C. Figueira Cardoso; Evelise Roman Corbalan Góis Freire; Willian M. Watanabe; André Pimenta Freire

doi:10.5753/jis.2023.3325

Authors

Hérlon Manollo C. Guedes Universidade Federal de Lavras https://orcid.org/0000-0002-9465-7518
Paula C. Figueira Cardoso Universidade Federal de Lavras https://orcid.org/0000-0003-3621-8960
Evelise Roman Corbalan Góis Freire Universidade Federal de Lavras https://orcid.org/0000-0003-0245-6055
Willian M. Watanabe Universidade Tecnológica Federal do Paraná https://orcid.org/0000-0001-6910-5730
André Pimenta Freire Universidade Federal de Lavras https://orcid.org/0000-0001-7894-9740

DOI:

https://doi.org/10.5753/jis.2023.3325

Keywords:

Screen reader, Mathematical formulas, Visual Disabilities

Abstract

Although screen readers have made significant technological advancements, mathematics remains a challenging subject for people with visual disabilities. Due to its complex notations and abstract nature, mathematics presents difficulties in understanding through means other than visual. Consequently, reading mathematical content with screen readers poses challenges such as ambiguity, comprehension of long formulas, and identification of specific elements. Furthermore, even with reading difficulties, few screen readers support reading this type of content in Portuguese. This study presents an extension of a previous study which described the development and evaluation of an add-on for NVDA, which enables contextual exploration and navigation of mathematical formulas. The add-on, called Access8Math-NavMatBR, allows for internal exploration of formulas by providing contextual delineations of mathematical elements with support for the Brazilian variant of the Portuguese language. Based on the open-source Access8Math add-on, the new version was developed and evaluated in usability tests with six people with visual disabilities. Results showed that the new system improved understanding of formulas and provided better access to specific elements through formulas abstraction. The evaluation identified 52 issues, such as problems with commands and interaction approaches, verbalization by the screen reader, and platform structure. This extended version extends the analysis by presenting a teaching scenario in the context of numerical analysis and how the contextual exploration can be applied to aid in the understanding of complex elements. The paper presents design implications for systems for reading mathematical formulas in the Brazilian context and considerations for exploring patterns used by Brazilian users when reading and browsing mathematical formulas, dialoguing with the practical example presented.

Downloads

Download data is not yet available.

References

Abreu, S., Silva, J. S. R., Anjos, G. P. d., Guedes, H. M. C., Prietch, S. S., Cardoso, P. C. F., and Freire, A. P. (2019). Usability evaluation of a resource to read mathematical formulae in a screen reader for people with visual disabilities. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, IHC ’19, New York, NY, USA. Association for Computing Machinery.

Ahmetovic, D., Armano, T., Bernareggi, C., Capietto, A., Coriasco, S., Doubrov, B., Kozlovskiy, A., and Murru, N. (2020). Automatic tagging of formulae in pdf documents and assistive technologies for visually impaired people: The latex package axessibility 3.0. In ICCHP - Future Perspectives of AT, eAccessibility and eInclusion, pages 69–73, online. Springer.

Bier, A. and Sroczyński, Z. (2015). Adaptive math-to-speech interface. In Proceedings of the Mulitimedia, Interaction, Design and Innnovation, MIDI ’15, New York, NY, USA. Association for Computing Machinery.

Boonprakong, N., Pudpadee, P., Chalidabhongse, T. H., and Punyabukkana, P. (2017). Reading mathematical expression in thai. In Proceedings of the 11th International Convention on Rehabilitation Engineering and Assistive Technology, page 9, Singapore. Singapore Therapeutic, Assistive & Rehabilitative Technologies (START) Centre.

Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2):77– 101.

Brzostek-Pawłowska, J., Rubin, M., and Salamończyk, A. (2019). Enhancement of math content accessibility in epub3 educational publications. New Review of Hypermedia and Multimedia, 25(1-2):31–56.

Cervone, D., Krautzberger, P., and Sorge, V. (2016). Employing semantic analysis for enhanced accessibility features in mathjax. In 2016 13th IEEE Annual Consumer Communications & Networking Conference (CCNC), pages 1129–1134, Las Vegas, NV, USA. IEEE.

da Paixão Silva, L. F., de Faria Oliveira, O., Freire, E. R. C. G., Mendes, R. M., and Freire, A. P. (2017). How much effort is necessary for blind users to read web-based mathematical formulae?: A comparison using task models with different screen readers. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems, IHC 2017, pages 29:1–29:10, New York, NY, USA. ACM.

da Paixão Silva, L. F., de O Barbosa, A. A., Freire, E. R. C. G., Cardoso, P. C. F., Durelli, R. S., and Freire, A. P. (2018). Content-based navigation within mathematical formulae on the web for blind users and its impact on expected user effort. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, pages 23–32, New York, NY. ACM.

de Lima, M. A., Rodrigues, D., Almeida, P. V., Cardoso, P. C. F., and Freire, A. P. (2019). Análise de verbalizações de fórmulas matemáticas por professores com experiência no ensino de pessoas com deficiência visual/analysis of mathematical formulas verbalizations by teachers with experience in teaching visually impaired people. Revista de Estudos da Linguagem, 27(3):1371–1397.

Doush, I. A. and Pontelli, E. (2009). Building a programmable architecture for non-visual navigation of mathematics: Using rules for guiding presentation and switching between modalities. In International Conference on Universal Access in Human-Computer Interaction, pages 3–13. San Diego, California, USA.

Edwards, A. D., McCartney, H., and Fogarolo, F. (2006). Lambda:: a multimodal approach to making mathematics accessible to blind students. In Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility, pages 48–54, Portland, Oregon, USA. ACM.

Edwards, A. D. N. (1989). Modelling blind users interactions with an auditory computer interface. International Journal of Man-Machine Studies, 30(5):575–589.

Elkabani, I. and Zantout, R. (2015). A framework for helping the visually impaired learn and practice math. In 2015 5th International Conference on Information & Communication Technology and Accessibility (ICTA), pages 1–5, Marrakech, Morocco. IEEE.

Fajardo-Flores, S. and Archambault, D. (2014). Evaluation of a prototype of a multimodal interface to work with mathematics. AMSE IFRATH, 75:106–118.

Ferreira, H. and Freitas, D. (2005). Audiomath: using mathml for speaking mathematics. In XATA05 - XML and Associated Technologies, Braga, Portugal. Universidade do Minho.

Frankel, L., Brownstein, B., and Soiffer, N. (2014). Navigable, customizable tts for algebra. In 28th Annual International Technology and Persons with Disabilities Conference Scientific/Research Proceedings, pages 13–24, San Diego, CA. California State University, Northridge.

Gaura, P. (2002). Remathex — reader and editor of the mathematical expressions for blind students. In Miesenberger, K., Klaus, J., and Zagler, W., editors, Computers Helping People with Special Needs, pages 486–493. Springer Berlin Heidelberg, Berlin, Heidelberg.

Gruber, M., Matousek, J., Hanzlícek, Z., Krnoul, Z., and Zajíc, Z. (2016). ARET-Automatic Reading of Educational Texts for Visually Impaired Students. In INTERSPEECH - Annual Conference of the International Speech Communication Association, pages 383–384, San Francisco, CA. ISCA.

Guedes, H. M. C., Cardoso, P. C., Watanabe, W. M., and Freire, A. P. (2022). Contextual exploration of mathematical formulae on the web for people with visual disabilities in brazil with an open-source screen reader. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems, pages 1–12.

Gulley, A. P., Smith, L. A., Price, J. A., Prickett, L. C., and Ragland, M. F. (2017). Process-driven math: An auditory method of mathematics instruction and assessment for students who are blind or have low vision. Journal of visual impairment & blindness, 111(5):465–471.

Junior, A. S. (2021). Math2Text: Ferramenta Tecnológica para Acessibilidade de Estudantes Cegos a Expressões Matemáticas. PhD thesis, Programa de Pós-Graduação em Ensino de Ciência e Tecnologia, Universidade Tecnológica Federal do Paraná, Ponta Grossa, PR.

Junior, A. S., Mendes, L. R., and da Silva, S. d. C. R. (2020). Math2text: Software para geração e conversão de equações matemáticas em texto-limitações e possibilidades de inclusão (math2text: Software to generation and convertion of mathematical equations to text: limitations and possibilities for inclusion). Revista Ibérica de Sistemas e Tecnologias de Informação, (37):99–115.

Kirkpatrick, A., Connor, J. O., Campbell, A., and Cooper, M. (2018). Web content accessibility guidelines (wcag) 2.1. Available online at [link], last accessed on 2 October 2023.

Maćkowski, M., Brzoza, P., Żabka, M., and Spinczyk, D. (2018). Multimedia platform for mathematics’ interactive learning accessible to blind people. Multimedia Tools and Applications, 77(5):6191–6208.

Matoušek, J., Hanzlíček, Z., Campr, M., Krňoul, Z., Campr, P., and Grber, M. (2011). Web-based system for automatic reading of technical documents for vision impaired students. In International Conference on Text, Speech and Dialogue, pages 364–371, Berlin. Springer.

Mejía, P., Martini, L. C., Grijalva, F., Larco, J. C., and Rodríguez, J. C. (2021). A survey on mathematical software tools for visually impaired persons: A practical perspective. IEEE Access, 9:66929–66947.

Miner, R. R., Ion, P. D. F., and Carlisle, D. (2014). Mathematical markup language (MathML) version 3.0 2nd edition. W3C recommendation, W3C, Cambridge, MA. Available online at [link], last access 02/10/2023.

NTT Data (2018). Resultados da pesquisa do uso de leitores de tela (result from the screen reader user survey). Research results, Everis Brasil, São Paulo, SP. Available online at [link], last accessed 2 October 2023.

Power, C., Petrie, H., Swallow, D., Murphy, E., Gallagher, B., and Velasco, C. A. (2013). Navigating, discovering and exploring the web: strategies used by people with print disabilities on interactive websites. In IFIP Conference on Human-Computer Interaction, pages 667–684, Berlin. Springer.

Rahman, M. F. (2005). Hierarchical manipulation of mathematical expressions for visually impaired students. In Proceedings of the 5th Winona Computer Science Undergraduate Research Symposium, pages 8–15, Winona, MN. Winona State University.

Raman, T. (1994). Audio system for technical readings. PhD thesis, Cornell University, Dept. of Computer Science., Ithaca, NY, USA.

Salamonczyk, A. and Brzostek-Pawlowska, J. (2015). Translation of mathml formulas to polish text, example applications in teaching the blind. In 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), pages 240–244, Gdynia, Poland. IEEE.

Soiffer, N. (2005). Mathplayer: Web-based math accessibility. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility, Assets ’05, pages 204–205, New York, NY, USA. ACM.

Soiffer, N. (2007). Mathplayer v2.1: Web-based math accessibility. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, Assets ’07, pages 257–258, New York, NY, USA. ACM.

Soiffer, N. (2015). Browser-independent accessible math. In Proceedings of the 12th Web for All Conference, W4A ’15, pages 28:1–28:3, New York, NY, USA. ACM.

Sorge, V., Chen, C., Raman, T. V., and Tseng, D. (2014). Towards making mathematics a first class citizen in general screen readers. In Proceedings of the 11th Web for All Conference, W4A ’14, pages 40:1–40:10, New York, NY, USA. ACM.

Stevens, R. D., Edwards, A. D., and Harling, P. A. (1997). Access to mathematics for visually disabled students through multimodal interaction. Human–computer interaction, 12(1-2):47–92.

Sutherland, N. B. (1972). Braille display device. Patente número US3659354A.

Suzuki, M., Kanahori, T., Ohtake, N., and Yamaguchi, K. (2004). An integrated ocr software for mathematical documents and its output with accessibility. In International Conference on Computers for Handicapped Persons, pages 648–655. Springer.

Tseng, W. (2018). Access8math. Disponível em [link], último acesso em 4 de junho de 2022.

Watanabe, T. (2007). Experimental evaluation of usability and accessibility of heading elements. In Proceedings of the 2007 International Cross-disciplinary Conference on Web Accessibility (W4A), W4A ’07, pages 157–164, New York, NY, USA. ACM.

Watanabe, T. (2009). Experimental evaluation of usability and accessibility of heading elements. Disability and Rehabilitation: Assistive Technology, 4(4):236–247.

Yamaguchi, K., Suzuki, M., and Kanahori, T. (2014). Braille capability in accessible e-textbooks for math and science. In Miesenberger, K., Fels, D., Archambault, D., Peňáz, P., and Zagler, W., editors, Computers Helping People with Special Needs, pages 557–563, Cham. Springer International Publishing.