Exploring the Influence of Software Evolution on Mobile App Accessibility: Insights from User Reviews

Alberto Dumont Alves Oliveira; Paulo Sergio Henrique dos Santos; Wajdi Aljedaani; Marcelo Medeiros Eler

doi:10.5753/jbcs.2024.4321

Authors

Alberto Dumont Alves Oliveira University of São Paulo https://orcid.org/0000-0002-7912-991X
Paulo Sergio Henrique dos Santos University of São Paulo https://orcid.org/0000-0002-8739-0640
Wajdi Aljedaani University of North Texas https://orcid.org/0000-0002-6700-719X
Marcelo Medeiros Eler University of São Paulo https://orcid.org/0000-0001-5872-0607

DOI:

https://doi.org/10.5753/jbcs.2024.4321

Keywords:

Accessibility, mobile, app, update, evolution, user reviews

Abstract

Software systems constantly evolve to accommodate stakeholders' and environments' requirements that change over time. In this process, the frequent modifications can increase software complexity and negatively impact its global quality when conducted in an unstructured way. The evolutive nature of mobile environments led researchers to investigate how mobile app evolution impacts complexity, security, resource consumption, maintainability, usability, and accessibility. In particular, there has been limited research on the impact of app updates on mobile accessibility: most studies focused on tracking the number of accessibility violations found by automated tools across successive versions of a small set of applications. In a previous work, we made a contribution to this field by identifying accessibility reviews associated with app updates and prompting ChatGPT-4 to provide an overview of the main accessibility issues and enhancements perceived by users in the new releases of a mobile app. In this manuscript, we extend our previous work by adopting manual content analysis to delve deeper into our research questions and by adding new research questions associated with the identification of reviews linked to app updates, user characteristics, WCAG principles and guidelines, and user demands reported in accessibility reviews. Our results show that the accessibility barriers reported by users are mostly linked to the WCAG 2.2 Perceivable principle, and the Distinguishable and Adaptable guidelines, which includes poor color scheme, small font size, unlabeled elements, and lack of customization options. Accordingly, the consequences of the lack of accessibility is mainly connected to the difficult users experience to perceive elements of the interface (e.g. difficult to read and distinguish content, watch videos) and to use screen readers, in addition to feel discriminated against. The most common demand developers and organizations receive is to bring back some accessible feature or to fix accessibility bugs.

Downloads

Download data is not yet available.

References

Aljedaani, W., Alkahtani, M., Ludi, S., Mkaouer, M. W., Eler, M. M., Kessentini, M., and Ouni, A. (2023). The state of accessibility in blackboard: Survey and user reviews case study. In Proceedings of the 20th International Web for All Conference, W4A '23, page 84–95, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3587281.3587291.

Aljedaani, W., Mkaouer, M. W., Ludi, S., and Javed, Y. (2022a). Automatic classification of accessibility user reviews in android apps. In 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA), pages 133-138. IEEE. DOI: 10.1109/CDMA54072.2022.00027.

Aljedaani, W., Mkaouer, M. W., Ludi, S., Ouni, A., and Jenhani, I. (2022b). On the identification of accessibility bug reports in open source systems. In Proceedings of the 19th International Web for All Conference, pages 1-11. DOI: 10.1145/3493612.3520471.

Aljedaani, W., Rustam, F., Ludi, S., Ouni, A., and Mkaouer, M. W. (2021). Learning sentiment analysis for accessibility user reviews. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), pages 239-246. IEEE. DOI: 10.1109/ASEW52652.2021.00053.

AlOmar, E. A., Aljedaani, W., Tamjeed, M., Mkaouer, M. W., and El-Glaly, Y. N. (2021). Finding the needle in a haystack: On the automatic identification of accessibility user reviews. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1-15. DOI: 10.1145/3411764.3445281.

Alshayban, A., Ahmed, I., and Malek, S. (2020). Accessibility issues in android apps: state of affairs, sentiments, and ways forward. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE), pages 1323-1334. IEEE. DOI: 10.1145/3377811.3380392.

Bi, T., Xia, X., Lo, D., Grundy, J., Zimmermann, T., and Ford, D. (2021). Accessibility in software practice: A practitioner’s perspective. ACM Transactions on Software Engineering and Methodology. DOI: 10.1145/3503508.

Chen, S., Chen, C., Fan, L., Fan, M., Zhan, X., and Liu, Y. (2022). Accessible or not? an empirical investigation of android app accessibility. IEEE Transactions on Software Engineering, 48(10):3954-3968. DOI: 10.1109/TSE.2021.3108162.

Ciurumelea, A., Schaufelbühl, A., Panichella, S., and Gall, H. C. (2017). Analyzing reviews and code of mobile apps for better release planning. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 91-102. IEEE. DOI: 10.1109/SANER.2017.7884612.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37-46. DOI: 10.1177/001316446002000104.

Dos Santos, P. S. H., Oliveira, A. D. A., De Jesus, T. B. N., Aljedaani, W., and Eler, M. M. (2023). Evolution may come with a price: analyzing user reviews to understand the impact of updates on mobile apps accessibility. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, IHC '23, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3638067.3638081.

Eler, M. M., Orlandin, L., and Oliveira, A. D. A. (2019). Do android app users care about accessibility? an analysis of user reviews on the google play store. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, pages 1-11. DOI: 10.1145/3357155.3358477.

Eler, M. M., Rojas, J. M., Ge, Y., and Fraser, G. (2018). Automated accessibility testing of mobile apps. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST), pages 116-126. IEEE. DOI: 10.1109/ICST.2018.00021.

Fleiss, J. L., Levin, B., Paik, M. C., et al. (1981). The measurement of interrater agreement. Statistical methods for rates and proportions, 2(212-236):22-23. DOI: 10.1002/0471445428.ch18.

Gao, J., Li, L., Bissyandé, T. F., and Klein, J. (2019). On the evolution of mobile app complexity. In 2019 24th International Conference on Engineering of Complex Computer Systems (ICECCS), pages 200-209. DOI: 10.1109/ICECCS.2019.00029.

Haggag, O., Grundy, J., Abdelrazek, M., and Haggag, S. (2022). Better addressing diverse accessibility issues in emerging apps: A case study using covid-19 apps. In 2022 IEEE/ACM 9th International Conference on Mobile Software Engineering and Systems (MobileSoft), pages 50-61. DOI: 10.1145/3524613.3527817.

Iacob, C. and Harrison, R. (2013). Retrieving and analyzing mobile apps feature requests from online reviews. In 2013 10th Working Conference on Mining Software Repositories (MSR), pages 41-44. DOI: 10.1109/MSR.2013.6624001.

Kraft, F. B. and Martin, C. L. (2001). Customer compliments as more than complementary feedback. Journal of Consumer Satisfaction, Dissatisfaction & Complaining Behavior, 14:1–13. Available online [link].

Lehman, M. (1980). Programs, life cycles, and laws of software evolution. Proceedings of the IEEE, 68(9):1060-1076. DOI: 10.1109/PROC.1980.11805.

Lehman, M. M. (1996). Laws of software evolution revisited. In Montangero, C., editor, Software Process Technology, pages 108-124, Berlin, Heidelberg. Springer Berlin Heidelberg. DOI: 10.1007/BFb0017737.

Liu, T., Wang, C., Huang, K., Liang, P., Zhang, B., Daneva, M., and van Sinderen, M. (2023). Rosematcher: Identifying the impact of user reviews on app updates. Information and Software Technology, page 107261. DOI: https://doi.org/10.1016/j.infsof.2023.107261.

McIlroy, S., Ali, N., and Hassan, A. E. (2016). Fresh apps: an empirical study of frequently-updated mobile apps in the google play store. Empirical Software Engineering, 21(3):1346-1370. DOI: 10.1007/s10664-015-9388-2.

Nayebi, M., Kuznetsov, K., Chen, P., Zeller, A., and Ruhe, G. (2018). Anatomy of functionality deletion: An exploratory study on mobile apps. In Proceedings of the 15th International Conference on Mining Software Repositories, MSR '18, page 243–253, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3196398.3196410.

Oliveira, A. D. A., Dos Santos, P. S. H., Marcílio Júnior, W. E., Aljedaani, W. M., Eler, D. M., and Eler, M. M. (2023). Analyzing accessibility reviews associated with visual disabilities or eye conditions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI '23, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3544548.3581315.

Othman, A., Dhouib, A., and Nasser Al Jabor, A. (2023). Fostering websites accessibility: A case study on the use of the large language models chatgpt for automatic remediation. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '23, page 707–713, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3594806.3596542.

Palomba, F., Linares-Vásquez, M., Bavota, G., Oliveto, R., Penta, M. D., Poshyvanyk, D., and Lucia, A. D. (2018). Crowdsourcing user reviews to support the evolution of mobile apps. Journal of Systems and Software, 137:143-162. DOI: 10.1016/j.jss.2017.11.043.

Potharaju, R., Rahman, M., and Carbunar, B. (2017). A longitudinal study of google play. IEEE Transactions on Computational Social Systems, PP:1-15. DOI: 10.1109/TCSS.2017.2732167.

Power, C., Freire, A., Petrie, H., and Swallow, D. (2012). Guidelines are only half of the story: accessibility problems encountered by blind users on the web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '12, page 433–442, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/2207676.2207736.

Reyes Arias, J. E., Kurtzhall, K., Pham, D., Mkaouer, M. W., and Elglaly, Y. N. (2022). Accessibility feedback in mobile application reviews: A dataset of reviews and accessibility guidelines. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA '22, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3491101.3519625.

Sandoval Alcocer, J. P., Merino, L., Fernandez-Blanco, A., Ravelo-Mendez, W., Escobar-Velásquez, C., and Linares-Vásquez, M. (2024). A developer’s guide to building and testing accessible mobile apps. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering, FSE 2024, page 713–715, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3663529.3663821.

Santiago, M. T. and Marques, A. B. (2022). Are user reviews useful for identifying accessibility issues that autistic users face? an exploratory study. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems, IHC '22, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3554364.3559114.

Taylor, V. F. and Martinovic, I. (2017). To update or not to update: Insights from a two-year study of android app evolution. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, ASIA CCS '17, page 45–57, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3052973.3052990.

W3C, W. W. W. C. (2023). Web content accessibility guidelines - wcag 2.2. Available online [link].

Yan, S. and Ramachandran, P. (2019). The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS), 12(1):1-31. DOI: 10.1145/3300176.

Yang, A. Z. H., Hassan, S., Zou, Y., and Hassan, A. E. (2022). An empirical study on release notes patterns of popular apps in the google play store. Empirical Software Engineering, 27(2):55. DOI: 10.1007/s10664-021-10086-2.