Lessons Learned in Human Evaluation: A Case Study of Augmented Reality in Books

Bruno Adachi; Júlia Gonzaga; Paula Fernandes; Sílvia Bim; Silvio Boss; Saul Delabrida; Natasha Valentim

doi:10.5753/jis.2026.6605

Authors

Bruno Adachi Federal University of Ouro Preto https://orcid.org/0009-0000-9740-1959
Júlia Gonzaga Federal University of Ouro Preto https://orcid.org/0009-0001-2743-5309
Paula Fernandes Federal University of Ouro Preto https://orcid.org/0000-0003-0338-7231
Sílvia Bim Federal University of Technology – Paraná https://orcid.org/0000-0001-8089-8284
Silvio Boss Federal University of Technology – Paraná https://orcid.org/0009-0000-9968-4665
Saul Delabrida Federal University of Ouro Preto https://orcid.org/0000-0002-8961-5313
Natasha Valentim Federal University of Paraná https://orcid.org/0000-0002-6027-3452

DOI:

https://doi.org/10.5753/jis.2026.6605

Keywords:

Augmented Reality, AR, Books, Human Evaluation, Usability

Abstract

Books adapted for Augmented Reality (AR) combine traditional printed pages with digital layers of interactive content such as 3D models, animations, or audio, accessed through mobile devices, creating a hybrid reading experience that merges physical and virtual elements. This paper presents a case study on the use of AR in printed books, focusing on the challenges and lessons learned from human-centered evaluation. Using the children’s book “Alan Turing: His Machines and His Secrets” as a case, we analyzed the feasibility of integrating AR features into works not originally designed for this purpose. The study involved testing two AR Software Development Kits (SDKs), Vuforia and ARCore. Results indicated that Vuforia offered the most efficient image recognition, supporting 71% of page detections when the smartphone was handheld and 76% with device support. A usability assessment with 18 participants, based on the System Usability Scale (SUS), revealed positive acceptance of AR interactions but also highlighted ergonomic limitations when handling books and mobile devices simultaneously. The findings emphasize the importance of experimental design, pilot testing, and consideration of user habits in AR evaluation studies. Overall, this work provides practical insights to guide future illustrators, developers, and researchers in adapting books to AR contexts.

Downloads

Download data is not yet available.

References

Abazi-Bexheti, L., Kadriu, A., and Apostolova, M. (2022). Research on vr/ar integration in education. In 2022 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO), pages 563–567. DOI: https://doi.org/10.23919/MIPRO55190.2022.9803398.

Adachi, B., Bim, S., and Delabrida, S. (2025). Lessons learned in human evaluation: A case study of augmented reality in books. In Proceedings of the ACM International Conference on Interactive Media Experiences Workshops, pages 136–139, Porto Alegre, RS, Brasil. SBC. DOI: https://doi.org/10.5753/imxw.2025.8905.

Adachi, B. H., Gonzaga, J. C., Fernandes, P. C., Silva, S. E. D., Bim, S. A., and Boss, S. L. B. (2024). Augmented reality in books: An evaluation of alan turing book. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, IHC '23, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3638067.3638124.

Al-Ali, H., Bazzaza, M. W., Zemerly, M. J., and Ng, J. W. P. (2016). Myvision air: An augmented interactive reality book mobile application. In 2016 IEEE Global Engineering Education Conference (EDUCON), pages 741–745. DOI: https://doi.org/10.1109/EDUCON.2016.7474634.

Alfailakawi, A. (2016). The effects of visual training on vision functions and shooting perfomance level among young handball players. Ovidius University Annals, Series Physical Education & Sport/Science, Movement & Health, 16(1).

Apple (2026). Detecting images in an ar experience. [link]. Accessed on 09 July 2026.

Bittencourt, P. A. S. and Albino, J. P. (2017). O uso das tecnologias digitais na educação do século xxi. Revista Ibero-Americana de Estudos em Educação, page 205–214. DOI: https://doi.org/10.21723/riaee.v12.n1.9433.

Boss, S. L. B. and Bim, S. A. (2022). Alan Turing: suas máquinas e seus segredos. Editora Blucher, São Paulo, SP, Brasil.

BRASIL (2018). Base Nacional Comum Curricular. [link]. Accessed on 09 July 2026.

BRASIL (2022). Base Nacional Comum Curricular: Computação – Complemento à BNCC. [link]. Anexo ao Parecer CNE/CEB nº 2/2022. Accessed on 09 July 2026.

Brooke, J. (1995). Sus: A quick and dirty usability scale. Usability Eval. Ind., 189.

Chen, Y., Wang, Q., Chen, H., Song, X., Tang, H., and Tian, M. (2019). An overview of augmented reality technology. Journal of Physics: Conference Series, 1237:022082. DOI: https://doi.org/10.1088/1742-6596/1237/2/022082.

Cheng, D., Wang, Q., Liu, Y., Chen, H., Ni, D., Wang, X., Yao, C., Hou, Q., Hou, W., Luo, G., and Wang, Y. (2021). Design and manufacture ar head-mounted displays: A review and outlook. Light: Advanced Manufacturing, 2(3):350–369. DOI: https://doi.org/10.37188/lam.2021.024.

Cheng, K.-H. (2017). Reading an augmented reality book: An exploration of learners' cognitive load, motivation, and attitudes. Australasian Journal of Educational Technology, 33(4). DOI: https://doi.org/10.14742/ajet.2820.

Cheng, K.-H. (2019). Parents' user experiences of augmented reality book reading: perceptions, expectations, and intentions. Educational Technology Research and Development, 67(2):303–315. DOI: https://doi.org/10.1007/s11423-018-9611-0.

Cumbo, B. and Selwyn, N. (2021). Using participatory design approaches in educational research. International Journal of Research Method in Education, 45:1–13. DOI: https://doi.org/10.1080/1743727X.2021.1902981.

Danaei, D., Jamali, H. R., Mansourian, Y., and Rastegarpour, H. (2020). Comparing reading comprehension between children reading augmented reality and print storybooks. Computers & Education, 153:103900. DOI: https://doi.org/10.1016/j.compedu.2020.103900.

De Barros, A. F. (2019). O uso das tecnologias na educação como ferramentas de aprendizado. Revista Científica Semana Acadêmica, pages 01–13.

Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(5910):66–69. DOI: https://doi.org/10.1126/science.1167311.

Del-Masso, M. C. S., Cotta, M. A. d. C., and Santos, M. A. P. (2014). Ética em pesquisa científica: conceitos e finalidades. São Paulo: Núcleo de Educação a Distância da Unesp.

Demitriadou, E., Stavroulia, K.-E., and Lanitis, A. (2019). Comparative evaluation of virtual and augmented reality for teaching mathematics in primary education. Education and Information Technologies, 25(1):381–401. DOI: https://doi.org/10.1007/s10639-019-09973-5.

Dias, B., Keller, B., and Delabrida, S. (2019). Evaluation of augmented reality sdks for classroom teaching. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, IHC '19, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3357155.3358447.

Dong, C. and Si, Z. (2018). The Research and Application of Augmented Reality in 3D Interactive Books for Children, pages 293–299. DOI: https://doi.org/10.1007/978-981-10-7629-9_35.

Dong, Z., Zhang, J., Bai, X., Clark, A., Lindeman, R. W., He, W., and Piumsomboon, T. (2022). Touch-move-release: Studies of surface and motion gestures for mobile augmented reality. Frontiers in Virtual Reality, Volume 3 - 2022. DOI: https://doi.org/10.3389/frvir.2022.927258.

Freitas, S. (2018). Are games effective learning tools? a review of educational games. Educational Technology and Society, 21:74–84.

Friesen, S. and Jacobsen, M. (2021). Collaborative design of professional graduate programs in education. International Journal of Designs for Learning, 12(1):64–76. DOI: https://doi.org/10.14434/ijdl.v12i1.25778.

GitHub (2025). What is an sdk? [link]. Accessed on 11 July 2026.

Google (2026). Add dimension to images. [link]. Accessed on 09 July 2026.

Interaction Design Foundation (2016). User centered design (ucd). [link]. Accessed on 11 July 2026.

Itoh, Y., Langlotz, T., Sutton, J., and Plopski, A. (2021). Towards indistinguishable augmented reality: A survey on optical see-through head-mounted displays. ACM Comput. Surv., 54(6). DOI: https://doi.org/10.1145/3453157.

Kim, K., Billinghurst, M., Bruder, G., Duh, H. B.-L., and Welch, G. F. (2018). Revisiting trends in augmented reality research: A review of the 2nd decade of ismar (2008–2017). IEEE Transactions on Visualization and Computer Graphics, 24(11):2947–2962. DOI: https://doi.org/10.1109/tvcg.2018.2868591.

Kryeziu, S. A., Avdiu, T. A., and Avdiu, A. (2021). Examining the teachers, administrators and parents' view on drawbacks of technology use in education. Ilkogretim Online, 20(2). DOI: https://doi.org/10.17762/pae.v58i2.3713.

Lai, A. S., Wong, C. Y., and Lo, O. C. (2015). Applying augmented reality technology to book publication business. In 2015 IEEE 12th International Conference on e-Business Engineering, pages 281–286. DOI: https://doi.org/10.1109/ICEBE.2015.55.

Ma, Q. and Liu, L. (2005). The Technology Acceptance Model. DOI: https://doi.org/10.4018/9781591404743.ch006.ch000.

Maciel, C., Bim, S. A., and da Silva Figueiredo, K. (2018). Digital girls program: disseminating computer science to girls in brazil. New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3195570.3195574.

Mann, S., Havens, J. C., Iorio, J., Yuan, Y., and Furness, T. (2018). All reality: values, taxonomy, and continuum, for virtual, augmented, extended/mixed (x), mediated (x, y), and multimediated reality/intelligence. In Presented at the AWE 2018. DOI: https://doi.org/10.48550/arXiv.1804.08386.

Mazon, J., de Castro Pasquini, G., da Rosa Pimentel, A. C., Souza Capistrano, V., Lucas de Oliveira Ramalho, J., Marino Costa, A., Pozzebon, E., and Bolan Frigo, L. (2022). Curso rafec - realidade aumentada facilitando o ensino das ciências: Uma nova possibilidade ao professor. RECIMA21 - Revista Científica Multidisciplinar, 3(8):e381797. DOI: https://doi.org/10.47820/recima21.v3i8.1797.

Ribeiro, L., Foss, L., Cavalheiro, S. A. D. C., Kniphoff da Cruz, M. E. J., and Soares de França, R. (2023). The brazilian school computing standard. SIGCSE 2023, page 53–58, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3545945.3569863.

Skarbez, R., Smith, M., and Whitton, M. C. (2021). Revisiting milgram and kishino's reality-virtuality continuum. Frontiers in Virtual Reality, Volume 2 - 2021. DOI: https://doi.org/10.3389/frvir.2021.647997.

Søegaard, M. (2024). System usability scale for datadriven ux. [link]. Accessed on 11 July 2025.

Unsplash (2026). Unsplash. [link]. Accessed on 09 July 2026.

Vuforia (2026). Best practices for designing and developing image-based targets. [link]. Accessed on 09 July 2026.

Zambiasi, P. and Pozzebon, E. (2025). Provadores virtuais de realidade aumentada: Diferenças geracionais na aceitação da tecnologia: Um estudo experimental no contexto brasileiro de e-commerce de moda. Revista de Geopolítica, 16(5):e811. DOI: https://doi.org/10.56238/revgeov16n5-043.