An Updated Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holographic Solutions

Thiago Campos; Maria Castello; Eduardo Damasceno; Natasha Valentim

doi:10.5753/jis.2025.4694

Authors

Thiago Campos Federal University of Paraná https://orcid.org/0000-0003-1038-4004
Maria Castello Federal University of Paraná https://orcid.org/0009-0002-2735-3108
Eduardo Damasceno Federal University of Technology of Paraná https://orcid.org/0000-0002-6246-1246
Natasha Valentim Federal University of Paraná https://orcid.org/0000-0002-6027-3452

DOI:

https://doi.org/10.5753/jis.2025.4694

Keywords:

Usability, User Experience, Evaluation, Touchable Hologram, Mixed Reality, Augmented Reality, Systematic Mapping Study

Abstract

This article presents an extended Systematic Mapping Study (SMS) focused on usability and user experience (UX) valuation technologies for Touchable Holographic Solutions (THS). Given the growing integration of holograms in Augmented Reality (AR) and Mixed Reality (MR) settings, evaluating usability and UX becomes highly important. Our study expands on previous research by analyzing an additional two years of publications, covering 5429 studies, and selecting 65 that discuss 200 evaluation technologies. The main problem addressed is the gap in comprehensive evaluation frameworks that integrate usability and UX criteria. We followed systematic guidelines to identify and analyze evaluation technologies, highlighting an increased focus on UX alongside traditional usability. Key findings include the persistent emphasis on time efficiency in usability evaluations and the dominance of generic UX, usability, and pleasure/fun in UX assessments. However, unique aspects of MR, such as presence, are often overlooked. The study also reveals a preference for empirical validation through controlled experiments and case studies, although few technologies have undergone such validation. Head-mounted displays (HMDs) and smart glasses, especially Microsoft Hololens™, remain prevalent due to their advanced capabilities. Our findings underscore the need for integrated evaluation technologies and empirical validation to ensure reliability. This work contributes to the Human-Computer Interaction (HCI) area by mapping current evaluation technologies, identifying research gaps, and providing a foundation for developing innovative and effective evaluation methods for THS, thus advancing the understanding and improvement of user interaction in immersive environments.

Downloads

References

Adhikarla, V. K., Jakus, G., and Sodnik, J. (2015). Design and Evaluation of Freehand Gesture Interaction for Light Field Display. In Kurosu, M., editor, Human-Computer Interaction: Interaction Technologies, volume 9170, pages 54–65. Springer International Publishing, Cham. DOI: https://doi.org/0.1007/978-3-319-20916-6_6.

Aigner, R., Wigdor, D., Benko, H., Haller, M., Lindbauer, D., Ion, A., Zhao, S., and Koh, J. T. K. V. (2012). Understanding Mid-Air Hand Gestures: A Study of Human Preferences in Usage of Gesture Types for HCI. Technical Report MSR-TR-2012-111, Microsoft Research.

Altman, D. G. (1990). Practical Statistics for Medical Research. Chapman and Hall/CRC, London, 1st edition edition.

Aslan, I., Dang, C. T., Schlagowski, R., Dietz, M., Brain, F., and André, E. (2019). Put that Hologram there - Probing Mobile Interaction Experiences for a Vision of Mixed Material Public Spaces. In Proceedings of the 9th International Conference on the Internet of Things, pages 1–4, Bilbao Spain. ACM. DOI: https://doi.org/0.1145/3365871.3365877.

Bai, Z. and Blackwell, A. F. (2012). Analytic review of usability evaluation in ISMAR. Interacting with Computers, 24(6):450–460. DOI: https://doi.org/0.1016/j.intcom.2012.07.004.

Barbosa, S., Silva, B., Silveira, M., Gasparini, I., Darin, T., and Barbosa, G. (2021). Interação Humano-Computador e Experiência do Usuário. Autopublicação, Rio de Janeiro, 1a ed. edition.

Basili, V. R., Caldiera, G., and Rombach, H. D. (1994). The Goal Question Metric Approach.

Becker, V., Rauchenstein, F., and Sörös, G. (2019). Investigating Universal Appliance Control through Wearable Augmented Reality. In Proceedings of the 10th Augmented Human International Conference 2019, pages 1–9, Reims France. ACM. DOI: https://doi.org/0.1145/3311823.3311853.

Berkman, M. I. and Akan, E. (2019). Presence and Immersion in Virtual Reality. In Lee, N., editor, Encyclopedia of Computer Graphics and Games, pages 1–10. Springer International Publishing, Cham. DOI: https://doi.org/0.1007/978-3-319-08234-9_162-1.

Bevan, N., Carter, J., and Harker, S. (2015). ISO 9241-11 Revised: What Have We Learnt About Usability Since 1998? In Kurosu, M., editor, Human-Computer Interaction: Design and Evaluation, Lecture Notes in Computer Science, pages 143–151, Cham. Springer International Publishing. DOI: https://doi.org/0.1007/978-3-319-20901-2_13.

Borg, G. (1998). Borg’s perceived exertion and pain scales. Borg’s perceived exertion and pain scales. Human Kinetics, Champaign, IL, US. Pages: viii, 104.

Bowman, D. A. and McMahan, R. P. (2007). Virtual Reality: How Much Immersion Is Enough? Computer, 40(7):36–43. DOI: https://doi.org/0.1109/MC.2007.257.

Bozgeyikli, E. and Bozgeyikli, L. L. (2021). Evaluating Object Manipulation Interaction Techniques in Mixed Reality: Tangible User Interfaces and Gesture. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pages 778–787. ISSN: 2642-5254. DOI: https://doi.org/0.1109/VR50410.2021.00105.

Brancati, N., Caggianese, G., Pietro, G. D., Frucci, M., Gallo, L., and Neroni, P. (2015). Usability Evaluation of a Wearable Augmented Reality System for the Enjoyment of the Cultural Heritage. In 2015 11th International Conference on Signal-Image Technology Internet-Based Systems (SITIS), pages 768–774, Bangkok, Thailand. IEEE. DOI: https://doi.org/0.1109/SITIS.2015.98.

Brooke, J. (1996). SUS: A ’Quick and Dirty’ Usability Scale. In Usability Evaluation In Industry, pages 207–212. CRC Press, London, 1st edition.

Campos, T. P. D., Damasceno, E. F., and Valentim, N. M. C. (2022). Porifera: A Collaborative Tool to Support Systematic Literature Review and Systematic Mapping Study. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering, SBES ’22, pages 452–457, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3555228.3555273.

Campos, T. P. d., Damasceno, E. F., and Valentim, N. M. C. (2023b). Usability and User Experience Evaluation of Touchable Holographic solutions: A Systematic Mapping Study. In IHC ’23: Proceedings of the 22st Brazilian Symposium on Human Factors in Computing Systems, IHC ’23, pages 1–13, Maceio, Brazil. ACM. DOI: https://doi.org/0.1145/3638067.3638071.

Campos, T., Castello, M., Valentim, N., and Damasceno, E. (2024). An Updated Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holograms: Technical Report. page 944822 Bytes. DOI: https://doi.org/0.6084/M9.FIGSHARE.26261951.

Campos, T., Valentim, N., and Damasceno, E. (2023a). A Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holograms: Technical Report. Technical report, Federal University of Paraná, Curitiba, Brazil. DOI: https://doi.org/10.6084/m9.figshare.22114355.

Cao, Y., Xu, Z., Glenn, T., Huo, K., and Ramani, K. (2018). Ani-Bot: A Modular Robotics System Supporting Creation, Tweaking, and Usage with Mixed-Reality Interactions. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction, pages 419–428, Stockholm Sweden. ACM. DOI: https://doi.org/0.1145/3173225.3173226.

Caruso, G., Carulli, M., and Bordegoni, M. (2015). Augmented Reality System for the Visualization and Interaction with 3D Digital Models in a Wide Environment. Computer-Aided Design and Applications, 12(1):86–95. DOI: https://doi.org/0.1080/16864360.2014.949579.

Chaoui, K., Bouzidi-Hassini, S., and Bellik, Y. (2023). SUIL: a modeling language for spatial user interaction. Journal of Reliable Intelligent Environments, 9(2):161–181. DOI: https://doi.org/0.1007/s40860-021-00164-z.

Chien, P. H. and Lin, Y. C. (2021). Gesture-based headmounted augmented reality game development using leap motion and usability evaluation: 15th International Conference on Interfaces and Human Computer Interaction, IHCI 2021 and 14th International Conference on Game and Entertainment Technologies, GET 2021 - Held at the 15th Multi-Conference on Computer Science and Information Systems, MCCSIS 2021. 15th International Conference on Interfaces and Human Computer Interaction, IHCI 2021 and 14th International Conference on Game and Entertainment Technologies, GET 2021 - Held at the 15th Multi-Conference on Computer Science and Information Systems, MCCSIS 2021, pages 149–156.

Daskalogrigorakis, G., McNamara, A., Marinakis, A., Antoniadis, A., and Mania, K. (2022). Glance-Box: Multi-LOD Glanceable Interfaces for Machine Shop Guidance in Augmented Reality using Blink and Hand Interaction. In 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 315–321. ISSN: 2771-1110. DOI: https://doi.org/0.1109/ISMARAdjunct57072.2022.00070.

Dehghani, M., Lee, S. H. M., and Mashatan, A. (2020). Touching holograms with windows mixed reality: Renovating the consumer retailing services. Technology in Society, 63:101394. DOI: https://doi.org/0.1016/j.techsoc.2020.101394.

Dudley, J. J., Vertanen, K., and Kristensson, P. O. (2018). Fast and Precise Touch-Based Text Entry for Head-Mounted Augmented Reality with Variable Occlusion. ACM Transactions on Computer-Human Interaction, 25(6):1–40. DOI: https://doi.org/0.1145/3232163.

Dünser, A., Grasset, R., and Billinghurst, M. (2008). A survey of evaluation techniques used in augmented reality studies. In ACM SIGGRAPH ASIA 2008 courses on - SIGGRAPH Asia ’08, pages 1–27, Singapore. ACM Press. DOI: https://doi.org/0.1145/1508044.1508049.

Easterbrook, S., Singer, J., Storey, M.-A., and Damian, D. (2008). Selecting Empirical Methods for Software Engineering Research. In Shull, F., Singer, J., and Sjøberg, D. I. K., editors, Guide to Advanced Empirical Software Engineering, pages 285–311. Springer, London. DOI: https://doi.org/0.1007/978-1-84800-044-5_11.

Favreau, J. (2010). Iron Man 2. Translated title: Homem de Ferro 2 IMDb ID: tt1228705 event-location: USA publisher-place: Estados Unidos.

Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378–382. Place: US Publisher: American Psychological Association. DOI: https://doi.org/0.1037/h0031619.

Flick, C. D., Harris, C. J., Yonkers, N. T., Norouzi, N., Erickson, A., Choudhary, Z., Gottsacker, M., Bruder, G., and Welch, G. (2021). Trade-offs in Augmented Reality User Interfaces for Controlling a Smart Environment. In Proceedings of the 2021 ACM Symposium on Spatial User Interaction, SUI ’21, pages 1–11, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3485279.3485288.

Frutos-Pascual, M., Creed, C., and Williams, I. (2019). Head Mounted Display Interaction Evaluation: Manipulating Virtual Objects in Augmented Reality. IN Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., and Zaphiris, P., editors, Human-Computer Interaction – INTERACT 2019, volume 11749, pages 287–308. Springer International Publishing, Cham. DOI: https://doi.org/0.1007/978-3-030-29390-1_16.

Ha, T., Feiner, S., and Woo, W. (2014). WeARHand: Headworn, RGB-D camera-based, bare-hand user interface with visually enhanced depth perception. In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 219–228, Munich, Germany. IEEE. DOI: https://doi.org/0.1109/ISMAR.2014.6948431.

Hart, S. G. (2006). Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 50(9):904–908. DOI: https://doi.org/0.1177/154193120605000909.

Hart, S. G. and Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Hancock, P. A. and Meshkati, N., editors, Advances in Psychology, volume 52 of Human Mental Workload, pages 139–183. North-Holland, Amsterdan, Netherlands. DOI: https://doi.org/0.1016/S0166-4115(08)62386-9.

Hassenzahl, M. (2011). User Experience and Experience Design. In The Encyclopedia of Human-Computer Interaction. Interaction Design Fundation, online, 2nd edition.

He, Y., Hu, Y., Feng, H., Li, C., and Shen, X. (2022). Comparative Analysis of 3D Interactive Modes in Different Object Layouts in Mixed Reality. In Proceedings of the Ninth International Symposium of Chinese CHI, Chinese CHI ’21, pages 120–126, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3490355.3490371.

Higuchi, M. and Komuro, T. (2013). AR typing interface for mobile devices. In Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia - MUM ’13, pages 1–8, Lule 229;, Sweden. ACM Press. DOI: https://doi.org/0.1145/2541831.2541847.

Hu, J., Dudley, J. J., and Kristensson, P. O. (2022). An Evaluation of Caret Navigation Methods for Text Editing in Augmented Reality. In 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 640–645. ISSN: 2771-1110. DOI: https://doi.org/0.1109/ISMARAdjunct57072.2022.00132.

Huang, C. D., Goo, J., Nam, K., and Yoo, C. W. (2017). Smart tourism technologies in travel planning: The role of exploration and exploitation. Information Management, 54(6):757–770. DOI: https://doi.org/0.1016/j.im.2016.11.010.

Huang, Z., Li, W., and Hui, P. (2015). Ubii: Towards Seamless Interaction between Digital and Physical Worlds. In Proceedings of the 23rd ACM international conference on Multimedia, pages 341–350, Brisbane Australia. ACM. DOI: https://doi.org/0.1145/2733373.2806266.

ISO (2011). ISO/IEC 25010:2011 - Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. Technical report, International Organization for Standardization, Geneva, Switzerland.

ISO (2018). ISO 9241-11:2018 Ergonomics of humansystem interaction — Part 11: Usability: Definitions and concepts. Technical report, International Organization for Standardization, Geneva, Switzerland.

International Society for Presence Research (2000). Presence defined.

Ivory, M. Y. and Hearst, M. A. (2001). The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys, 33(4):470–516. DOI: https://doi.org/0.1145/503112.503114.

Jang, J., Frier, W., and Park, J. (2022). Multimodal Volume Data Exploration through Mid-Air Haptics. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 243–251. ISSN: 1554-7868. DOI: https://doi.org/0.1109/ISMAR55827.2022.00039.

Jasche, F. and Ludwig, T. (2020). PrintARface: Supporting the Exploration of Cyber-Physical Systems through Augmented Reality. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, pages 1–12, Tallinn Estonia. ACM. DOI: https://doi.org/0.1145/3419249.3420162.

Kang, H. J., Shin, J.-h., and Ponto, K. (2020). A Comparative Analysis of 3D User Interaction: How to Move Virtual Objects in Mixed Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 275–284, Atlanta, GA, USA. IEEE. DOI: https://doi.org/0.1109/VR46266.2020.00047.

Kim, J. H., Ari, H., Madasu, C., and Hwang, J. (2020). Evaluation of the biomechanical stress in the neck and shoulders during augmented reality interactions. Applied Ergonomics, 88:103175. DOI: https://doi.org/0.1016/j.apergo.2020.103175.

Kim, M. and Lee, J. Y. (2016). Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in mobile augmented reality. Multimedia Tools and Applications, 75(23):16529–16550. DOI: https://doi.org/0.1007/s11042-016-3355-9.

Kim, M., Choi, S. H., Park, K.-B., and Lee, J. Y. (2019). User Interactions for Augmented Reality Smart Glasses: A Comparative Evaluation of Visual Contexts and Interaction Gestures. Applied Sciences, 9(15):3171. DOI: https://doi.org/0.3390/app9153171.

Kitchenham, B. A., Budgen, D., and Brereton, P. (2016). Evidence-based software engineering and systematic reviews. Chapman Hall/CRC innovations in software engineering and software development. CRC Press, Boca Raton. OCLC: ocn932588149.

Koçer Özgün, F. N. and Alaçam, S. (2023). AN EVALUATION OF AUGMENTED REALITY-BASED USER INTERFACES IN THE DESIGN PROCESS. Architecture and Planning Journal (APJ), 28(3). DOI: https://doi.org/0.54729/2789-8547.1234.

Landis, J. R. and Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1):159. Publisher: JSTOR. DOI: https://doi.org/0.2307/2529310.

Laugwitz, B., Held, T., and Schrepp, M. (2008). Construction and Evaluation of a User Experience Questionnaire. In Holzinger, A., editor, HCI and Usability for Education and Work, Lecture Notes in Computer Science, pages 63–76, Graz, Austria. Springer. DOI: https://doi.org/0.1007/978-3-540-89350-9_6.

Lee, J., Olwal, A., Ishii, H., and Boulanger, C. (2013). SpaceTop: integrating 2D and spatial 3D interactions in a see-through desktop environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 189–192, Paris France. ACM. DOI: https://doi.org/0.1145/2470654.2470680.

Lee, L. H., Braud, T., Bijarbooneh, F. H., and Hui, P. (2019a). TiPoint: detecting fingertip for mid-air interaction on computational resource constrained smartglasses. In Proceedings of the 23rd International Symposium on Wearable Computers, pages 118–122, London United Kingdom. ACM. DOI: https://doi.org/0.1145/3341163.3347723.

Lee, L. H., Braud, T., Bijarbooneh, F. H., and Hui, P. (2020). UbiPoint: towards non-intrusive mid-air interaction for hardware constrained smart glasses. In Proceedings of the 11th ACM Multimedia Systems Conference, pages 190–201, Istanbul Turkey. ACM. DOI: https://doi.org/0.1145/3339825.3391870.

Lee, L. H., Yung Lam, K., Yau, Y. P., Braud, T., and Hui, P. (2019b). HIBEY: Hide the Keyboard in Augmented Reality. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom, pages 1–10, Kyoto, Japan. IEEE. DOI: https://doi.org/0.1109/PERCOM.2019.8767420.

Li, Y., Hu, Y., Wang, Z., and Shen, X. (2022). Evaluating the Object-Centered User Interface in Head-Worn Mixed Reality Environment. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 414–421. ISSN: 1554-7868. DOI: https://doi.org/0.1109/ISMAR55827.2022.00057.

Lin, S., Cheng, H. F., Li, W., Huang, Z., Hui, P., and Peylo, C. (2017). Ubii: Physical World Interaction Through Augmented Reality. IEEE Transactions on Mobile Computing, 16(3):872–885. DOI: https://doi.org/0.1109/TMC.2016.2567378.

Liu, X., Meng, X., Spittle, B., Xu, W., Gao, B., and Liang, H.-N. (2023). Exploring Text Selection in Augmented Reality Systems. In Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry, VRCAI ’22, pages 1–8, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3574131.3574459.

Lund, A. M. (2001). Measuring usability with the USE questionnaire. Usability interface, 8(2):3–6.

Lystbæk, M. N., Pfeuffer, K., Grønbæk, J. E. S., and Gellersen, H. (2022a). Exploring Gaze for Assisting Freehand Selection-based Text Entry in AR. Proceedings of the ACM on Human-Computer Interaction, 6(ETRA):141:1–141:16. DOI: https://doi.org/0.1145/3530882.

Lystbæk, M. N., Rosenberg, P., Pfeuffer, K., Grønbæk, J. E., and Gellersen, H. (2022b). Gaze-Hand Alignment: Combining Eye Gaze and Mid-Air Pointing for Interacting with Menus in Augmented Reality. Proceedings of the ACM on Human-Computer Interaction, 6(ETRA):145:1–145:18. DOI: https://doi.org/0.1145/3530886.

MDPI (2024). Augmented Reality: Current Trends, Challenges and Prospects.

Mahajan, K., Groechel, T., Pakkar, R., Cordero, J., Lee, H., and Matarić, M. J. (2020). Adapting Usability Metrics for a Socially Assistive, Kinesthetic, Mixed Reality Robot Tutoring Environment. In Wagner, A. R., Feil-Seifer, D., Haring, K. S., Rossi, S., Williams, T., He, H.,and Sam Ge, S., editors, Social Robotics, volume 12483, pages 381–391. Springer International Publishing, Cham. DOI: https://doi.org/0.1007/978-3-030-62056-1_32.

Marasco, A., Buonincontri, P., van Niekerk, M., Orlowski, M., and Okumus, F. (2018). Exploring the role of nextgeneration virtual technologies in destination marketing. Journal of Destination Marketing Management, 9:138–148. DOI: https://doi.org/0.1016/j.jdmm.2017.12.002.

Marques, L., Barcellos, M. P., Gadelha, B., and Conte, T. (2024). Characterizing UX Assessment in the Context of Immersive Experiences: A Systematic Mapping Study. International Journal of Human–Computer Interaction, pages 1–17. DOI: https://doi.org/0.1080/10447318.2024.2351711.

Matsumaru, T., Septiana, A. I., Horiuchi, K., and Graduate School of Information, Production, and Systems, Waseda University 2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808-0135, Japan (2019). Three-Dimensional Aerial Image Interface, 3DAII. Journal of Robotics and Mechatronics, 31(5):657–670. DOI: https://doi.org/0.20965/jrm.2019.p0657.

Mau-Tsuen Yang and Wan-Che Liao (2014). Computer-Assisted Culture Learning in an Online Augmented Reality Environment Based on Free-Hand Gesture Interaction. IEEE Transactions on Learning Technologies, 7(2):107–117. DOI: https://doi.org/0.1109/TLT.2014.2307297.

McCord, K. H., Ayer, S. K., Perry, L. A., Patil, K. R., London, J. S., Khoury, V., and Wu, W. (2022). Student Approaches and Performance in Element Sequencing Tasks Using 2D and Augmented Reality Formats. Education Sciences, 12(4):247. DOI: https://doi.org/0.3390/educsci12040247.

Mendes, E., Wohlin, C., Felizardo, K., and Kalinowski, M. (2020). When to update systematic literature reviews in software engineering. Journal of Systems and Software, 167:110607. DOI: https://doi.org/0.1016/j.jss.2020.110607.

Merino, L., Schwarzl, M., Kraus, M., Sedlmair, M., Schmalstieg, D., and Weiskopf, D. (2020). Evaluating Mixed and Augmented Reality: A Systematic Literature Review (2009-2019). In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 438–451, Porto de Galinhas, Brazil. IEEE. DOI: https://doi.org/0.1109/ISMAR50242.2020.00069.

Mestre, D. and Vercher, J.-L. (2011). Immersion and presence. In Virtual Reality: Concepts and Technologies, pages 93–104. CRC Press, Boca Raton, FL, USA.

Milgram, P. and Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12):1321–1329. Publisher: The Institute of Electronics, Information and Communication Engineers.

Munsinger, B., White, G., and Quarles, J. (2019). The Usability of the Microsoft HoloLens for an Augmented Reality Game to Teach Elementary School Children. In 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), pages 1–4, Vienna, Austria. IEEE. DOI: https://doi.org/0.1109/VSGames.2019.8864548.

Nielsen, J. (1993). Usability engineering. Academic Press, Boston.

Nielsen, J. (2012). Usability 101: Introduction to Usability.

Pei, S., Chen, A., Lee, J., and Zhang, Y. (2022). Hand Interfaces: Using Hands to Imitate Objects in AR/VR for Expressive Interactions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, pages 1–16, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3491102.3501898.

Pereira, G. A. F., Bacha, J. M. R., Silva, I. B. A. N., Kim, D. H. C., Pompeu, J. E., and Lopes, R. d. D. (2021). Virtual Reality and Augmented Reality Exergames for older fallers: considerations about design and applicability by physical therapists. In Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames), pages 949–956. SBC. DOI: https://doi.org/0.5753/sbgames_estendido.2021.19734.

Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008). Systematic mapping studies in software engineering. 12th International Conference on Evaluation and Assessment in Software Engineering, EASE 2008, I(June):1–10. DOI: https://doi.org/0.14236/ewic/ease2008.8.

Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18. DOI: https://doi.org/0.1016/j.infsof.2015.03.007.

Plasson, C., Blanch, R., and Nigay, L. (2022). Selection Techniques for 3D Extended Desktop Workstation with AR HMD. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 460–469. ISSN: 1554-7868. DOI: https://doi.org/0.1109/ISMAR55827.2022.00062.

Plasson, C., Cunin, D., Laurillau, Y., and Nigay, L. (2019). Tabletop AR with HMD and Tablet: A Comparative Study for 3D Selection. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces, pages 409–414, Daejeon Republic of Korea. ACM. DOI: https://doi.org/0.1145/3343055.3360760.

Qian, J., Ma, J., Li, X., Attal, B., Lai, H., Tompkin, J., Hughes, J. F., and Huang, J. (2019). Portal-ble: Intuitive Free-hand Manipulation in Unbounded Smartphone-based Augmented Reality. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pages 133–145, New Orleans LA USA. ACM. DOI: https://doi.org/0.1145/3332165.3347904.

Qian, X., He, F., Hu, X., Wang, T., and Ramani, K. (2022). ARnnotate: An Augmented Reality Interface for Collecting Custom Dataset of 3D Hand-Object Interaction Pose Estimation. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, pages 1–14, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3526113.3545663.

Roto, V., Obrist, M., and Väänänen-vainio mattila, K. (2009). User Experience Evaluation Methods in Academic and Industrial Contexts. In Proceedings of the Workshop UXEM’09, volume II, page 4 p, Uppsala, Sweden. Springer.

Sand, A., Rakkolainen, I., Isokoski, P., Raisamo, R., and Palovuori, K. (2015). Light-weight immaterial particle displays with mid-air tactile feedback. In 2015 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), pages 1–5, Ottawa, ON, Canada. IEEE. DOI: https://doi.org/0.1109/HAVE.2015.7359448.

Santos, G., Rocha, A. R., Conte, T., Barcellos, M. P., and Prikladnicki, R. (2012). Strategic Alignment between Academy and Industry: A Virtuous Cycle to Promote Innovation in Technology. In 26th Brazilian Symposium on Software Engineering SBES 2012, pages 196–200, Natal, Brazil. IEEE Computer Society. DOI: https://doi.org/0.1109/SBES.2012.31.

Schubert, T., Friedmann, F., and Regenbrecht, H. (2001). The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments, 10(3):266–281. DOI: https://doi.org/0.1162/105474601300343603.

Seguí, M. d. M., Cabrero-García, J., Crespo, A., Verdú, J., and Ronda, E. (2015). A reliable and valid questionnaire was developed to measure computer vision syndrome at the workplace. Journal of Clinical Epidemiology, 68(6):662–673. DOI: https://doi.org/0.1016/j.jclinepi.2015.01.015.

Seiger, R., Kühn, R., Korzetz, M., and Aßmann, U. (2021). HoloFlows: modelling of processes for the Internet of Things in mixed reality. Software and Systems Modeling, 20(5):1465–1489. DOI: https://doi.org/0.1007/s10270-020-00859-6.

Serrano, R., Morillo, P., Casas, S., and Cruz-Neira, C. (2022). An empirical evaluation of two natural hand interaction systems in augmented reality. Multimedia Tools and Applications, 81(22):31657–31683. DOI: https://doi.org/0.1007/s11042-022-12864-6.

Shareef, M. A., Baabdullah, A., Dutta, S., Kumar, V., and Dwivedi, Y. K. (2018). Consumer adoption of mobile banking services: An empirical examination of factors according to adoption stages. Journal of Retailing and Consumer Services, 43(C):54–67.

Shen, J., Hu, J., Dudley, J. J., and Kristensson, P. O. (2022). Personalization of a Mid-Air Gesture Keyboard using Multi-Objective Bayesian Optimization. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 702–710. ISSN: 1554-7868. DOI: https://doi.org/0.1109/ISMAR55827.2022.00088.

Shim, J., Yang, Y., Kang, N., Seo, J., and Han, T.-D. (2016). Gesture-based interactive augmented reality content authoring system using HMD. Virtual Reality, 20(1):57–69. DOI: https://doi.org/0.1007/s10055-016-0282-z.

Skarbez, R., Brooks, Jr., F. P., and Whitton, M. C. (2017). A Survey of Presence and Related Concepts. ACM Computing Surveys, 50(6):96:1–96:39. DOI: https://doi.org/0.1145/3134301.

Slater, M. (2003). A note on presence terminology. Presence connect, 3(3):1–5.

Slater, M. (2018). Immersion and the illusion of presence in virtual reality. British Journal of Psychology (London, England: 1953), 109(3):431–433. DOI: https://doi.org/0.1111/bjop.12305.

Swan, J. and Gabbard, J. L. (2005). Survey of User-Based Experimentation in Augmented Reality. In Proceedings 1st International Conference on Virtual Reality, pages 1–9, Las Vegas, Nevada, USA. Mira Digital Publishing.

Toet, A., Mioch, T., Gunkel, S. N., Niamut, O., and van Erp, J. B. (2021). Assessment of presence in augmented and mixed reality. preprint, PsyArXiv.

Uzor, S. and Kristensson, P. O. (2021). An Exploration of Freehand Crossing Selection in Head-Mounted Augmented Reality. ACM Transactions on Computer-Human Interaction, 28(5):33:1–33:27. DOI: https://doi.org/0.1145/3462546.

Vaquero-Melchor, D. and Bernardos, A. M. (2019). Enhancing Interaction with Augmented Reality through Mid-Air Haptic Feedback: Architecture Design and User Feedback. Applied Sciences, 9(23):5123. DOI: https://doi.org/0.3390/app9235123.

Venkatakrishnan, R., Venkatakrishnan, R., Raveendranath, B., Pagano, C. C., Robb, A. C., Lin, W.-C., and Babu, S. V. (2023). Give Me a Hand: Improving the Effectiveness of Near-field Augmented Reality Interactions By Avatarizing Users’ End Effectors. IEEE Transactions on Visualization and Computer Graphics, 29(5):2412–2422. DOI: https://doi.org/0.1109/TVCG.2023.3247105.

Venkatesh, Thong, and Xu (2012). Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Quarterly, 36(1):157. DOI: https://doi.org/0.2307/41410412.

Veraszto, E. V., Silva, D. d., Miranda, N. A., and Simon, F. O. (2009). Tecnologia: buscando uma definição para o conceito. PRISMA.COM, 0(8):19–46. Number: 8.

Wang, G., Ren, G., Hong, X., Peng, X., Li, W., and O’Neill, E. (2022). Freehand Gestural Selection with Haptic Feedback in Wearable Optical See-Through Augmented Reality. Information, 13(12):566. DOI: https://doi.org/0.3390/info13120566.

Wang, T., Qian, X., He, F., Hu, X., Cao, Y., and Ramani, K. (2021a). GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications. In The 34th Annual ACM Symposium on User Interface Software and Technology, UIST ’21, pages 552–567, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3472749.3474769.

Wang, T., Qian, X., He, F., and Ramani, K. (2021b). LightPaintAR: Assist Light Painting Photography with Augmented Reality. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA ’21, pages 1–6, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/0.1145/3411763.3451672.

Weerasinghe, M., Biener, V., Grubert, J., Quigley, A. J., Toniolo, A., Pucihar, K. C., and Kljun, M. (2022). VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations. arXiv:2207.00896 [cs].

Weichel, C., Lau, M., Kim, D., Villar, N., and Gellersen, H. W. (2014). MixFab: a mixed-reality environment for personal fabrication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 3855–3864, Toronto Ontario Canada. ACM. DOI: https://doi.org/0.1145/2556288.2557090.

Whitlock, M., Harnner, E., Brubaker, J. R., Kane, S., and Szafir, D. A. (2018). Interacting with Distant Objects in Augmented Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 41–48, Reutlingen. IEEE. DOI: https://doi.org/0.1109/VR.2018.8446381.

Whitlock, M., Mitchell, J., Pfeufer, N., Arnot, B., Craig, R., Wilson, B., Chung, B., and Szafir, D. A. (2020). MRCAT: In Situ Prototyping of Interactive AR Environments. In Chen, J. Y. C. and Fragomeni, G., editors, Virtual, Augmented and Mixed Reality. Design and Interaction, volume 12190, pages 235–255. Springer International Publishing, Cham. DOI: https://doi.org/0.1007/978-3-030-49695-1_16.

Wieringa, R., Maiden, N., Mead, N., and Rolland, C. (2006). Requirements engineering paper classification and evaluation criteria: a proposal and a discussion. Requirements Engineering, 11(1):102–107. DOI: https://doi.org/0.1007/s00766-005-0021-6.

Williams, A. S., Garcia, J., and Ortega, F. (2020). Understanding Multimodal User Gesture and Speech Behavior for Object Manipulation in Augmented Reality Using Elicitation. IEEE Transactions on Visualization and Computer Graphics, 26(12):3479–3489. DOI: https://doi.org/0.1109/TVCG.2020.3023566.

Woolf, N. (2015). Google Glass ceases production ’in present form’.

Wright, T., de Ribaupierre, S., and Eagleson, R. (2019). Leap Motion Performance in an Augmented Reality Workspace: Integrating Devices with an Interactive Platform. IEEE Consumer Electronics Magazine, 8(1):36–41. DOI: https://doi.org/0.1109/MCE.2018.2816302.

Xu, W., Liang, H.-N., Chen, Y., Li, X., and Yu, K. (2020). Exploring Visual Techniques for Boundary Awareness During Interaction in Augmented Re ality Head-Mounted Displays. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 204–211, Atlanta, GA, USA. IEEE. DOI: https://doi.org/0.1109/VR46266.2020.00039.

Xu, W., Liang, H.-N., He, A., and Wang, Z. (2019). Pointing and Selection Methods for Text Entry in Augmented Reality Head Mounted Displays. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 279–288, Beijing, China. IEEE. DOI: https://doi.org/0.1109/ISMAR.2019.00026.

Yim, M. Y.-C., Chu, S.-C., and Sauer, P. L. (2017). Is Augmented Reality Technology an Effective Tool for E-commerce? An Interactivity and Vividness Perspective. Journal of Interactive Marketing, 39:89–103. DOI: https://doi.org/0.1016/j.intmar.2017.04.001.

Yu, K., Eck, U., Pankratz, F., Lazarovici, M., Wilhelm, D., and Navab, N. (2022). Duplicated Reality for Co-located Augmented Reality Collaboration. IEEE Transactions on Visualization and Computer Graphics, 28(5):2190–2200. DOI: https://doi.org/0.1109/TVCG.2022.3150520.

Zhao, C., Li, K. W., and Peng, L. (2023). Movement Time for Pointing Tasks in Real and Augmented Reality Environments. Applied Sciences, 13(2):788. DOI: https://doi.org/0.3390/app13020788.

Zhu, F. and Grossman, T. (2020). BISHARE: Exploring Bidirectional Interactions Between Smartphones and Head-Mounted Augmented Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–14, Honolulu HI USA. ACM. DOI: https://doi.org/0.1145/3313831.3376233.

tom Dieck, M. C., Jung, T. H., and Rauschnabel, P. A. (2018). Determining visitor engagement through augmented reality at science festivals: An experience economy perspective. Computers in Human Behavior, 82:44–53. DOI: https://doi.org/0.1016/j.chb.2017.12.043.

van der Heijden (2004). User Acceptance of Hedonic Information Systems. MIS Quarterly, 28(4):695. DOI: https://doi.org/0.2307/25148660.