Virtual Reality System for Inspection and Training: A case study in Wheel-Loader Operations

Luiz Felipe Muniz Rocha Borges; Arthur Spinassé Miranda; João Vitor Ladislau Ferreira; Thales Pinheiro Lazarini; Alexandre De Angeli Neto; Matheus Gianordoli Novais; Marcelo Queiroz Schimidt; Rodrigo Varejão Andreão; Mário Mestria

doi:10.5753/jis.2025.5171

Authors

Luiz Felipe Muniz Rocha Borges Federal Institute of Espírito Santo https://orcid.org/0009-0007-2158-9255
Arthur Spinassé Miranda Federal Institute of Espírito Santo https://orcid.org/0009-0001-4278-1445
João Vitor Ladislau Ferreira Federal Institute of Espírito Santo https://orcid.org/0009-0006-4293-0419
Thales Pinheiro Lazarini Federal Institute of Espírito Santo https://orcid.org/0009-0003-1867-9154
Alexandre De Angeli Neto Federal Institute of Espírito Santo https://orcid.org/0009-0000-2038-8370
Matheus Gianordoli Novais Federal Institute of Espírito Santo https://orcid.org/0009-0006-6119-8454
Marcelo Queiroz Schimidt Federal Institute of Espírito Santo https://orcid.org/0000-0003-1307-9873
Rodrigo Varejão Andreão Federal Institute of Espírito Santo https://orcid.org/0000-0002-6800-5700
Mário Mestria Federal Institute of Espírito Santo https://orcid.org/0000-0001-8283-0806

DOI:

https://doi.org/10.5753/jis.2025.5171

Keywords:

Virtual reality, Serious game, Training, Inspection virtual environment, User interface game, Industry 4.0

Abstract

A virtual-reality-based serious game to inspect and train a wheel loader is proposed in this paper. The game is developed using a cross-platform game engine and it is integrated into various virtual reality (VR) headsets. The proposed game was conducted under inspection and training procedures on the wheel loader of a mining company with elements to engage field workers. The results reached demonstrated the benefits of the use of such implemented technology. In addition, 16 workers assessed the game and completed a survey. The survey results suggest that the system can be used for staff training before handling a real wheel loader in a mining company. The findings suggest that the implementation of a game for inspection and training improves the process, reduces expenses, and improves safety. The game favors the development of technical skills required for real-world procedures. The VR serious game proposed as a training tool prepares company staff for inspection and operating on a wheel loader. The serious game can be expanded to include various technical procedures, such as a maintenance procedure for a tamping machine.

Downloads

Download data is not yet available.

References

Akpan, I. J. and Offodile, O. F. (2024). The role of virtual reality simulation in manufacturing in industry 4.0. Systems, 12(1). DOI: https://doi.org/10.3390/systems12010026.

Alaloul, W. S., Saad, S., and Qureshi, A. H. (2022). Construction Sector: IR 4.0 Applications, pages 1341–1390. Springer International Publishing, Cham. DOI: https://doi.org/10.1007/978-3-030-84205-5_36.

Ali, S. G., Wang, X., Li, P., Jung, Y., Bi, L., Kim, J., Chen, Y., Feng, D. D., Magnenat Thalmann, N., Wang, J., and Sheng, B. (2023). A systematic review: Virtual-reality-based techniques for human exercises and health improvement. Frontiers in Public Health, 11. DOI: https://doi.org/10.3389/fpubh.2023.1143947.

Borges, L. F. M. R., Viana, P. H. P., de Oliveira, T. a. R., Martins, T. d. S., Andreão, R. V. a., Schimidt, M., and Mestria, M. (2024). Evaluating virtual reality simulations for wheel loader inspection. In Proceedings of the 25th Symposium on Virtual and Augmented Reality, SVR ’23, pages 8–16, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3625008.3625010.

Brooke, J. (1996). SUS: A quick and dirty usability scale. In Jordan, P. W., Thomas, B., Weerdmeester, B. A., and McClelland, I. L. (Eds.), Usability Evaluation in Industry, pages 189–194. Taylor and Francis, London.

Commission, E. (2018). Re-finding industry – Defining innovation. European Commission and Directorate-General for Research and Innovation - Publications Office. DOI: https://data.europa.eu/doi/10.2777/927953.

Corso, A. D., Stets, J. D., Luongo, A., Nielsen, J. B., Frisvad, J. R., and Aanæs, H. (2017). Virtual reality inspection and painting with measured BRDFs. In SIGGRAPH Asia 2017 VR Showcase, SA ’17, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3139468.3139472.

Creutzfeldt, J., Hedman, L., and Felländer-Tsai, L. (2012). Effects of pre-training using serious game technology on CPR performance—an exploratory quasi-experimental transfer study. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 20(79):79. DOI: https://doi.org/10.1186/1757-7241-20-79.

Dadhich, S. (2018). Automation of Wheel-Loaders. PhD thesis, Lulea University of Technology, Embedded Internet Systems Lab.

Eschen, H., Kötter, T., Rodeck, R., Harnisch, M., and Schüppstuhl, T. (2018). Augmented and virtual reality for inspection and maintenance processes in the aviation industry. Procedia Manufacturing, 19:156–163. DOI: https://doi.org/10.1016/j.promfg.2018.01.022.

Guo, Z., Zhou, D., Zhou, Q., Zhang, X., Geng, J., Zeng, S., Lv, C., and Hao, A. (2020). Applications of virtual reality in maintenance during the industrial product lifecycle: A systematic review. Journal of Manufacturing Systems, 56:525–538. DOI: https://doi.org/10.1016/j.jmsy.2020.07.007.

Hart, S. G. (1986). NASA task load index (TLX). In NASA Ames Research Center Moffett Field, CA United States, pages 1–26.

Hess, T. and Gunter, G. (2011). Comparison of learning experiences and outcomes between a serious game-based and non-game-based online American history course. In Barton, S.-M., Hedberg, J., and Suzuki, K. (Eds.), Proceedings of Global Learn 2011, pages 1223–1228, Melbourne, Australia. Association for the Advancement of Computing in Education (AACE).

Javaid, M., Haleem, A., Singh, R. P., Suman, R., and Gonzalez, E. S. (2022). Understanding the adoption of Industry 4.0 technologies in improving environmental sustainability. Sustainable Operations and Computers, 3:203–217. DOI: https://doi.org/10.1016/j.susoc.2022.01.008.

Joshi, S., Hamilton, M., Warren, R., Faucett, D., Tian, W., Wang, Y., and Ma, J. (2021). Implementing virtual reality technology for safety training in the precast/prestressed concrete industry. Applied Ergonomics, 90:103286. DOI: https://doi.org/10.1016/j.apergo.2020.103286.

Korkut, E. H. and Surer, E. (2023). Visualization in virtual reality: A systematic review. Virtual Reality, 27:1447–1480. DOI: https://doi.org/10.1007/s10055-023-00753-8.

Liang, Z., Zhou, K., and Gao, K. (2019). Development of virtual reality serious game for underground rock-related hazards safety training. IEEE Access, 7:118639–118649. DOI: https://doi.org/10.1109/ACCESS.2019.2934990.

Liebrecht, C., Kandler, M., Lang, M., Schaumann, S., Stricker, N., Wuest, T., and Lanza, G. (2021). Decision support for the implementation of Industry 4.0 methods: Toolbox, assessment and implementation sequences. Journal of Manufacturing Systems, 58:412–430. DOI: https://doi.org/10.1016/j.jmsy.2020.12.008.

Luz Melo, R., Moreira, V. d. S., Amaral, E. M. H. d., and Domingues Júnior, J. S. (2025). Victus exergame: An approach to rehabilitation of amputees based in serious game. Journal on Interactive Systems, 16(1):137–147. DOI: https://doi.org/10.5753/jis.2025.4194.

Malpartida, K. F. C. and Rodrigues, K. R. d. H. (2025). Building serious games to exercise computational thinking: Initial evaluation with teachers of children on the autism spectrum. Journal on Interactive Systems, 16(1):148–162. DOI: https://doi.org/10.5753/jis.2025.4492.

Mitgutsch, K. and Alvarado, N. (2012). Purposeful by design? A serious game design assessment framework. In Proceedings of the International Conference on the Foundations of Digital Games, FDG ’12, pages 121–128, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/2282338.2282364.

Nguyen, A., Wüest, P., and Kunz, A. (2020). Human following behavior in virtual reality. In Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology, VRST ’20, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3385956.3422099.

Pilote, B. and Chiniara, G. (2019). Chapter 2 – The many faces of simulation. In Chiniara, G. (Ed.), Clinical Simulation (2nd ed.), pages 17–32. Academic Press. DOI: https://doi.org/10.1016/B978-0-12-815657-5.00002-4.

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147:103778. DOI: https://doi.org/10.1016/j.compedu.2019.103778.

Rahman, O. F., Kunze, K. N., Yao, K., Kwiecien, S. Y., Ranawat, A. S., Banffy, M. B., Kelly, B. T., and Galano, G. J. (2024). Hip arthroscopy simulator training with immersive virtual reality has similar effectiveness to nonimmersive virtual reality. Arthroscopy: The Journal of Arthroscopic & Related Surgery. DOI: https://doi.org/10.1016/j.arthro.2024.02.042.

Ribeiro de Oliveira, T., Biancardi Rodrigues, B., Moura da Silva, M., Spinassé, R. A. N., Ludke, G. G., Gaudio, M. R. S., Gomes, G. I. R., Cotini, L. G., Vargens, D. S., Schimidt, M. Q., Andreão, R. V., and Mestria, M. (2023). Virtual reality solutions employing artificial intelligence methods: A systematic literature review. ACM Computing Surveys, 55(10). DOI: https://doi.org/10.1145/3565020.

Ribeiro de Oliveira, T., Martinelli, T. F., Bello, B. P., Batista, J. D., Silva, M. M. d., Rodrigues, B. B., Spinassé, R. A. N., Andreão, R. V., Mestria, M., and Schimidt, M. Q. (2020). Virtual reality system for industrial motor maintenance training. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR), pages 119–128. DOI: https://doi.org/10.1109/SVR51698.2020.00031.

Romero-Gázquez, J. L., Cañavate-Cruzado, G., and Bueno-Delgado, M.-V. (2022). In4wood: A successful European training action of Industry 4.0 for academia and business. IEEE Transactions on Education, 65(2):200–209. DOI: https://doi.org/10.1109/TE.2021.3111696.

Rufino Júnior, R., Classe, T. M. d., Santos, R. P. d., and Siqueira, S. W. M. (2023). Current risk situation training in industry, and games as a strategy for playful, engaging and motivating training. Journal on Interactive Systems, 14(1):138–156. DOI: https://doi.org/10.5753/jis.2023.3222.

Sancho-Esper, F., Ostrovskaya, L., Rodriguez-Sanchez, C., and Campayo-Sanchez, F. (2023). Virtual reality in retirement communities: Technology acceptance and tourist destination recommendation. Journal of Vacation Marketing, 29(2):275–290. DOI: https://doi.org/10.1177/13567667221080567.

Schoneveld, E., Lichtwarck-Aschoff, A., and Granic, I. (2018). Preventing childhood anxiety disorders: Is an applied game as effective as a cognitive behavioral therapy-based program? Prevention Science, 19(2):220–232. DOI: https://doi.org/10.1007/s11121-017-0843-8.

Shao-Chen Chang, Hsu, T.-C., Chen, Y.-N., and Jong, M. S. Y. (2020). The effects of spherical video-based virtual reality implementation on students’ natural science learning effectiveness. Interactive Learning Environments, 28(7):915–929. DOI: https://doi.org/10.1080/10494820.2018.1548490.

Sinnott, C., Liu, J., Matera, C., Halow, S., Jones, A., Moroz, M., Mulligan, J., Crognale, M., Folmer, E., and MacNeilage, P. (2019). Underwater virtual reality system for neutral buoyancy training: Development and evaluation. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, VRST ’19, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3359996.3364272.

Soori, M., Arezoo, B., and Dastres, R. (2024). Virtual manufacturing in Industry 4.0: A review. Data Science and Management, 7(1):47–63. DOI: https://doi.org/10.1016/j.dsm.2023.10.006.

Tsai, W.-L., Chung, M.-F., Pan, T.-Y., and Hu, M.-C. (2017). Train in virtual court: Basketball tactic training via virtual reality. In Proceedings of the 2017 ACM Workshop on Multimedia-Based Educational and Knowledge Technologies for Personalized and Social Online Training, MultiEdTech ’17, pages 3–10, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3132390.3132394.

Wang, X. and Wang, X. (2018). Virtual reality training system for surgical anatomy. In Proceedings of the 2018 International Conference on Artificial Intelligence and Virtual Reality, AIVR 2018, pages 30–34, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3293663.3293670.

Zhu, W., Fan, X., and Zhang, Y. (2019). Applications and research trends of digital human models in the manufacturing industry. Virtual Reality & Intelligent Hardware, 1(6):558–579. DOI: https://doi.org/10.1016/j.vrih.2019.09.005.

Štaffenová, N. and Kucharčíková, A. (2024). Human capital management – values, competencies, and motivation – concerning Industry 4.0. Economic Research-Ekonomska Istraživanja, 37(1):2324160. DOI: https://doi.org/10.1080/1331677X.2024.2324160.