Exploratory testing for platform video games: strategies and lessons learned
DOI:
https://doi.org/10.5753/jis.2024.4156Keywords:
Game Testing, Software Testing, Playtesting, Platform Games, Manual TestingAbstract
With the growth of the digital games market, the occurrence of bugs in games has a significant impact and generates dissatisfaction among users. Therefore, conducting tests is necessary to avoid these events and ensure the quality of the distributed product. Among the tests that are performed in games, one that is particularly effective in identifying bugs from the user's perspective is exploratory testing, but this is little covered in game testing literature, not providing new testers with guides or paths to be used for specific occasions. This paper reports an experience of applying exploratory testing strategies in 2D and 3D platform games. We selected seven well-known strategies of exploratory testing and conducted a study that involved the definition of a game testing procedure and proper adaptations for the games under test. By applying the game testing procedure, several bugs with low, medium and high severity were uncovered. The lessons learned and routines carried out in this study can be used by new testers in games of the same category, in an attempt to obtain better results.
Downloads
References
Al-Azawi, R., Ayesh, A., and Obaidy, M. A. (2013). Generic evaluation framework for games development methodology. 2013 Third International Conference on Communications and Information Technology (ICCIT), pages 55–60.
Aleem, S., Capretz, L. F., and Ahmed, F. (2016). Critical success factors to improve the game development process from a developer’s perspective. Journal of Computer Science and Technology, 31:925–950.
Bach, J. (2003). Exploratory testing explained. [link].
Bankhurst, A. (2021). Cyberpunk 2077 teve o maior lançamento digital da história, mostra levantamento. [link]. Access on 02 July 2024.
Chueca, J., Verón, J., Font, J., Pérez, F., and Cetina, C. (2024). The consolidation of game software engineering: A systematic literature review of software engineering for industry-scale computer games. Information and Software Technology, 165:107330. DOI: https://doi.org/10.1016/j.infsof.2023.107330.
Clement, J. (2023). Digital video game market revenue worldwide from 2017 to 2027. [link]. Access on 02 July 2024.
Copche, R., Souza, M., Villanes, I. K., Durelli, V. H. S., Eler, M., Dias-Neto, A. C., and Endo, A. T. (2021). Exploratory testing of apps with opportunity maps. In SBQS ’21: XX Brazilian Symposium on Software Quality, Virtual Event, Brazil, November 8 - 11, 2021, page 6. ACM. DOI: https://doi.org/10.1145/3493244.3493248.
Demartini, F. (2021). Reembolsos de cyberpunk 2077 representaram 9% do faturamento da cd projekt red. [link]. Access on 02 July 2024.
Dias, G. (2021). Mario, pokémon: As franquias mais lucrativas da indústria. [link]. Access on 02 July 2024.
Duarte, Y., Durelli, V. H. S., Nardi, P. A., and Endo, A. T. (2023). Exploratory testing strategies for video games: an experience report. In Proceedings of the 22nd Brazilian Symposium on Games and Digital Entertainment, SBGames 2023, Rio Grande (RS), Brazil, November 6-9, 2023, pages 46–55. ACM. DOI: https://doi.org/10.1145/3631085.3631227.
Eletronic Arts Inc. (2021). It takes two media. [link]. Access on 02 July 2024.
Escada Games (2024). Diver Down by Escada Games. [link]. Access on 02 July 2024.
Godot (2024). Godot Engine - Free and open source 2D and 3D game engine. [link]. Access on 02 July 2024.
Harmony Honey (2024). Tiny Crate by Harmony Honey. [link]. Access on 02 July 2024.
Hendrickson, E. (2013). Explore it!: reduce risk and increase confidence with exploratory testing. Pragmatic Bookshelf.
Iftikhar, S., Iqbal, M. Z., Khan, M. U., and Mahmood, W. (2015). An automated model based testing approach for platform games. MODELS ’15, page 426–435. IEEE Press.
Itkonen, J. (2011). Empirical studies on exploratory software testing. PhD thesis. [link].
Juul, J. (2003). The game, the player, the world: looking for a heart of gameness. In Digital Games Research Conference 2003, 4-6 November 2003, University of Utrecht, The Netherlands.
Kaner, C., Falk, J. L., and Nguyen, H. Q. (1999). Testing Computer Software, Second Edition. John Wiley & Sons, Inc., 2nd edition.
Kasurinen, J. and Smolander, K. (2014). What do game developers test in their products? In 2014 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’14, Torino, Italy, September 18-19, 2014, pages 1:1–1:10. ACM. DOI: https://doi.org/10.1145/2652524.2652525.
Lovreto, G., Endo, A. T., Nardi, P., and Durelli, V. H. S. (2018). Automated tests for mobile games: An experience report. In 17th Brazilian Symposium on Computer Games and Digital Entertainment, SBGames 2018, Foz do Iguaçu, Brazil, October 29 - November 1, 2018, pages 48–56. IEEE Computer Society. DOI: https://doi.org/10.1109/SBGAMES.2018.00015.
Lyndsay, J. and Eeden, N. V. (2003). Adventures in session-based testing. Workroom Productions Ltd. May, 27.
Micallef, M., Porter, C., and Borg, A. (2016). Do exploratory testers need formal training? an investigation using hci techniques. In 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 305–314. DOI: https://doi.org/10.1109/ICSTW.2016.31.
Minkkinen, T. (2016). Basics of platform games. [link].
Murphy-Hill, E., Zimmermann, T., and Nagappan, N. (2014). Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development? In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, page 1–11, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/2568225.2568226.
Novak, J. (2017). Desenvolvimento de Games. Cengage Learning, 2nd edition.
OBS Project (2022). Wiki obs studio. [link]. Access on 02 July 2024.
Politowski, C., Petrillo, F., and Guéhéneuc, Y.-G. (2021a). A survey of video game testing. In 2021 IEEE/ACM International Conference on Automation of Software Test (AST), pages 90–99.
Politowski, C., Petrillo, F., Ullmann, G. C., and Gu’eh’eneuc, Y.-G. (2021b). Game industry problems: an extensive analysis on the gray literature. Inf. Softw. Technol., 134:106538.
Redavid, C. and Farid, A. (2011). An overview of game testing techniques. [link].
Schultz, C. P., Bryant, R., and Langdell, T. (2005). Game Testing All in One. Thomson/Course Technology.
Sriram, V. (2019). Automated playtesting of platformer games using reinforcement learning. Masters thesis. Northeastern University, Boston USA.
Valve Corporation (2023a). Portal 2 on steam. [link]. Access on 02 July 2024.
Valve Corporation (2023b). Portal on steam. [link]. Access on 02 July 2024.
Whittaker, J. A. (2009). Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design. Addison-Wesley Professional, 1st edition.
Winpress, M. (2024). Little Spy by Martin Winpress. [link]. Access on 02 July 2024.
Woods, A. (2020). Fall guys has sold over 11 million copies on pc and is now the most downloaded ps plus game. [link].
Xavier, B., Viana, D., and Santos, R. (2023). A dive into the state of the practice of the brazilian game software ecosystem. IEEE Transactions on Games, pages 1–10. DOI: https://doi.org/10.1109/TG.2023.3242217.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Yohan Duarte, Henrique Canella Mandelli, Vinicius Durelli, Paulo Augusto Nardi, André Takeshi Endo
This work is licensed under a Creative Commons Attribution 4.0 International License.
JIS is free of charge for authors and readers, and all papers published by JIS follow the Creative Commons Attribution 4.0 International (CC BY 4.0) license.
