MamuLEDs: Mixed Reality meets Mamulengo
DOI:
https://doi.org/10.5753/jis.2020.771Keywords:
Mixed Reality, Hand Tracking, Mamulengo, MamuLED, PuppetryAbstract
Techniques of Virtual, Augmented, and Mixed Reality (VR, AR, MR) have been used for puppet theater in different cultural contexts around the world. However, we are not yet aware of the use of these techniques in the Brazilian Northeast Popular Puppet Theater, a tradition known in Pernambuco as Mamulengo. We present here a system developed for “The Quarrel Between the Real Puppet and the Virtual Puppet,” a sketch created for the class Realidade Virtual e Aumentada of Computer Science program at CIn-UFPE. This paper aims to investigate how VR/AR/MR techniques can contribute to the creation of mamulengo shows. We used the method of qualitative case study, analyzing the audiovisual records of essays and presentation, the software solution developed in Unity 3D, and the data collected from a focus group interview with puppeteers. This study identified as weaknesses of our system: the hand tracking unreliability and the lack of tactile feedback, causing discomfort in experienced puppeteering professionals. As strengthens of the system, we found that the technology aroused interest because of its potential for expanding the options of tools available for popular puppetry shows creation in current times.
Downloads
Download data is not yet available.
References
Amaral, A. M. (1996). Teatro de Formas Animadas: Máscaras, Bonecos, Objetos (3rd ed.). São Paulo: Editora da Universidade de São Paulo.
Bar-Lev, A., Bruckstein, A. M., & Elber, G. (2005). Virtual marionettes: A system and paradigm for real-time 3D animation. Visual Computer, 21(7), 488–501.
Beltrame, V. (2003). Revelar e esconder: o trabalho do ator-animador. Revista Do Festival Nacional de Teatro Infantil de Blumenau, 16–22. Blumenau.
Borba Filho, H. (1966). Fisionomia e Espírito do Mamulengo. São Paulo:Companhia Editora Nacional.
Castro, K. E. de. (2015). O Teatro de Mamulengos de ontem e de hoje: a importância do reconhecimento do Teatro de Bonecos Tradicional Brasileiro como patrimônio imaterial cultural do Brasil. Resgate -Revista Interdisciplinar de Cultura, 23, 69–80.
Edwards, P. (2009). How to Rap: The Art and Science of the Hip-Hop MC. Chicago: Chicago Review Press.
Guna, J., Jakus, G., Pogacnik, M., Tomazic, S., & Sodnik, J. (2014). An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking. Sensors(Basel, Switzerland), 14, 3702–3720.
Gurgel, D. (1986). João redondo: Teatro de Bonecos do Nordeste. Natal: Vozes/UFRN.
Huang, A. P., Huang, F., & Jhu, J. S. (2018). Unreal Interactive Puppet Game Development Using Leap Motion. Journal of Physics: Conference Series, 1004(1).
Husinsky, M., & Bruckner, F. (2018). Virtual Stage: Interactive Puppeteering in Mixed Reality. IEEE 1st Workshop on Animation in Virtual and Augmented Environments, ANIVAE 2018, (March), 1–7.
Jácome, J., Oliveira, M., Alvim, F., Teichrieb, V., & Rama-lho, G. L. (2019). Realidade Mista, Mamulengos e MamuLEDs. Proceedings -2019 21st Symposium on Virtual and Augmented Reality, SVR 2019, 4, 252–259.
Jessel, J. P., Jaspart, C., & Flores, J. J. (2001). Computer animation and virtual reality for live art performance. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2197, 205–207.
Leite, L., & Orvalho, V. (2017). Mani-Pull-Action: Hand-based Digital Puppetry. Proc. ACM Hum.Comput. Inter-act., 1(EICS), 2:1--2:16.
Liang, H., Chang, J., Kazmi, I. K., Zhang, J. J., & Jiao, P. (2015). Puppet Narrator: Utilizing Motion Sensing Technology in Storytelling for Young Children. VS-Games 2015 -7th International Conference on Games and Virtual Worlds for Serious Applications.
Lin, C. Y., Yang, Z. H.,Zhou, H. W., Yang, T. N., Chen, H. N., & Shih, T. K. (2019). Combining leap motion with unity for virtual glove puppets. Proceedings -2018 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2018, 251–255.
Luo, Z., Chen, I.M., Yeo, S. H., Lin, C. C., & Li, T. Y. (2010). Building hand motion-based character animation: The case of puppetry. Proceedings -2010 International Conference on Cyberworlds, CW 2010, 46–52.
Macêdo, Z. R. de, & Assunção, L. (2015). Teatro de Mamulengos: tradição e modernidade na cultura popular. Anais Do V REA e XIV REABANE : Direitos Diferenciados, Conflitos e Produção de Conhecimentos. Maceió -AL.
Mazalek, A., Nitsche, M., Rebola, C., Wu, A., Clifton, P., Peer, F., & Drake, M. (2011). Pictures at an Exhibition: A Physical/Digital Puppetry Performance Piece. Proceedings of the 8th ACM Conference on Creativity and Cognition, 441–442.
Meschke, M. (1974). Algumas reflexões impopulares relativas à moral titiriteira na UNIMA -União Internacional da Marioneta. Revista Mamulengo, 10.
Oshita, M., Senju, Y., & Morishige, S. (2013). Character motion control interface with hand manipulation inspired by puppet mechanism. Proceedings of the 12th ACM SIG-GRAPH International Conference on Virtual-Reality Con-tinuum andIts Applications in Industry. Association for Computing Machinery. 131–138.
Perez, N. M. P., & Yamada, T. R. U. (2015). Atividades artísticas na terceira idade: um relato de experiência com pa-pel machê. VIII World Congress on Communication and Arts -Anais, 326–329. Salvador, Bahia.
Sautchuk, J. M. (2011). Poetic improvisation in the Brazilian Northeast. Vibrant: Virtual Brazilian Anthropology, 8(1), 260–290.
Slater, C. (1982). Stories on a String: The Brazilian Literatura de Cordel. Berkeley: University of California Press.
Speicher, M., Hall, B. D., & Nebeling, M. (2019). What is Mixed Reality? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems -CHI ’19, 1–15.
Stake, R. E. (2005). Qualitative Case Studies. In The Sage handbook of qualitative research, 3rd ed. (pp. 443–466). Thousand Oaks, CA: Sage Publications Ltd.
Velho, L., & Lupiac, B. (2020). Expanded Virtual Puppeteer-ing. Proceedings of the 15th International Joint Confer-ence on Computer Vision, Imaging and Computer GraphicsTheory and Applications -Volume 1: GRAPP, 61–70.
Weichert, F., Bachmann, D., Rudak, B., & Fisseler, D. (2013). Analysis of the accuracy and robustness of the Leap Motion Controller. Sensors(Switzerland), 13(5), 6380–6393.
Bar-Lev, A., Bruckstein, A. M., & Elber, G. (2005). Virtual marionettes: A system and paradigm for real-time 3D animation. Visual Computer, 21(7), 488–501.
Beltrame, V. (2003). Revelar e esconder: o trabalho do ator-animador. Revista Do Festival Nacional de Teatro Infantil de Blumenau, 16–22. Blumenau.
Borba Filho, H. (1966). Fisionomia e Espírito do Mamulengo. São Paulo:Companhia Editora Nacional.
Castro, K. E. de. (2015). O Teatro de Mamulengos de ontem e de hoje: a importância do reconhecimento do Teatro de Bonecos Tradicional Brasileiro como patrimônio imaterial cultural do Brasil. Resgate -Revista Interdisciplinar de Cultura, 23, 69–80.
Edwards, P. (2009). How to Rap: The Art and Science of the Hip-Hop MC. Chicago: Chicago Review Press.
Guna, J., Jakus, G., Pogacnik, M., Tomazic, S., & Sodnik, J. (2014). An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking. Sensors(Basel, Switzerland), 14, 3702–3720.
Gurgel, D. (1986). João redondo: Teatro de Bonecos do Nordeste. Natal: Vozes/UFRN.
Huang, A. P., Huang, F., & Jhu, J. S. (2018). Unreal Interactive Puppet Game Development Using Leap Motion. Journal of Physics: Conference Series, 1004(1).
Husinsky, M., & Bruckner, F. (2018). Virtual Stage: Interactive Puppeteering in Mixed Reality. IEEE 1st Workshop on Animation in Virtual and Augmented Environments, ANIVAE 2018, (March), 1–7.
Jácome, J., Oliveira, M., Alvim, F., Teichrieb, V., & Rama-lho, G. L. (2019). Realidade Mista, Mamulengos e MamuLEDs. Proceedings -2019 21st Symposium on Virtual and Augmented Reality, SVR 2019, 4, 252–259.
Jessel, J. P., Jaspart, C., & Flores, J. J. (2001). Computer animation and virtual reality for live art performance. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2197, 205–207.
Leite, L., & Orvalho, V. (2017). Mani-Pull-Action: Hand-based Digital Puppetry. Proc. ACM Hum.Comput. Inter-act., 1(EICS), 2:1--2:16.
Liang, H., Chang, J., Kazmi, I. K., Zhang, J. J., & Jiao, P. (2015). Puppet Narrator: Utilizing Motion Sensing Technology in Storytelling for Young Children. VS-Games 2015 -7th International Conference on Games and Virtual Worlds for Serious Applications.
Lin, C. Y., Yang, Z. H.,Zhou, H. W., Yang, T. N., Chen, H. N., & Shih, T. K. (2019). Combining leap motion with unity for virtual glove puppets. Proceedings -2018 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2018, 251–255.
Luo, Z., Chen, I.M., Yeo, S. H., Lin, C. C., & Li, T. Y. (2010). Building hand motion-based character animation: The case of puppetry. Proceedings -2010 International Conference on Cyberworlds, CW 2010, 46–52.
Macêdo, Z. R. de, & Assunção, L. (2015). Teatro de Mamulengos: tradição e modernidade na cultura popular. Anais Do V REA e XIV REABANE : Direitos Diferenciados, Conflitos e Produção de Conhecimentos. Maceió -AL.
Mazalek, A., Nitsche, M., Rebola, C., Wu, A., Clifton, P., Peer, F., & Drake, M. (2011). Pictures at an Exhibition: A Physical/Digital Puppetry Performance Piece. Proceedings of the 8th ACM Conference on Creativity and Cognition, 441–442.
Meschke, M. (1974). Algumas reflexões impopulares relativas à moral titiriteira na UNIMA -União Internacional da Marioneta. Revista Mamulengo, 10.
Oshita, M., Senju, Y., & Morishige, S. (2013). Character motion control interface with hand manipulation inspired by puppet mechanism. Proceedings of the 12th ACM SIG-GRAPH International Conference on Virtual-Reality Con-tinuum andIts Applications in Industry. Association for Computing Machinery. 131–138.
Perez, N. M. P., & Yamada, T. R. U. (2015). Atividades artísticas na terceira idade: um relato de experiência com pa-pel machê. VIII World Congress on Communication and Arts -Anais, 326–329. Salvador, Bahia.
Sautchuk, J. M. (2011). Poetic improvisation in the Brazilian Northeast. Vibrant: Virtual Brazilian Anthropology, 8(1), 260–290.
Slater, C. (1982). Stories on a String: The Brazilian Literatura de Cordel. Berkeley: University of California Press.
Speicher, M., Hall, B. D., & Nebeling, M. (2019). What is Mixed Reality? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems -CHI ’19, 1–15.
Stake, R. E. (2005). Qualitative Case Studies. In The Sage handbook of qualitative research, 3rd ed. (pp. 443–466). Thousand Oaks, CA: Sage Publications Ltd.
Velho, L., & Lupiac, B. (2020). Expanded Virtual Puppeteer-ing. Proceedings of the 15th International Joint Confer-ence on Computer Vision, Imaging and Computer GraphicsTheory and Applications -Volume 1: GRAPP, 61–70.
Weichert, F., Bachmann, D., Rudak, B., & Fisseler, D. (2013). Analysis of the accuracy and robustness of the Leap Motion Controller. Sensors(Switzerland), 13(5), 6380–6393.
Downloads
Published
2020-10-08
How to Cite
JÁCOME, J.; OLIVEIRA, M.; ALVIM, F.; TEICHRIEB, V.; RAMALHO, G. MamuLEDs: Mixed Reality meets Mamulengo. Journal on Interactive Systems, Porto Alegre, RS, v. 11, n. 1, p. 12–26, 2020. DOI: 10.5753/jis.2020.771. Disponível em: https://journals-sol.sbc.org.br/index.php/jis/article/view/771. Acesso em: 14 nov. 2024.
Issue
Section
Regular Paper
License
JIS is free of charge for authors and readers, and all papers published by JIS follow the Creative Commons Attribution 4.0 International (CC BY 4.0) license.
