A modeling approach to introduce haptic feedback in multimodal awareness for groupware systems

Andrés Rodríguez; Luis Mariano Bibbo; Cesar Collazos; Alejandro Fernandez

doi:10.5753/jis.2023.3219

Authors

Andrés Rodríguez Centro LIFIA, CIC - FI Universidad Nacional de La Plata, Argentina https://orcid.org/0000-0002-7600-6041
Luis Mariano Bibbo Centro LIFIA, CIC - FI Universidad Nacional de La Plata, Argentina https://orcid.org/0000-0003-4950-3605
Cesar Collazos Universidad del Cauca, Popayán, Colombia https://orcid.org/0000-0002-7099-8131
Alejandro Fernandez Centro LIFIA, CIC - FI Universidad Nacional de La Plata, Argentina https://orcid.org/0000-0002-7968-6871

DOI:

https://doi.org/10.5753/jis.2023.3219

Keywords:

haptic interfaces, awareness, groupware, model driven development

Abstract

The key to effective collaboration is awareness; traditional groupware systems have relied on visual cues to achieve it. Audio has been used to complement and overcome the limitations of graphical displays to maintain awareness. However, the emergence of alternative user interface strategies, such as haptic feedback, presents new opportunities to promote awareness. Yet, designing effective haptic awareness mechanisms is challenging due to the lack of documented experience. Therefore, we suggest using flexible prototyping and evaluation tools to facilitate an exploratory design process. We also propose that the design of awareness features should consider modalities to be used and the relationships between them from the outset. To support this idea, we introduce a modeling language and supporting tools to express haptic awareness features. We show how the language can be used in the context of model-driven development of groupware.

Downloads

Download data is not yet available.

References

Abowd, G. D. and Mynatt, E. D. (2000). Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput. Hum. Interact., 7(1):29–58.

Bailenson, J. N., Yee, N., Brave, S., Merget, D., and Koslow, D. (2007). Virtual interpersonal touch: expressing and recognizing emotions through haptic devices. Human–Computer Interaction, 22(3):325–353. Publisher: Taylor & Francis.

Basdogan, C., Giraud, F., Levesque, V., and Choi, S. (2020). A review of surface haptics: Enabling tactile effects on touch surfaces. IEEE transactions on haptics, 13(3):450–470.

Bibbo, L. M. (2022). Modelado de Sistemas Colaborativos. PhD thesis, Universidad Nacional de La Plata, La Plata.

Bibbo, L. M. (2023). Cssl: Collaborative software system language. [link].

Bibbo, L. M., Pons, C., and Giandini, R. (2022). Model-Driven Development of Groupware Systems:. International Journal of e-Collaboration, 18(1):1–28.

Bolt, R. A. (1980). “put-that-there” voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques, pages 262–270.

Brambilla, M., Cabot, J., and Wimmer, M. (2012). Model-Driven Software Engineering in Practice. Synthesis Lectures on Software Engineering, 1(1):1–182.

Brewster, S. and Brown, L. M. (2004). Tactons: Structured tactile messages for non-visual information display. In Proceedings of the Fifth Conference on Australasian User Interface - Volume 28, AUIC ’04, page 15–23, AUS. Australian Computer Society, Inc.

Brown, L. M., Brewster, S. A., and Purchase, H. C. (2005). A first investigation into the effectiveness of tactons. In First joint eurohaptics conference and symposium on haptic interfaces for virtual environment and teleoperator systems. world haptics conference, pages 167–176. IEEE.

Canché, M. and Ochoa, S. F. (2018). Modeling computer-mediated user interactions in ubiquitous collaborative systems. In Processing International Conference on Ubiquitous Computing and Ambient Intelligence, Punta Cana, Dominican Republic.

Chan, A., MacLean, K., and McGrenere, J. (2008). Designing haptic icons to support collaborative turn-taking. International Journal of Human-Computer Studies, 66(5):333–355.

Chen, M., Katagami, D., and Nitta, K. (2010). An experimental investigation of haptic interaction in online negotiation. In 2010 World Automation Congress, pages 1–6. IEEE.

Collazos, C. A., Gutiérrez, F. L., Gallardo, J., Ortega, M., Fardoun, H. M., and Molina, A. I. (2019). Descriptive theory of awareness for groupware development. Journal of Ambient Intelligence and Humanized Computing, 10(12):4789–4818.

Dourish, P. and Bly, S. (1992). Portholes: Supporting awareness in a distributed work group. dl.acm.org.

Edwards, J., Liu, H., Zhou, T., Gould, S. J. J., Clark, L., Doyle, P., and Cowan, B. R. (2019). Multitasking with alexa: How using intelligent personal assistants impacts language-based primary task performance. In Proceedings of the 1st International Conference on Conversational User Interfaces, CUI ’19, New York, NY, USA. Association for Computing Machinery.

Eid, M. A. and Al Osman, H. (2015). Affective haptics: Current research and future directions. IEEE Access, 4:26–40. Publisher: IEEE.

Enriquez, M. J. and MacLean, K. E. (2003). The hapticon editor: a tool in support of haptic communication research. In 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings., pages 356–362. IEEE.

Frey, J., Grabli, M., Slyper, R., and Cauchard, J. R. (2018). Breeze: Sharing biofeedback through wearable technologies. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pages 1–12. Association for Computing Machinery.

Gaffary, Y., Martin, J.-C., and Ammi, M. (2014). Haptic expressions of stress during an interactive game. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pages 266–274. Springer.

Gallardo, J., Molina, A. I., Bravo, C., Redondo, M. A., and Collazos, C. A. (2011). An ontological conceptualization approach for awareness in domain-independent collaborative modeling systems: Application to a model-driven development method. In Expert Systems with Applications.

Gaver, W. W. (1991). Sound support for collaboration. In Bannon, L., Robinson, M., and Schmidt, K., editors, Proceedings of the Second European Conference on Computer-Supported Cooperative Work ECSCW ’91, pages 293–308. Springer Netherlands.

Gaver, W. W., Smith, R. B., and O’Shea, T. (1991). Effective sounds in complex systems: the ARKOLA simulation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’91, pages 85–90. Association for Computing Machinery.

Geldard, F. A. (1957). Adventures in tactile literacy. American Psychologist, 12(3):115.

Grudin, J. (1988). Why cscw applications fail: problems in the design and evaluation of organizational interfaces. In Proceedings of the 1988 ACM conference on Computer-supported cooperative work, pages 85–93.

Gutwin, C. and Greenberg, S. (1998). Effects of awareness support on groupware usability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’98, page 511–518, USA. ACM Press/Addison-Wesley Publishing Co.

Gutwin, C. and Greenberg, S. (2002). A descriptive framework of workspace awareness for real-time groupware. Computer Supported Cooperative Work.

Gutwin, C., Greenberg, S., and Roseman, M. (1996). Workspace awareness in real-time distributed groupware: Framework, widgets, and evaluation. In Proceedings of HCI ’96.

Gutwin, C., Schneider, O., Xiao, R., and Brewster, S. (2011). Chalk sounds: the effects of dynamic synthesized audio on workspace awareness in distributed groupware. In Proceedings of the ACM 2011 conference on Computer supported cooperative work, CSCW ’11, pages 85–94. Association for Computing Machinery.

Haans, A., de Nood, C., and IJsselsteijn, W. A. (2007). Investigating response similarities between real and mediated social touch: a first test. In CHI’07 Extended Abstracts on Human Factors in Computing Systems, pages 2405–2410.

Haans, A. and IJsselsteijn, W. (2006). Mediated social touch: a review of current research and future directions. Virtual Reality, 9:149–159.

Hoggan, E. and Brewster, S. (2007). New parameters for tacton design. In CHI ’07 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’07, page 2417–2422, New York, NY, USA. Association for Computing Machinery.

Huang, K., Starner, T., Do, E., Weinberg, G., Kohlsdorf, D., Ahlrichs, C., and Leibrandt, R. (2010). Mobile music touch: mobile tactile stimulation for passive learning. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 791–800.

Huisman, G., Frederiks, A. D., Van Dijk, B., Hevlen, D., and Kröse, B. (2013). The tasst: Tactile sleeve for social touch. In 2013 World Haptics Conference (WHC), pages 211–216. IEEE.

IJsselsteijn, W. (2003). Presence in the past: what can we learn from media history? In Being there: concepts, effects and measurements of user presence in synthetic environments, pages 17–40. Ios Press.

Jang, C. Y., Steinfield, C., and Pfaff, B. (2002). Virtual team awareness and groupware support: An evaluation of the teamscope system. International Journal of Human Computer Studies, 56.

Joule, R.-V. and Guéguen, N. (2007). Touch, compliance, and awareness of tactile contact. Perceptual and motor skills, 104(2):581–588.

Ju, Y., Zheng, D., Hynds, D., Chernyshov, G., Kunze, K., and Minamizawa, K. (2021). Haptic empathy: Conveying emotional meaning through vibrotactile feedback. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery.

Kamoun, A., Tazi, S., and Drira, K. (2012). FADYRCOS, a semantic interoperability framework for collaborative model-based dynamic reconfiguration of networked services. Computers in Industry, 63(8):756–765.

Kuchenbecker, K. J., Romano, J., and McMahan, W. (2011). Haptography: Capturing and recreating the rich feel of real surfaces. In Robotics Research, pages 245–260. Springer.

Lahtinen, R. M. (2008). Haptices and haptemes: A case study of developmental process in social-haptic communication of acquired deafblind people. A1 Management UK.

Ledo, D., Houben, S., Vermeulen, J., Marquardt, N., Oehlberg, L., and Greenberg, S. (2018). Evaluation strategies for hci toolkit research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pages 1–17.

Linden, D. J. (2016). Touch: The science of the hand, heart, and mind. Penguin Books.

MacLean, K. E., Schneider, O. S., and Seifi, H. (2017). Multisensory haptic interactions: understanding the sense and designing for it. In The Handbook of Multimodal-Multisensor Interfaces: Foundations, User Modeling, and Common Modality Combinations-Volume 1, pages 97–142. ACM.

Maggioni, E., Agostinelli, E., and Obrist, M. (2017). Measuring the added value of haptic feedback. In 2017 ninth international conference on quality of multimedia experience (QoMEX), pages 1–6. IEEE.

Molina, A. I., Gallardo, J., Navarro, C. X., and Redondo, M. (2016). A model-driven approach for the development of cscl tools that considers pedagogical usability. In Proceedings of the 1st international workshop on collaborative modelling in MDE (COMMitMDE 2016), pages 45–53.

Molina, A. I., Giraldo, W. J., Ortega, M., Redondo, M. A., and Collazos, C. A. (2014). Model-driven development of interactive groupware systems: Integration into the software development process. Science of Computer Programming, 89:320–349.

Oviatt, S. (2017). Theoretical Foundations of Multimodal Interfaces and Systems, page 19–50. Association for Computing Machinery and Morgan & Claypool.

Oviatt, S., Schuller, B., Cohen, P., Sonntag, D., and Potamianos, G. (2017). The handbook of multimodal-multisensor interfaces, volume 1: Foundations, user modeling, and common modality combinations. Morgan & Claypool.

Parisi, D. (2018). Archaeologies of touch: Interfacing with haptics from electricity to computing. U of Minnesota Press.

Pielot, M., Henze, N., and Boll, S. (2009). Supporting map-based wayfinding with tactile cues. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, pages 1–10.

Poyraz, G. G. and Tamer, O. (2019). Different haptic senses with multiple vibration motors. In 2019 11th International Conference on Electrical and Electronics Engineering (ELECO), pages 870–874.

Salisbury, K., Conti, F., and Barbagli, F. (2004). Haptics rendering: Introductory concepts. IEEE computer graphics and applications, 24(ARTICLE):24–32.

Sarter, N. B. (2006). Multimodal information presentation: Design guidance and research challenges. International journal of industrial ergonomics, 36(5):439–445.

Schneider, O., MacLean, K., Swindells, C., and Booth, K. (2017). Haptic experience design: What hapticians do and where they need help. International Journal of Human-Computer Studies, 107:5–21.

Seifi, H., Zhang, K., and MacLean, K. E. (2015). Vibviz: Organizing, visualizing and navigating vibration libraries. In 2015 IEEE World Haptics Conference (WHC), pages 254–259.

Smith, J. and MacLean, K. (2007). Communicating emotion through a haptic link: Design space and methodology. International Journal of Human-Computer Studies, 65(4):376–387.

Sohlenkamp, M. (1999). Supporting group awareness in multi user environments through perceptualization. PhD thesis, University of Paderborn, Germany.

Stahl, T., Völter, M., Bettin, J., Haase, A., and Helsen, S. (2006). Model-driven software development - technology, engineering, management. Pitman.

Stivers, T. and Sidnell, J. (2005). Introduction: Multimodal interaction. Semiotica, 2005(156):1–20.

Ternes, D. and MacLean, K. E. (2008). Designing large sets of haptic icons with rhythm. In Haptics: Perception, Devices and Scenarios: 6th International Conference, EuroHaptics 2008 Madrid, Spain, June 10-13, 2008 Proceedings 6, pages 199–208. Springer.

Teruel, M. A., Navarro, E., López-Jaquero, V., Montero, F., Jaen, J., and González, P. (2012). Analyzing the understandability of Requirements Engineering languages for CSCW systems: A family of experiments. Information and Software Technology.

Triyason, T., Tassanaviboon, A., and Kanthamanon, P. (2020). Hybrid classroom: Designing for the new normal after covid-19 pandemic. In Proceedings of the 11th International Conference on Advances in Information Technology, pages 1–8.

Wong, S., Singhal, S., and Neustaedter, C. (2017). Smartcrew: A smart watch design for collaboration amongst flight attendants. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’17 Companion, pages 41–44. Association for Computing Machinery. event-place: Portland, Oregon, USA.

Yamamura, F., Tanichi, T., Pai, Y. S., and Minamizawa, K. (2021). Comado: Communication system for ambient connection between distance locations. In 2021 IEEE World Haptics Conference (WHC), pages 588–588.

Yoo, Y., Yoo, T., Kong, J., and Choi, S. (2015). Emotional responses of tactile icons: Effects of amplitude, frequency, duration, and envelope. In 2015 IEEE World Haptics Conference (WHC), pages 235–240. IEEE.

Zhang, S., Wang, D., Afzal, N., Zhang, Y., and Wu, R. (2016). Rhythmic haptic stimuli improve short-term attention. IEEE transactions on haptics, 9(3):437–442.

Zhang, Z., Alvina, J., Héron, R., Safin, S., Detienne, F., and Lecolinet, E. (2021). Touch without touching: Overcoming social distancing in semi-intimate relationships with sanstouch. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–13.