User perception as a factor for improving Trustworthiness in e-commerce systems

Andréia Rodrigues Casare; Celmar Guimarães da Silva; Regina Moraes

doi:10.5753/jis.2024.3748

Authors

Andréia Rodrigues Casare Universidade Estadual de Campinas https://orcid.org/0000-0002-8009-4929
Celmar Guimarães da Silva Universidade Estadual de Campinas https://orcid.org/0000-0001-6112-892X
Regina Moraes Universidade Estadual de Campinas https://orcid.org/0000-0003-0678-4777

DOI:

https://doi.org/10.5753/jis.2024.3748

Keywords:

Trust, Trustworthiness, User Experience, Quality Model, Usability, Accessibility

Abstract

The User Interface (UI) is the first artefact that the user interacts with while developing a sense of trust that motivates him to use software applications more effectively. A badly designed UI can deceive users and bring the system into disrepute. Trustworthiness in UI is mandatory, as a poorly implemented UI can lead to the user misusing the system and jeopardizing the expected result. Trust in computational systems involves not only technical aspects, such as computational infrastructure, storage space, and service composition, but also aspects of Human-Computer Interaction (HCI). While technological aspects have received considerable attention, there are few research on human-computer interaction in terms of trust. This paper describes a way for assessing a system’s trustworthiness based on the user’s perception. The approach relies on a quality model to aggregate interface quality criteria in order to get a trustworthiness score. Three sets of experiments involving more than 300 individuals were carried out to validate the suggested methodology. A comparison was made between the trustworthiness score obtained through the methodology and the answers to open questions obtained through the users’ questionnaires. The results were consistent, and statistical analysis corroborated the positive assessment. Based on these results, examples of improvements were developed to highlight the usefulness of the approach for developing more trustworthiness interfaces.

Downloads

Download data is not yet available.

References

Al-Azzawi, G., Miskon, S., Abdullah, N., and Mat Ali, N. (2021). Factors influencing customers’ trust in ecommerce during covid-19 pandemic. In 7th International Conference on Research and Innovation in Information Systems-ICRIIS. DOI: https://doi.org/10.1109/ICRIIS53035.2021.9617021.

Albert, B. (2012). How quick are we to judge? a case study of trust and web site design. [link], Accessed: 16 January 2024.

Aljazzaf, Z. M., Perry, M., and Capretz, M. A. (2010). Online trust: Definition and principles. In 2010 Fifth International Multi-conference on Computing in the Global Information Technology, pages 163–168. IEEE. DOI: https://doi.org/10.1109/ICCGI.2010.17.

Ang, L. and Lee, B.-C. (2000). Influencing perceptions of trustworthiness in internet commerce: A rational choice framework. In Fifth CollECTer Conference on Electronic Commerce, pages 1–12. CollECTeR.

Attar, R., Shanmugam, M., and Hajli, N. (2020). Investigating the antecedents of e-commerce satisfaction in social commerce context. British Food Journal, 123:849–868. DOI: https://doi.org/10.1108/BFJ-08-2020-0755.

Bangor, A., Kortum, P., and Miller, J. (2009). Determining what individual sus scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3):114–123.

Basso, A., Goldberg, D., Greenspan, S., and Weimer, D. (2001). First impressions: Emotional and cognitive factors underlying judgments of trust e-commerce. In Proceedings of the 3rd ACM Conference on Electronic Commerce, pages 137–143. DOI: https://doi.org/10.1145/501158.501173.

Bauer, P. C. (2019). Conceptualizing trust and trustworthiness. Technical report, Revised version of working paper published in: Political Concepts Working Paper Series.

Bonisoli, L. and Castillo Leyva, K. (2022). Regulatory beliefs and trust: Adaptation of the technological acceptance model to e-commerce during the covid-19 pandemic in ecuador. Innovar, 32:135–149. DOI: https://doi.org/10.15446/innovar.v32n86.104666.

Brooke, J. (1996). Sus: a “quick and dirty’usability. Usability evaluation in industry, 189(3):189–194.

Casare, A., Basso, T., and Moraes, R. (2020). Trust metrics to measure website user experience. In The 13th Int. Conf. on Advances in Computer-Human Interactions, pages 1–8.

Casare, A., da Silva, C. G., Basso, T., and Moraes, R. (2021). Towards usability interface trustworthiness in e-commerce systems. In 15th Int. Conf. on Interfaces and Human Computer Interaction, pages 1–8.

Casare, A., Silva, C. G., and Moraes, R. (2022a). Do dependable systems need good user interfaces? In Proceedings of the 11th Latin-American Symposium on Dependable Computing, pages 21–28. DOI: https://doi.org/10.1145/3569902.3569905.

Casare, A. R., Basso, T., da Silva, C. G., and Moraes, R. (2022b). Using a quality model to evaluate user interface trustworthiness of e-commerce systems: Scoring strategies and preliminary results. In VISIGRAPP (2: HUCAPP), pages 209–216. DOI: https://doi.org/10.5220/0010889700003124.

Chen, J. and Dibb, S. (2010). Consumer trust in the online retail context: Exploring the antecedents and consequences. Psychology & Marketing, 27(4):323–346. DOI: https://doi.org/10.1002/mar.20334.

Cho, J.-H., Chan, K., and Adali, S. (2015). A survey on trust modeling. ACM Computing Surveys (CSUR), 48(2):1–40. DOI: https://doi.org/10.1145/2815595.

Cho, J.-H., Xu, S., Hurley, P. M., Mackay, M., Benjamin, T., and Beaumont, M. (2019). Stram: Measuring the trustworthiness of computer-based systems. ACM Computing Surveys (CSUR), 51(6):1–47. DOI: https://doi.org/10.1145/3277666.

Dujmovic, J. J. (2007). Continuous preference logic for system evaluation. IEEE Transactions on fuzzy systems, 15(6):1082–1099. DOI: https://doi.org/10.1109/TFUZZ.2007.902041.

Filho, G. K., Guerino, G., and Valentim, N. (2022). A systematic mapping study on usability and user experience evaluation of multi-touch systems. In Anais do XXI Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais. SBC. DOI: https://doi.org/10.1145/3554364.3559131.

Finstad, K. (2010). The usability metric for user experience. Interacting with computers, 22(5):323–327. DOI: https://doi.org/10.1016/j.intcom.2010.04.004.

Friginal, J., Martínez, M., De Andrés, D., and Ruiz, J.-C. (2016). Multi-criteria analysis of measures in benchmarking: Dependability benchmarking as a case study. Journal of Systems and Software, 111:105–118. DOI: https://doi.org/10.1016/j.jss.2015.08.052.

Gao, X., Ma, Y., and Zhou, W. (2021). The trustworthiness measurement model of component-based software based on the subjective and objective weight allocation method. In 2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C), pages 478–486. IEEE. DOI: https://doi.org/10.1109/QRS-C55045.2021.00076.

Gol Mohammadi, N., Paulus, S., Bishr, M., Metzger, A., Könnecke, H., Hartenstein, S., Weyer, T., and Pohl, K. (2014). Trustworthiness attributes and metrics for engineering trusted internet-based software systems. In Cloud Computing and Services Science: Third International Conference, CLOSER 2013, Aachen, Germany, May 8-10, 2013, Revised Selected Papers 3, pages 19–35. Springer. DOI: https://doi.org/10.1007/978-3-319-11561-0_2.

Granollers, T. (2018). Usability evaluation with heuristics. new proposal from integrating two trusted sources. In Design, User Experience, and Usability: Theory and Practice: 7th International Conference, DUXU 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part I 7, pages 396–405. Springer. DOI: https://doi.org/10.1007/978-3-319-91797-9_28.

Guerino, G. and Valentim, N. (2020). Usability and user experience evaluation of conversational systems: A systematic mapping study. In Anais do XXXIV Simpósio Brasileiro de Engenharia de Software. SBC. DOI: https://doi.org/10.1145/3422392.3422421.

Habib, S., Hamadneh, N. N., and Hassan, A. (2022). The relationship between digital marketing, customer engagement, and purchase intention via ott platforms. Journal of Mathematics, Special Issue - Analysis of Financial Problems Based on Mathematical Models, 2022. DOI: https://doi.org/10.1155/2022/5327626.

Hendradjaya, B. and Praptini, R. (2015). A proposal for a quality model for e-govemment website. In 2015 International Conference on Data and Software Engineering (ICoDSE), pages 19–24. IEEE. DOI: https://doi.org/10.1109/ICODSE.2015.7436965.

Henry, S., participants of the Education, and (EOWG), O. W. G. (2023). Introduction to web accessibility. [link], Accessed: 16 January 2024.

Hussain, F. K. and Chang, E. (2007). An overview of the interpretations of trust and reputation. In The Third Advanced International Conference on Telecommunications (AICT’07), pages 30–30. IEEE. DOI: https://doi.org/10.1109/AICT.2007.11.

Hussin, A. R. C., Macaulay, L., and Keeling, K. (2007). The importance ranking of trust attributes in e-commerce website. PACIS 2007 Proceedings, page 99.

ISO (2014). International organization for standardization. when the world agrees (ISO/IEC).

ISO (2016a). Systems and software engineering - systems and software quality requirements and evaluation (square) - measurement of quality in use (ISO/IEC).

ISO (2016b). Systems and software engineering - systems and software quality requirements and evaluation (square) - measurement of system and software product quality (ISO/IEC).

ISO (2019). Ergonomics of human system interaction — part 210: Human-centred design for interactive systems.

ITA Publishing (2024). Impact of covid pandemic on ecommerce. [link], 16 January 2024.

Jiménez, D. L., Dittmar, E. C., and Portillo, J. P. V. (2021). The use of trust seals in european and latin american commercial transactions. Journal of Open Innovation: Technology, Market, and Complexity, 7(2):150. DOI: https://doi.org/10.3390/joitmc7020150.

John Joseph, A. J. and Mariappan, M. (2018). A novel trust-scoring system using trustability coefficient of variation for identification of secure agent platforms. PloS one, 13(8):e0201600. DOI: https://doi.org/10.1371/journal.pone.0201600.

Lew, P., Olsina, L., and Zhang, L. (2010). Integrating quality, quality in use, actual usability and user experience. In 2010 6th Central and Eastern European Software Engineering Conference (CEE-SECR), pages 117–123. IEEE. DOI: https://doi.org/10.1109/CEE-SECR.2010.5783161.

Lewis, J. R. (1995). Ibm computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1):57–78. DOI: https://doi.org/10.1080/10447319509526110.

Lewis, J. R., Utesch, B. S., and Maher, D. E. (2013). Umux-lite: when there’s no time for the sus. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 2099–2102. DOI: https://doi.org/10.1145/2470654.2481287.

Lin, H. X., Choong, Y.-Y., and Salvendy, G. (1997). A proposed index of usability: a method for comparing the relative usability of different software systems. Behaviour & information technology, 16(4-5):267–277. DOI: https://doi.org/10.1080/014492997119833.

Lund, A. M. (2001). Measuring usability with the use questionnaire12. Usability interface, 8(2):3–6.

Malhotra, R. (2016). Empirical research in software engineering: concepts, analysis, and applications. CRC press.

Mayer, R. C., Davis, J. H., and Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of management review, 20(3):709–734. DOI: https://doi.org/10.5465/amr.1995.9508080335.

McKibbin, W. and Fernando, R. (2021). The global macroeconomic impacts of covid-19: Seven scenarios. Asian Economic Papers, 20(2):1–30. DOI: https://doi.org/10.1162/asep_a_00796.

McKnight, D. H., Choudhury, V., and Kacmar, C. (2002). Developing and validating trust measures for e-commerce: An integrative typology. Information systems research, 13(3):334–359. DOI: https://doi.org/10.1287/isre.13.3.334.81.

Mohammadi, N. G., Paulus, S., Bishr, M., Metzger, A., Koennecke, H., Hartenstein, S., and Pohl, K. (2013). An analysis of software quality attributes and their contribution to trustworthiness. In CLOSER, pages 542–552. DOI: https://doi.org/10.5220/0004502705420552.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and Group, P. (2009). Reprint—preferred reporting items for systematic reviews and meta-analyses: the prisma statement. Physical therapy, 89(9):873–880.

Mouratidis, H. and Cofta, P. (2010). Practitioner’s challenges in designing trust into online systems. Journal of theoretical and applied electronic commerce research, 5(3):65–77. DOI: https://doi.org/10.4067/S0718-18762010000300007.

Nielsen, J. (2024). Usability 101: Introduction to usability. [link], Accessed: 16 January 2024.

Nietzio, A., Strobbe, C., and Velleman, E. (2008). The unified web evaluation methodology (uwem) 1.2 for wcag 1.0. In International Conference on Computers for Handicapped Persons, pages 394–401. Springer. DOI: https://doi.org/10.1007/978-3-540-70540-6_57.

Norman, D. and Nielsen, J. (2024). The definition of user experience (ux). [link], Accessed: 16 January 2024.

OECD (2017). OECD Guidelines on Measuring Trust. OECD. [link], Accessed: 16 January 2024.. DOI: https://doi.org/10.1787/9789264278219-en.

Olsina, L., Papa, F., and Molina, H. (2008). Ontological support for a measurement and evaluation framework. International Journal of Intelligent Systems, 23(12):1282–1300. DOI: https://doi.org/10.1002/int.20320.

Parlakkiliã, A., Ãzmez, M., and Mertoälu, S. (2020). Digital transformation of e-commerce: How did covid-19 affect customers’ online shopping behaviors? Journal of Business in The Digital Age, 3:117–122. DOI: https://doi.org/10.46238/jobda.823955.

Parmanto, B. and Zeng, X. (2005). Metric for web accessibility evaluation. Journal of the American Society for Information Science and Technology, 56(13):1394–1404. DOI: https://doi.org/10.1002/asi.20233.

Patterson, P. G., Cowley, E., and Prasongsukarn, K. (2006). Service failure recovery: The moderating impact of individual-level cultural value orientation on perceptions of justice. International Journal of Research in Marketing, 23(3):263–277. DOI: https://doi.org/10.1016/j.ijresmar.2006.02.004.

Perlman, G. (1997). Practical heuristics for usability evaluation. In CHI ’97 Extended Abstracts on Human Factors in Computing Systems.

Ramadhan, B. A. and Iqbal, B. M. (2018). User experience evaluation on the cryptocurrency website by trust aspect. In 2018 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), volume 3, pages 274–279. IEEE. DOI: https://doi.org/10.1109/ICI-IBMS.2018.8550019.

Sauro, J. and Kindlund, E. (2005). A method to standardize usability metrics into a single score. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 401–409. DOI: https://doi.org/10.1145/1054972.1055028.

Sauro, Jeff (2024). A practical guide to the system usability scale. [link], 17 January 2024.

Seffah, A., Donyaee, M., Kline, R. B., and Padda, H. K. (2006). Usability measurement and metrics: A consolidated model. Software quality journal, 14:159–178. DOI: https://doi.org/10.1007/s11219-006-7600-8.

Shankar, V., Urban, G. L., and Sultan, F. (2002). Online trust: a stakeholder perspective, concepts, implications, and future directions. The Journal of strategic information systems, 11(3-4):325–344. DOI: https://doi.org/10.1016/S0963-8687(02)00022-7.

Sharma, G. and Lijuan, W. (2015). The effects of online service quality of e-commerce websites on user satisfaction. The electronic library, 33(3):468–485. DOI: https://doi.org/10.1108/EL-10-2013-0193.

Sommerville, I. (2011). Engenharia de software. Pearson Prentice Hall.

Song, S., Bu, J., Shen, C., Artmeier, A., Yu, Z., and Zhou, Q. (2018). Reliability aware web accessibility experience metric. In Proceedings of the 15th International Web for All Conference, pages 1–4. DOI: https://doi.org/10.1145/3192714.3192836.

Song, S., Wang, C., Li, L., Yu, Z., Lin, X., and Bu, J. (2017). Waem: a web accessibility evaluation metric based on partial user experience order. In Proceedings of the 14th International Web for All Conference, pages 1–4. DOI: https://doi.org/10.1145/3058555.3058576.

Tao, H., Chen, Y., and Pang, J. (2015). A software trustworthiness measure based on the decompositions of trustworthy attributes and its validation. In Industrial Engineering, Management Science and Applications 2015, pages 981–990. Springer. DOI: https://doi.org/10.1007/978-3-662-47200-2_102.

Tao, H. and Zhao, J. (2018). Source codes oriented software trustworthiness measure based on validation. Mathematical Problems in Engineering, 2018:1–10. DOI: https://doi.org/10.1155/2018/6982821.

Tsuda, N., Washizaki, H., Honda, K., Nakai, H., Fukazawa, Y., Azuma, M., Komiyama, T., Nakano, T., Suzuki, H., Morita, S., et al. (2019). Wsqf: Comprehensive software quality evaluation framework and benchmark based on square. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 312–321. IEEE. DOI: https://doi.org/10.1109/ICSE-SEIP.2019.00045.

Veral, R. and Macías, J. A. (2019). Supporting user-perceived usability benchmarking through a developed quantitative metric. International Journal of Human-Computer Studies, 122:184–195. DOI: https://doi.org/10.1016/j.ijhcs.2018.09.012.

Vigo, M., Arrue, M., Brajnik, G., Lomuscio, R., and Abascal, J. (2007). Quantitative metrics for measuring web accessibility. In Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A), pages 99–107. DOI: https://doi.org/10.1145/1243441.1243465.

Wang, Y. D. and Emurian, H. H. (2005). An overview of online trust: Concepts, elements, and implications. Computers in human behavior, 21(1):105–125. DOI: https://doi.org/10.1016/j.chb.2003.11.008.