A User Evaluation of a Collaborator Recommender based on Co-Changed Files
DOI:
https://doi.org/10.5753/jis.2024.3728Keywords:
Open-Source Software Projects, Collaborative Software Development, Distributed Collaboration, Developer RecommendationAbstract
Active collaboration is essential for the success of software projects across the development life-cycle. Unfortunately, in social coding platforms, such as GitHub, it is still challenging for developers to identify potential collaborators with whom they could engage to create new/stronger ties and enhance the quality of contributions. To this end, we implemented developer recommendation strategies and prototype tool to help project contributors improve their collaborations. Thus, in this work, we described a controlled experimental study concerned usability and user satisfaction to investigate the developers’ perceptions of using CoopFinder, a prototype tool to support two strategies for recommending collaborations. These developer recommendation strategies aim to connect developers of a specific project based on their similar interests. The study involved 35 participants, 18 of which were GitHub users, and 17 were non–GitHub users. We asked participants to perform the experiment tasks to find collaborators with similar interests using a prototype recommendation tool and GitHub. We reported a quantitative and qualitative evaluation of strategies and tool using the state of the practice as a baseline. As a result, we observed that recommender based on co–changed files can provide suitable collaborator recommendations to developers of a specific project. About 66% of the participants confirmed they would use or recommend this tool.
Downloads
References
Avelino, G., Passos, L., Hora, A., and Valente, M. T. (2016). A novel approach for estimating truck factors. In Proc. of the 24th International Conference on Program Comprehension (ICPC), pages 1–10.
Barcomb, A., Stol, K.-J., Fitzgerald, B., and Riehle, D. (2020). Managing episodic volunteers in free/libre/open source software communities. IEEE Transactions on Software Engineering (TSE), 48(1):260–277. DOI: 10.1109/TSE.2020.2985093.
Barcomb, A., Stol, K.-J., Riehle, D., and Fitzgerald, B. (2019). Why do episodic volunteers stay in floss communities? In Proc. of the 41st International Conference on Software Engineering (ICSE), pages 948–959.
Basili, V. R., Shull, F., and Lanubile, F. (1999). Building knowledge through families of experiments. IEEE Transactions on Software Engineering (TSE), 25(4):456–473.
Basili, V. R. and Weiss, D. M. (1984). A methodology for collecting valid software engineering data. IEEE Transactions on Software Engineering (TSE), (6):728–738.
Bird, C. (2011). Sociotechnical coordination and collaboration in open source software. In Proc. of the 27th International Conference on Software Maintenance (ICSM), pages 568–573.
Blincoe, K., Sheoran, J., Goggins, S., Petakovic, E., and Damian, D. (2016). Understanding the popular users: Following, affiliation influence and leadership on github. Information and Software Technology (IST), 70:30–39.
Canfora, G., Di Penta, M., Oliveto, R., and Panichella, S. (2012). Who is going to mentor newcomers in open source projects? In Proc. of the 20th International Symposium on the Foundations of Software Engineering (FSE), pages 1–11.
Constantino, K., Belém, F., and Figueiredo, E. (2023a). Dual analysis for helping developers to find collaborators based on co-changed files: An empirical study. Journal of Software: Practice and Experience (JSPE), pages 1–27. DOI: https://doi.org/10.1002/spe.3194.
Constantino, K. and Figueiredo, E. (2022). Coopfinder: Finding collaborators based on co–Changed files. In Proc. of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pages 1–3.
Constantino, K. and Figueiredo, E. (2023). Finding collaborations based on co-changed files. In Anais Estendidos do XVIII Simpósio Brasileiro de Sistemas Colaborativos, pages 57–66. DOI: 10.5753/sbscestendido.2023.229735.
Constantino, K., Prates, R., and Figueiredo, E. (2023b). Recommending collaborators based on co–changed files: A controlled experiment. In Proc. of the 18th Brazilian Symposium on Collaborative Systems, pages 154–168.
Constantino, K., Souza, M., Zhou, S., Figueiredo, E., and Kästner, C. (2021). Perceptions of open-source software developers on collaborations: An interview and survey study. Journal of Software: Evolution and Process (JSEP), 33:e2393.
Constantino, K., Zhou, S., Souza, M., Figueiredo, E., and Kästner, C. (2020). Understanding collaborative software development: An interview study. In Proc. of the 15th International Conference on Global Software Engineering (ICGSE), page 55–65. DOI: 10.1145/3372787.3390442.
Corbin, J. and Strauss, A. (2014). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory.
Costa, C., Figueirêdo, J., Pimentel, J. F., Sarma, A., and Murta, L. (2021). Recommending participants for collaborative merge sessions. IEEE Transactions on Software Engineering (TSE), 47(6):1198–1210. DOI: 10.1109/TSE.2019.2917191.
Crowston, K. and Fagnot, I. (2018). Stages of motivation for contributing user-generated content: A theory and empirical test. International Journal of Human-Computer Studies (IJHCS), 109:89–101.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly (MISQ), pages 319–340.
de Neira, A. B., Steinmacher, I., and Wiese, I. S. (2018). Characterizing the hyperspecialists in the context of crowdsourcing software development. Journal of the Brazilian Computer Society (JBCS), 24(1):1–16.
Ferreira, M., Valente, M. T., and Ferreira, K. (2017). A comparison of three algorithms for computing truck factors. In Proc. of the 25th International Conference on Program Comprehension (ICPC), pages 207–217.
Fisher, R. A. (1992). The arrangement of field experiments. In Breakthroughs in Statistics, pages 82–91.
Flick, U. (2018). Designing Qualitative Research. Qualitative Research Kit.
Franco, M. F., Rodrigues, B., and Stiller, B. (2019). Mentor: The design and evaluation of a protection services recommender system. In Proc. of the 15th International Conference on Network and Service Management (CNSM), pages 1–7. DOI: 10.23919/CNSM46954.2019.9012686.
Gamalielsson, J. and Lundell, B. (2014). Sustainability of open source software communities beyond a fork: How and why has the libreoffice project evolved? Journal of Systems and Software (JSS), 89:128–145.
Gousios, G., Pinzger, M., and Deursen, A. v. (2014). An exploratory study of the pull-based software development model. In Proc. of the 36th International Conference on Software Engineering (ICSE), pages 345–355.
Gousios, G., Storey, M.-A., and Bacchelli, A. (2016). Work practices and challenges in pull-based development: The contributor’s perspective. In Proc. of the 38th International Conference on Software Engineering (ICSE), pages 285–296.
Gousios, G., Zaidman, A., Storey, M.-A., and Deursen, A. v. (2015). Work practices and challenges in pull-based development: The integrator’s perspective. In Proc. of the 37th International Conference on Software Engineering (ICSE), volume 1, pages 358–368. DOI: 10.1109/ICSE.2015.55.
Jiang, J., He, J.-H., and Chen, X.-Y. (2015). Coredevrec: Automatic core member recommendation for contribution evaluation. Journal of Computer Science and Technology (JCST), 30(5):998–1016.
Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., El Emam, K., and Rosenberg, J. (2002). Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering (TSE), 28(8):721–734.
Kononenko, O., Baysal, O., and Godfrey, M. W. (2016). Code review quality: How developers see it. In Proc. of the 38th International Conference on Software Engineering (ICSE), pages 1028–1038.
Miller, R. and Siegmund, D. (1982). Maximally selected chi square statistics. Biometrics, pages 1011–1016.
Minto, S. and Murphy, G. (2007). Recommending emergent teams. In Proc. of the 4th International Conference on Mining Software Repositories (MSR), pages 5–5. DOI: 10.1109/MSR.2007.27.
Oliveira, J., Pinheiro, D., and Figueiredo, E. (2020). Jexpert: A tool for library expert identification. In Proc. of the 34th Brazilian Symposium on Software Engineering (SBES), pages 386–392.
Oliveira, J., Viggiato, M., and Figueiredo, E. (2019). How well do you know this library? mining experts from source code analysis. In Proc. of the XVIII Brazilian Symposium on Software Quality (SBQS), pages 49–58.
Pham, R., Singer, L., Liskin, O., Figueira Filho, F., and Schneider, K. (2013). Creating a shared understanding of testing culture on a social coding site. In Proc. of the 35th International Conference on Software Engineering (ICSE), pages 112–121.
Pinto, G., Steinmacher, I., and Gerosa, M. (2016). More common than you think: An in-depth study of casual contributors. In Proc. of the 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), volume 1, pages 112–123.
Qiu, H. S., Nolte, A., Brown, A., Serebrenik, A., and Vasilescu, B. (2019). Going farther together: The impact of social capital on sustained participation in open source. In Proc. of the 41st International Conference on Software Engineering (ICSE), pages 688–699.
Rahman, M. M., Roy, C. K., Redl, J., and Collins, J. A. (2016). Correct: Code reviewer recommendation at github for vendasta technologies. In Proc. of the 31st International Conference on Automated Software Engineering (ASE), page 792–797.
Ricci, F., Rokach, L., and Shapira, B. (2011). Introduction to recommender systems handbook. In Recommender Systems Handbook, pages 1–35.
Salton, G. (1971). The smart retrieval system: Experiments in automatic information retrieval.
Salton, G. (1989). Automatic text processing: The transformation, analysis, and retrieval of. Reading: Addison-Wesley, 169.
Salton, G. and Harman, D. (2003). Information retrieval. In Encyclopedia of Computer Science.
Shah, S. K. (2006). Motivation, governance, and the viability of hybrid forms in open source software development. Management Science, 52(7):1000–1014.
Steinmacher, I., Pinto, G., Wiese, I. S., and Gerosa, M. A. (2018). Almost there: A study on quasi-contributors in open-source software projects. In Proc. of the 40th International Conference on Software Engineering (ICSE), pages 256–266.
Steinmacher, I., Silva, M. A. G., Gerosa, M. A., and Redmiles, D. F. (2015). A systematic literature review on the barriers faced by newcomers to open source software projects. Information and Software Technology (IST), 59:67–85.
Surian, D., Liu, N., Lo, D., Tong, H., Lim, E.-P., and Faloutsos, C. (2011). Recommending people in developers’ collaboration network. In Proc. of the 18th Working Conference on Reverse Engineering (WCRE), pages 379–388.
Tamburri, D. A., Kruchten, P., Lago, P., and Van Vliet, H. (2015). Social debt in software engineering: Insights from industry. Journal of Internet Services and Applications (JISA), 6(1):1–17.
Thongtanunam, P., Tantithamthavorn, C., Kula, R. G., Yoshida, N., Iida, H., and Matsumoto, K.-i. (2015). Who should review my code? a file location-based code-reviewer recommendation approach for modern code review. In Proc. of the 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), pages 141–150.
Vasilescu, B., Filkov, V., and Serebrenik, A. (2015a). Perceptions of diversity on github: A user survey. In Proc. of the 8th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), pages 50–56.
Vasilescu, B., Posnett, D., Ray, B., van den Brand, M. G., Serebrenik, A., Devanbu, P., and Filkov, V. (2015b). Gender and tenure diversity in github teams. In Proc. of the 33rd International Conference on Human Factors in Computing Systems (CHI), pages 3789–3798.
Wilcoxon, F. (1992). Individual comparisons by ranking methods. In Breakthroughs in Statistics, pages 196–202.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., and Regnell, B. (2012). Experimentation in Software Engineering.
Wu, Y., Kropczynski, J., Shih, P. C., and Carroll, J. M. (2014). Exploring the ecosystem of software developers on github and other platforms. In Proc. of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW), pages 265–268.
Yu, Y., Wang, H., Filkov, V., Devanbu, P., and Vasilescu, B. (2015). Wait for it: Determinants of pull request evaluation latency on github. In Proc. of the 12th International Conference on Mining Software Repositories (MSR), pages 367–371.
Zhou, M. and Mockus, A. (2011). Does the initial environment impact the future of developers? In Proc. of the 33rd International Conference on Software Engineering (ICSE), pages 271–280.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Kattiana Constantino, Raquel Prates, Eduardo Figueiredo
This work is licensed under a Creative Commons Attribution 4.0 International License.
JIS is free of charge for authors and readers, and all papers published by JIS follow the Creative Commons Attribution 4.0 International (CC BY 4.0) license.
