Um Método para Determinar a Complexidade de Microtasks Executadas em Ambientes Crowdsourcing
DOI:
https://doi.org/10.5753/isys.2018.375Keywords:
desenvolvimento de software, Microtasks, crowdsourcing, Complexidade, microtasks, Desenvolvimento de Software Crowdsourcing, Complexidade de Software, CrowdsourcingAbstract
O desenvolvimento de software crowdsourcing (CS) emprega a terceirização das tarefas de design, desenvolvimento e teste, conhecidas como microtasks, para um grupo anônimo de participantes. O uso das microtaks em ambientes CS vem promovendo a diminuição de custos e redução de tempo de um projeto, o que remete sua importância. Apesar disso, a literatura ainda não concentra uma maneira eficaz sobre como determinar a sua dificuldade de execução. Assim, o objetivo deste trabalho é apresentar um método capaz de avaliar a complexidade que uma microtask possui. Para isso foi gerado um método de determinação de complexidade baseado nas próprias características de uma microtask. Tal método foi avaliado via estudo de caso em uma plataforma de desenvolvimento de software CS. Os resultados mostraram que o método foi eficiente alcançando um índice próximo de 90% de assertividade em classificar microtasks como simples ou complexas na amostra analisada. Além disso, o método também evidenciou que as microtasks mais simples alcançaram maior índice de registro e submissão em relação as microtasks complexas.
Downloads
Não há dados estatísticos.
Referências
Abhinav, K., Dubey, A., Jain, S., Bhatia, G. K., McCartin, B., and Bhardwaj, N. (2018). Crowdassistant: A virtual buddy for crowd worker. In 2018 IEEE/ACM 5th International Workshop on Crowd Sourcing in Software Engineering (CSI-SE), pages 17–20. doi: https://doi.org/10.1145/3195863.3195865
Aipe, A. and Gadiraju, U. (2018). Similarhits: Revealing the role of task similarity in microtask crowdsourcing. HT.
Chandra, G., Gupta, D. L., and Malviya, K. (2012). Some observations based on comparison of mood and ck software metrics suites for object oriented system. International Journal of Computer Science and Technology, 3(3).
Deus, W. S., Barros, R. M., and L’Erario, A. (2016). Um modelo para o gerenciamento do crowdsourcing em projetos de software. In I Workshop sobre Aspectos Sociais, Humanos e Economicos de Software (WASHES’2016).
Deus, W. S., Fabri, J. A., and L’Erario, A. (2017). The management of crowdsourcing software projects: A systematic mapping. In 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), pages 1–7. doi: https://doi.org/10.23919/CISTI.2017.7975711
de Freitas Junior, M., Fantinato, M., and Sun, V. (2015). Improvements to the function point analysis method: A systematic literature review. IEEE Transactions on Engineering Management, 62(4):495–506. doi: https://doi.org/10.1109/TEM.2015.2453354
Dubey, A., Abhinav, K., Taneja, S., Virdi, G., Dwarakanath, A., Kass, A., and Kuriakose, M. S. (2016). Dynamics of software development crowdsourcing. In 2016 IEEE 11th International Conference on Global Software Engineering (ICGSE), pages 49–58. doi: https://doi.org/10.1109/ICGSE.2016.13
Dwarakanath, A., Chintala, U., Shrikanth, N. C., Virdi, G., Kass, A., Chandran, A., Sengupta, S., and Paul, S. (2015). Crowd build: A methodology for enterprise software development using crowdsourcing. In 2015 IEEE/ACM 2nd International Workshop on CrowdSourcing in Software Engineering, pages 8–14. doi: https://doi.org/10.1109/CSI-SE.2015.9
Goke, N. and Freitag, E. (2014). Microtask platforms a win/win/win situation. In ¨ Colletive Inteligence. Greicius, T. (2018). Multi-planet system found through crowdsourcing. disponivel em: https://www.nasa.gov/feature/jpl/multi-planet-system-found-through-crowdsourcing. NASA.
Hosseini, M., Phalp, K., Taylor, J., and Ali, R. (2014). The four pillars of crowdsourcing: A reference model. In 2014 IEEE Eighth International Conference on Research Challenges in Information Science (RCIS), pages 1–12. doi: https://doi.org/10.1109/RCIS.2014.6861072
Howe, J. (2006a). Crowdsourcing: A definition. https://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html, Janeiro, 2019.
Howe, J. (2006b). The rise of crowdsourcing. Wired magazine, 14(6):1–4. https://www.wired.com/2006/06/crowds/, Janeiro, 2019.
Jacques, J. T. (2018). Microtask Design: Value, Engagement, Context, and Complexity. PhD thesis, University of Cambridge. doi: http://dx.doi.org/10.17863/CAM.18777
Karim, M. R., Yang, Y., Messinger, D., and Ruhe, G. (2018). Learn or earn? - intelligent task recommendation for competitive crowdsourced software development. doi: http://dx.doi.org/10.24251/HICSS.2018.700
Kittur, A., Smus, B., Khamkar, S., and Kraut, R. E. (2011). Crowdforge: Crowdsourcing complex work. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pages 43–52, New York, NY, USA. ACM. doi: https://doi.org/10.1145/2047196.2047202
Kurve, A., Miller, D. J., and Kesidis, G. (2015). Multicategory crowdsourcing accounting for variable task difficulty, worker skill, and worker intention. IEEE Transactions on Knowledge and Data Engineering, 27(3):794–809. doi: https://doi.org/10.1109/TKDE.2014.2327026
LaToza, T. D., Towne, W. B., Adriano, C. M., and van der Hoek, A. (2014). Microtask programming: Building software with a crowd. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14, pages 43–54, New York, NY, USA. ACM. doi: https://doi.org/10.1145/2642918.2647349
LaToza, T. D. and v. d. Hoek, A. (2015). A vision of crowd development. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, volume 2, pages 563–566. doi: https://doi.org/10.1109/ICSE.2015.194
Mao, K., Capra, L., Harman, M., and Jia, Y. (2017). A survey of the use of crowdsourcing in software engineering. Journal of Systems and Software, 126:57 – 84. doi: https://doi.org/10.1016/j.jss.2016.09.015
Mao, K., Yang, Y., Li, M., and Harman, M. (2013). Pricing crowdsourcing-based software development tasks. In 2013 35th International Conference on Software Engineering (ICSE), pages 1205–1208. doi: https://doi.org/10.1109/ICSE.2013.6606679
Naik, N. (2016). Crowdsourcing, open-sourcing, outsourcing and insourcing software development: A comparative analysis. In 2016 IEEE Symposium on Service-Oriented System Engineering (SOSE), pages 380–385. doi: https://doi.org/10.1109/SOSE.2016.68
Nakatsu, R. T., Grossman, E. B., and Iacovou, C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6):823–834. doi: https://doi.org/10.1177%2F0165551514550140
Sui, D., Elwood, S., and Goodchild, M. (2012). Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. Springer Publishing Company, Incorporated. doi: https://www.doi.org/10.1007/978-94-007-4587-2
TopCoder (2017). Projects - topcoder crowdsourcing.
Tranquillini, S., Daniel, F., Kucherbaev, P., and Casati, F. (2015). Modeling, enacting, and integrating custom crowdsourcing processes. ACM Trans. Web, 9(2). Doi: https://doi.org/10.1145/2746353
Wei Li, Michael N. Huhns, W.-T. T. W. W. (2015). Crowdsourcing Cloud-Based Software Development. Springer, first edition. doi: https://www.doi.org/10.1007/978-3-662-47011-4
Winkler, D., Sabou, M., Petrovic, S., Carneiro, G., Kalinowski, M., and Biffl, S. (2017). Improving Model Inspection with Crowdsourcing. IEEE/ACM 4th International Workshop on Crowdsourcing in Software Engineering (CSI-SE), pages 30-40. doi: http://doi.ieeecomputersociety.org/10.1109/CSI-SE.2017.2
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. (2012). Experimentation in software engineering. Springer Science & Business Media. doi: https://www.doi.org/10.1007/978-3-642-29044-2
Xiao, L. and Paik, H. Y. (2014). Supporting complex work in crowdsourcing platforms: A view from service-oriented computing. In 2014 23rd Australian Software Engineering Conference, pages 11–14. doi: https://doi.org/10.1109/ASWEC.2014.11
Yang J. Y. Redi, J. Demartini, G. B. A. (2016). Modeling task complexity in crowdsourcing. Fourth AAAI Conference on Human Computation and Crowdsourcing.
Yin, R. K. (2015). Estudo de Caso: Planejamento e Métodos. Bookman editora.
Aipe, A. and Gadiraju, U. (2018). Similarhits: Revealing the role of task similarity in microtask crowdsourcing. HT.
Chandra, G., Gupta, D. L., and Malviya, K. (2012). Some observations based on comparison of mood and ck software metrics suites for object oriented system. International Journal of Computer Science and Technology, 3(3).
Deus, W. S., Barros, R. M., and L’Erario, A. (2016). Um modelo para o gerenciamento do crowdsourcing em projetos de software. In I Workshop sobre Aspectos Sociais, Humanos e Economicos de Software (WASHES’2016).
Deus, W. S., Fabri, J. A., and L’Erario, A. (2017). The management of crowdsourcing software projects: A systematic mapping. In 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), pages 1–7. doi: https://doi.org/10.23919/CISTI.2017.7975711
de Freitas Junior, M., Fantinato, M., and Sun, V. (2015). Improvements to the function point analysis method: A systematic literature review. IEEE Transactions on Engineering Management, 62(4):495–506. doi: https://doi.org/10.1109/TEM.2015.2453354
Dubey, A., Abhinav, K., Taneja, S., Virdi, G., Dwarakanath, A., Kass, A., and Kuriakose, M. S. (2016). Dynamics of software development crowdsourcing. In 2016 IEEE 11th International Conference on Global Software Engineering (ICGSE), pages 49–58. doi: https://doi.org/10.1109/ICGSE.2016.13
Dwarakanath, A., Chintala, U., Shrikanth, N. C., Virdi, G., Kass, A., Chandran, A., Sengupta, S., and Paul, S. (2015). Crowd build: A methodology for enterprise software development using crowdsourcing. In 2015 IEEE/ACM 2nd International Workshop on CrowdSourcing in Software Engineering, pages 8–14. doi: https://doi.org/10.1109/CSI-SE.2015.9
Goke, N. and Freitag, E. (2014). Microtask platforms a win/win/win situation. In ¨ Colletive Inteligence. Greicius, T. (2018). Multi-planet system found through crowdsourcing. disponivel em: https://www.nasa.gov/feature/jpl/multi-planet-system-found-through-crowdsourcing. NASA.
Hosseini, M., Phalp, K., Taylor, J., and Ali, R. (2014). The four pillars of crowdsourcing: A reference model. In 2014 IEEE Eighth International Conference on Research Challenges in Information Science (RCIS), pages 1–12. doi: https://doi.org/10.1109/RCIS.2014.6861072
Howe, J. (2006a). Crowdsourcing: A definition. https://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html, Janeiro, 2019.
Howe, J. (2006b). The rise of crowdsourcing. Wired magazine, 14(6):1–4. https://www.wired.com/2006/06/crowds/, Janeiro, 2019.
Jacques, J. T. (2018). Microtask Design: Value, Engagement, Context, and Complexity. PhD thesis, University of Cambridge. doi: http://dx.doi.org/10.17863/CAM.18777
Karim, M. R., Yang, Y., Messinger, D., and Ruhe, G. (2018). Learn or earn? - intelligent task recommendation for competitive crowdsourced software development. doi: http://dx.doi.org/10.24251/HICSS.2018.700
Kittur, A., Smus, B., Khamkar, S., and Kraut, R. E. (2011). Crowdforge: Crowdsourcing complex work. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pages 43–52, New York, NY, USA. ACM. doi: https://doi.org/10.1145/2047196.2047202
Kurve, A., Miller, D. J., and Kesidis, G. (2015). Multicategory crowdsourcing accounting for variable task difficulty, worker skill, and worker intention. IEEE Transactions on Knowledge and Data Engineering, 27(3):794–809. doi: https://doi.org/10.1109/TKDE.2014.2327026
LaToza, T. D., Towne, W. B., Adriano, C. M., and van der Hoek, A. (2014). Microtask programming: Building software with a crowd. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14, pages 43–54, New York, NY, USA. ACM. doi: https://doi.org/10.1145/2642918.2647349
LaToza, T. D. and v. d. Hoek, A. (2015). A vision of crowd development. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, volume 2, pages 563–566. doi: https://doi.org/10.1109/ICSE.2015.194
Mao, K., Capra, L., Harman, M., and Jia, Y. (2017). A survey of the use of crowdsourcing in software engineering. Journal of Systems and Software, 126:57 – 84. doi: https://doi.org/10.1016/j.jss.2016.09.015
Mao, K., Yang, Y., Li, M., and Harman, M. (2013). Pricing crowdsourcing-based software development tasks. In 2013 35th International Conference on Software Engineering (ICSE), pages 1205–1208. doi: https://doi.org/10.1109/ICSE.2013.6606679
Naik, N. (2016). Crowdsourcing, open-sourcing, outsourcing and insourcing software development: A comparative analysis. In 2016 IEEE Symposium on Service-Oriented System Engineering (SOSE), pages 380–385. doi: https://doi.org/10.1109/SOSE.2016.68
Nakatsu, R. T., Grossman, E. B., and Iacovou, C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6):823–834. doi: https://doi.org/10.1177%2F0165551514550140
Sui, D., Elwood, S., and Goodchild, M. (2012). Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. Springer Publishing Company, Incorporated. doi: https://www.doi.org/10.1007/978-94-007-4587-2
TopCoder (2017). Projects - topcoder crowdsourcing.
Tranquillini, S., Daniel, F., Kucherbaev, P., and Casati, F. (2015). Modeling, enacting, and integrating custom crowdsourcing processes. ACM Trans. Web, 9(2). Doi: https://doi.org/10.1145/2746353
Wei Li, Michael N. Huhns, W.-T. T. W. W. (2015). Crowdsourcing Cloud-Based Software Development. Springer, first edition. doi: https://www.doi.org/10.1007/978-3-662-47011-4
Winkler, D., Sabou, M., Petrovic, S., Carneiro, G., Kalinowski, M., and Biffl, S. (2017). Improving Model Inspection with Crowdsourcing. IEEE/ACM 4th International Workshop on Crowdsourcing in Software Engineering (CSI-SE), pages 30-40. doi: http://doi.ieeecomputersociety.org/10.1109/CSI-SE.2017.2
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. (2012). Experimentation in software engineering. Springer Science & Business Media. doi: https://www.doi.org/10.1007/978-3-642-29044-2
Xiao, L. and Paik, H. Y. (2014). Supporting complex work in crowdsourcing platforms: A view from service-oriented computing. In 2014 23rd Australian Software Engineering Conference, pages 11–14. doi: https://doi.org/10.1109/ASWEC.2014.11
Yang J. Y. Redi, J. Demartini, G. B. A. (2016). Modeling task complexity in crowdsourcing. Fourth AAAI Conference on Human Computation and Crowdsourcing.
Yin, R. K. (2015). Estudo de Caso: Planejamento e Métodos. Bookman editora.
Downloads
Published
2018-12-21
Como Citar
Deus, W. S. de, Fabri, J. A., & L’Erario, A. (2018). Um Método para Determinar a Complexidade de Microtasks Executadas em Ambientes Crowdsourcing. ISys - Revista Brasileira De Sistemas De Informação, 11(4), 05–30. https://doi.org/10.5753/isys.2018.375
Issue
Section
Artigos de Edição Especial
