Surveying the Academic Literature on the Use of OKR (Objectives and Key Results) – An Update
DOI:
https://doi.org/10.5753/isys.2024.3885Keywords:
OKR, Objectives and Key Results, Software and IT MeasurementAbstract
OKR (Objectives and Key Results) has emerged as a lightweight framework for business objectives measurement and strategic alignment. Although growing interest in OKR, academic literature on it is still scarce. We executed a systematic mapping study. We found 47 studies and identified associated difficulties, benefits, and lessons learned. In general, OKR use is under-documented from a theoretical point of view. Among the topics associated with using OKR are communication, performance evaluation, tasks planning and prioritization, transparency, team alignment, and goal fulfillment. We found few academic studies addressing the topic in depth. We expect this study to serve as a basis for other researchers to continue to seek and deepen the understanding of how to support the use of OKR and, thus, help build knowledge on the subject.
Downloads
References
Abimaulana, K. C., K.Budiardjo, E., Mahatma, K., and Hidayati, A. (2021). Evaluation of scrum-based software development process maturity using the smm and amm: A case of education technology startup. In 2021 International Conference on Advanced Computer Science and Information Systems (ICACSIS), pages 1–5, Depok, Indonesia. IEEE.
Anatolyevna, K. T., Alexandrovna, A. D., and Alexandrovich, S. N. (2020). Company human capital development based on okr methodology. Process Management and Scientific Developments, 17:5.
Azhar, G. A. and Dewi, M. K. (2023). Refining performance measurement in zakat insti tutions: a holistic proposal of okr implementation. International Journal of Research in Business and Social Science (2147- 4478), 12.
Basili, V., Trendowicz, A., Kowalczyk, M., Heidrich, J., Seaman, C., Munch, J., and Rom- ¨ bach, D. (2014). GQM+Strategies in a Nutshell, pages 9–17. Springer International Publishing, Cham.
Belfort, R., Fernandes, F., and Campos, F. (2021). Combining objective key results, net promoter score and social return of investment to measure project impact. In Intelligent Human Systems Integration 2021, volume 1322, pages 428–433, Cham.
Berntzen, M., Hoda, R., Moe, N. B., and Stray, V. (2022). A taxonomy of inter-team coor dination mechanisms in large-scale agile. IEEE Transactions on Software Engineering, pages 1–1.
Berntzen, M., Stray, V., and Moe, N. B. (2021). Coordination strategies: Managing inter-team coordination challenges in large-scale agile. In Agile Processes in Software Engineering and Extreme Programming, pages 140–156, Cham. Springer International Publishing.
Berntzen, M., Stray, V., Moe, N. B., and Hoda, R. (2023). Responding to change over time: A longitudinal case study on changes in coordination mechanisms in large-scale agile. Empirical Software Engineering, 28.
Boudon, G. (2022). Why Objectives and Key Results (OKRs) Are Not A Substitute for Strategy. talent management excellence essentials – [link], last accessed 2023–11–15.
Bowring, J. and Paasivaara, M. (2021). Keeping the momentum: Driving continuous improvement after the large-scale agile transformation. volume 13126 LNCS.
Brodbeck, A. F. and Hoppen, N. (2003). Alinhamento estratégico entre os planos de negócio e de tecnologia de informação: um modelo operacional para implementação. Revista de Administração Contemporânea ˆ , 7:9–33.
Caldiera, V. R. B.-G. and Rombach, H. D. (1994). Goal question metric approach. Encyclopedia of software engineering, 1(528-532):6.
Cao, R. (2021). Research on teaching evaluation system of higher vocational colleges based on okr and big data. In 2021 16th International Conference on Computer Science & Education (ICCSE), pages 676 – 680, Lancaster, United kingdom.
Castro, F. (2018). The Beginner’s Guide to OKR. What is OKR? [link], last accessed 2023–11–15.
Chen, D., Chen, J., and Ning, M. (2022). Research on enterprise performance management from the perspective of okr. volume 650.
Claudiu-Dan, G. (2015). Internet involvement and metrics in adult education services online marketing in Romania. an evaluation study. Procedia Computer Science, 65:950– 960.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46.
Costa, T. J. L., Fidelis, R., Munck, L., Horst, D. J., and Junior, P. P. D. A. (2023). Using the okr method and fuzzy logic to determine the level of sustainability in restaurants. Sustainability (Switzerland), 15.
Dang, Q.-V., Truong, M.-T., and Huynh, M.-H. (2021). Studying the fuzzy clustering methods to understand employee performance. In 2021 4th International Conference on Artificial Intelligence and Big Data (ICAIBD), pages 541–544, Chengdu, China. IEEE.
den Haak, B. (2021). Moving the Needle with Lean OKRs : Setting Objectives and Key Results to Reach Your Most Ambitious Goal. Business Expert Press, 1st edition edition.
Distante, D., Faralli, S., Rittinghaus, S., Rosso, P., and Samsami, N. (2022). Domain senticnet: An ontology and a methodology enabling domain-aware sentic computing. Cognitive computation, 14(1):62–77.
Doerr, J. (2019). Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs. Portfolio/Penguin, New York, 1st. edition.
Erp, T. V., Rytter, N. G. M., Sieckmann, F., Larsen, M. B., Blichfeldt, H., and Kohl, H. (2021). Management, design, and implementation of innovation projects: Towards a framework for improving the level of automation and digitalization in manufacturing systems. In 2021 9th International Conference on Control, Mechatronics and Automation (ICCMA), pages 211–217, Luxembourg.
Fernandes, F., Belfort, R., and Campos, F. (2021). Agile cmii: A methodology for assessing social project impacts within agile contexts. In Kantola, J. I., Nazir, S., and Salminen, V., editors, Advances in Human Factors, Business Management and Leadership, pages 356–362, Cham. Springer International Publishing.
Ferreira, L. G. A., Viegas, P. B., and Trento, D. (2018). An agile approach applied in enterprise project management office. In Santos, V. A. d., Pinto, G. H. L., and Serra Seca Neto, A. G., editors, Agile Methods, pages 95–102, Cham. Springer International Publishing.
Goldman, S. L. (2012). Smart communities and networked organizations. In Putnik, G. D. and Cruz-Cunha, M. M., editors, Virtual and Networked Organizations, Emergent Technologies and Tools, pages 304–315, Berlin, Heidelberg. Springer Berlin Heidelberg.
Grove, A. S. (2020). High Output Management. Business Essentials. Vahlen, Munchen, ¨ 1 edition.
Gudigantala, N., Madhavaram, S., and Bicen, P. (2023). An ai decision-making framework for business value maximization. AI Magazine, 44.
Harke, W. (2020). OKR Alignment with OKR examples. [link], last accessed 2023–11–15.
Herkenrath, C., Hoeborn, G., and Stich, V. (2023). Why companies fail with objectives and key results: An analysis of implementation frameworks.
Higgins, D. and Madai, V. I. (2020). From bit to bedside: A practical framework for artificial intelligence product development in healthcare. Advanced Intelligent Systems, 2.
Hoeft, F. (2021). The case of sales in the automotive industry during the COVID-19 pandemic. Strategic Change, 30(2):117–125.
ISO (2017). Iso/iec/ieee international standard - systems and software engineering– measurement process. ISO/IEC/IEEE 15939:2017(E), pages 1–49.
Kilu, E., Milani, F., Scott, E., and Pfahl, D. (2019). Agile software process improvement by learning from financial and fintech companies: Lhv bank case study. In Software Quality: The Complexity and Challenges of Software Engineering and Software Quality in the Cloud, pages 57–69, Cham. Springer International Publishing.
Kitchenham, B., Charters, S., et al. (2007). Guidelines for performing systematic literature reviews in software engineering. Keele University, Keele, UK., page 65.
Klanwaree, N. and Choemprayong, S. (2019). Objectives & key results for active knowledge sharing in consulting enterprises: A feasibility study. Proceedings of the Association for Information Science and Technology, 56(1):441–444.
Koldyshev, M. V., Stoliaruk, K. S., Shpynkovskyi, O. O., Mital, O. H., Yamnenko, H. Y., and Dovban, I. M. (2021). Hr management efficiency factors and their impact on creation of a commercial innovative product. Journal of Management Information & Decision Sciences, 24(6).
Kovynyov, I., Buerck, A., and Mikut, R. (2021). Design of transformation initiatives implementing organisational agility: an empirical study. SN Business & Economics, 1.
Lannoo, K. and Parlour, R. (2021). Anti-Money Laundering in the EU: Time to get serious. ECRI Papers 31980, Centre for European Policy Studies.
Mangipudi, M. R., Prasad, K., and Vaidya, R. (2021). Objectives and key results for higher educational institutions–a blended approaches part of post covid-19 initiatives for keeping the institutions abreast of the industry innovations, create future leaders and build the nation. Pacific Business Review International, pages 46–56.
Mann, G., Jevon, S., and Bachmann, B. (2020). How are firms measuring digital trans formation at a corporate-level in organisations? In Australasian Conference on Infor mation Systems: Navigating Our Digital Future, He waka eke noa-We are all in this together, page 88, Melbourne Australia. AIS Electronic Library.
McGarry, J., Card, D., Jones, C., Layman, B., Clark, E., Dean, J., and Hall, F. (2002). Practical Software Measurement Objective Information for Decision Makers. Addison-Wesley, Boston, 1st. edition.
Mikalsen, M., Stray, V., Moe, N. B., and Backer, I. (2020). Shifting conceptualization of control in agile transformations. In Paasivaara, M. and Kruchten, P., editors, Agile Processes in Software Engineering and Extreme Programming – Workshops, pages 173–181, Cham. Springer International Publishing.
Mourao, E., Pimentel, J. F., Murta, L., Kalinowski, M., Mendes, E., and Wohlin, C. (2020). On the performance of hybrid search strategies for systematic literature reviews in software engineering. Information and Software Technology, 123:106294.
Muniz, A., Krieger, C., Patane, V., Krause, W., and Krause, W. (2022). Jornada OKR na Prática: Unindo Práticas e Experiências que Potencializam Resultados - [OKR Journey in Practice: Uniting Practices and Experiences that Potentialize Results]. Brasport, Brazil, 1st. edition.
Mäntylä, V., Lehtelä, B., and Fagerholm, F. (2022). The viability of continuous experimentation in early-stage software startups: A descriptive multiple-case study. volume 13709 LNCS.
Nag, R. K. (2022). Is india ready to accept an edtech-intensive system in post pande mic times? a strategic analysis of india’s “readiness” in terms of basic infrastructural support. DECISION, 49.
Niven, P. R. and Lamorte, B. (2016). Objectives and key results: Driving focus, alignment, and engagement with OKRs. John Wiley & Sons, 1st. edition.
Petersen, K. and Gencel, C. (2013). Worldviews, research methods, and their relationship to validity in empirical software engineering research. In 2013 joint conference of the23rd international workshop on software measurement and the 8th international con ference on software process and product measurement, pages 81–89, Ankara, Turkey. IEEE, IEEE.
Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting syste matic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.
Quantive (2023a). How to run an okr pilot. https://quantive.com/resources/articles/how to-run-okr-pilot, last accessed 2023–11–15.
Quantive (2023b). Ultimate OKRs Playbook. [link], last accessed 2023–11–15.
Rafidah, J. and Dewi, E. (2020). Improving performance management system of human resource aspect in health drink industry startup. Malaysian Journal of Social Sciences and Humanities (MJSSH), 5(8):85 – 95.
Rahmah, A., Sukmasetya, P., Romadhon, M. S., and Adriansyah, A. R. (2020). Developing distance learning monitoring dashboard with Google sheet: An approach for flexible and low-price solution in the pandemic era. In 7th International Conference on ICT for Smart Society: AIoT for Smart Society, ICISS 2020 - Proceeding, Virtual, Ban dung, Indonesia. IEEE.
Roberts, H. and Petticrew, M. (2006). Systematic reviews in the social sciences. Blackwell Pub.
Rojas-Chipana, I., Aguilar-Pelizzoli, N., Quiroz-Flores, J. C., Collao-Diaz, M., and Flores-Perez, A. (2021). Increasing profitability through the implementation of an integrated balance score card & objectives key results model in a digital agency: A research in peru. IEEE Transactions on Software Engineering.
Rompho, N. (2023). Do objectives and key results solve organizational performance measurement issues? Benchmarking.
Schull, M., Hofmann, P., Philipp, P., and Urbach, N. (2023). Reporting in large-scale ¨ agile organizations: insights and recommendations from a case study in software de velopment. Information Systems and e-Business Management.
Scott, E., Milani, F., Kilu, E., and Pfahl, D. (2021). Enhancing agile software develop ment in the banking sector—a comprehensive case study at lhv. Journal of Software: Evolution and Process, 33(7):e2363.
Silva, R. V. and Souza, G. D. S. (2023). Surveying the academic literature on the use of okr (objective and key results). In Proceedings of the XIX Brazilian Symposium on Information Systems, SBSI ’23, page 427–434, New York, NY, USA. Association for Computing Machinery.
Silvius, A. G. (2007). Business & it alignment in theory and practice. In 2007 40th Annual Hawaii International Conference on System Sciences (HICSS’07), pages 211b–211b, Hawaii. IEEE.
Sowkasem, C. and Kirawanich, P. (2021). A deliverable delay management of software development in railway project using an okr-based scrum process. In 2021 The 4th International Conference on Software Engineering and Information Management, IC SIM 2021, page 10–16, New York, NY, USA. Association for Computing Machinery.
Staron, M. and Meding, W. (2018). Software development measurement programs. Springer. [link], 10:3281333
Stray, V., Gundelsby, J. H., Ulfsnes, R., and Moe, N. (2022a). How agile teams make objectives and key results (okrs) work. In ICSSP’22: Proceedings of the International Conference on Software and System Processes and International Conference on Glo bal Software Engineering, page 6, New York, NY, USA. Association for Computing Machinery.
Stray, V., Moe, N. B., Vedal, H., and Berntzen, M. (2022b). Using objectives and key results (okrs) and slack: A case study of coordination in large-scale distributed agile.
Sultan, W. A. M. (2023). Key performance indicators (kpis), key result indicator (kris) and objectives and key results (okrs) - a new key incorporated results (kirs) approach. Information and Knowledge Management.
Tkalich, A., Ulfsnes, R., and Moe, N. B. (2022). Toward an agile product management: What do product managers do in agile companies? volume 445 LNBIP.
Trieflinger, S., Munch, J., Bogazk ¨ oy, E., Eißler, P., Schneider, J., and Roling, B. (2020). ¨ Product roadmap alignment – achieving the vision together: A grey literature review. volume 396 LNBIP.
Trinkenreich, B. and Santos, G. (2015). Avaliação do processo de medição para serviços de ti em uma empresa global a luz do mr-mps-sv - [evaluation of the measurement ` process for it services in a global enterprise based on the mr-mps-sv]. iSys-Brazilian Journal of Information Systems, 8(2):58–77.
Trinkenreich, B., Santos, G., and Barcellos, M. P. (2018). Sinis: A gqm+ strategies-based approach for identifying goals, strategies and indicators for it services. Information and Software Technology, 100:147–164.
Trinkenreich, B., Santos, G., Barcellos, M. P., and Conte, T. (2019). Combining gqm+strategies and okr - preliminary results from a participative case study in industry. In Product-Focused Software Process Improvement, pages 103–111, Cham. Springer International Publishing.
Troian, T. A., Gori, R. S. L., Weber, J. R., Lacerda, D. P., and Gauss, L. (2022). Okrs as a results-focused management model: a systematic literature review. 28th IJCIEOM – International Joint Conference on Industrial Engineering and Operations Manage ment.
van Erp, T. and Rytter, N. G. (2023). Design and operations framework for the twin transition of manufacturing systems. Advances in Production Engineering And Mana gement, 18.
Vedal, H., Stray, V., Berntzen, M., and Moe, N. B. (2021). Managing dependencies in large-scale agile. volume 426, pages 52 – 61, Virtual, Online.
Vellore, V. (2022). OKRs for All: Making Objectives and Key Results Work for your Entire Organization. John Wiley & Sons, 1st edition edition.
Weekdone (2023). What is OKR? Everything to Know. [link], last accessed 2023– 11–15.
Wibawa, A. S., Budiardjo, E. K., and Mahatma, K. (2021). Improving the quality of requirements engineering process in software development with agile methods: A case study telemedicine startup xyz.
Wodtke, C. (2016). Introduction to OKRs. O’Reilly Media, Sebastopol, CA, USA, 1st. edition.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE ’14, New York, NY, USA. Association for Computing Machinery.
Workpath (2022). A complete guide on how to succeed with objectives and key results (okrs) as an enterprise.
Wulff, K. and Finnestrand, H. (2022). It is like taking a ball for a walk: on boundary work in software development. AI & SOCIETY, 37(2):711–724.
Zhou, H. and He, Y.-L. (2018). Comparative study of okr and kpi. In 2018 Interna tional Conference on e-commerce and contemporary economic development (ECED 2018), Destech Transactions on Economics Business and Management, pages 319– 323, Hangzhou, China. Destech Publications.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 iSys - Brazilian Journal of Information Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
