Towards Design Justice and Gender Perspectives - An Inclusive Audit of Facial Recognition Systems

Aildson Silva; George Valença

doi:10.5753/jis.2026.7084

Authors

Aildson Silva Federal Rural University of Pernambuco https://orcid.org/0009-0003-5704-2359
George Valença Federal Rural University of Pernambuco https://orcid.org/0000-0001-9375-5354

DOI:

https://doi.org/10.5753/jis.2026.7084

Keywords:

Facial Recognition, Automatic Gender Recognition, Design Justice, Algorithmic Bias, Inclusion

Abstract

This study presents a qualitative audit of commercially available facial recognition tools featuring Automated Gender Recognition (AGR), specifically Amazon Rekognition, Face++, and Google's Vision AI, to investigate their inherent intersectional biases and broader ethical and social implications. While Artificial Intelligence offers efficiency, evaluating AI bias using isolated demographic categories overlooks intersectional discrimination, allowing deeper systemic inequalities to persist, disproportionately harming marginalized groups such as Black, transgender, and non-binary individuals. To counteract these biases, we evaluate the tools across four critical dimensions: legal and corporate responsibility, ethical considerations, social implications, and Justice, Equity, Diversity, and Inclusion (JEDI). Our findings reveal a mixed landscape. Google demonstrated a proactive shift toward algorithmic justice by removing gender labels from its Vision AI, acknowledging the problematic nature of inferring gender from appearance. Conversely, Amazon Rekognition and Face++ exhibit a concerning lack of transparency regarding their training datasets and misalignment with inclusive best practices, particularly concerning non-binary gender views and the necessity of gender identification. Ultimately, the findings reinforce that achieving algorithmic justice requires moving beyond mere inclusion to ensure user autonomy, privacy, and systemic change. Relying solely on technical accuracy is insufficient; early integration of frameworks like Design Justice and IEEE Ethically Aligned Design is essential. Furthermore, policy must evolve to prevent AI from being constrained by binary legal definitions. Establishing ethical, human-centric technology demands strong interdisciplinary collaboration to protect vulnerable populations and ensure algorithms genuinely respect the diversity of the communities they serve.

Downloads

Download data is not yet available.

References

Adjabi, I., Ouahabi, A., Benzaoui, A., and Taleb-Ahmed, A. (2020). Past, present, and future of face recognition: A review. Electronics, 9(8). DOI: https://doi.org/10.3390/electronics9081188.

Amazon (2019). Amazon rekognition launches enhanced face analysis. [link], Accessed: 10 July 2026.

Amazon (2023). Our commitment to the responsible use of ai. [link], Accessed: 10 July 2026.

Amazon (2025). What is amazon rekognition? [link], Accessed: 10 July 2026.

Beretta, E., Voto, C., and Rozera, E. (2024). Decoding faces: Misalignments of gender identification in automated systems. Journal of Responsible Technology, 19:100089. DOI: https://doi.org/10.1016/j.jrt.2024.100089.

Buolamwini, J. and Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Friedler, S. A. and Wilson, C., editors, Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pages 77–91. PMLR. [link], Accessed: 10 July 2026.

Collins, P. (1990). Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment. Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment. Routledge.

Costanza-Chock, S. (2020). Design Justice: Community-Led Practices to Build the Worlds We Need. The MIT Press.

Coston, A., Kawakami, A., Zhu, H., Holstein, K., and Heidari, H. (2023). A validity perspective on evaluating the justified use of data-driven decision-making algorithms. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 690–704, Los Alamitos, CA, USA. IEEE Computer Society. DOI: https://doi.org/10.1109/SaTML54575.2023.00050.

Duan, W., Li, L., Freeman, G., and McNeese, N. (2025). A scoping review of gender stereotypes in artificial intelligence. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI '25, pages 1–20, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3706598.3713093.

Google (2018). Our ai principles. [link], Accessed: 10 July 2026.

Google (2025a). Google's model cards. [link], Accessed: 10 July 2026.

Google (2025b). Vision ai: Image and visual ai tools. [link], Accessed: 10 July 2026.

Grannis, W. (2025). 2025 and the next chapter(s) of ai. [link], Accessed: 10 July 2026.

Hamidi, F., Scheuerman, M. K., and Branham, S. M. (2018). Gender recognition or gender reductionism? the social implications of embedded gender recognition systems. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI '18, page 1–13, New York, NY, USA. Association for Computing Machinery. DOI: https://doi.org/10.1145/3173574.3173582.

HFSecurity (2023). what is face++ face recognition. [link], Accessed: 10 July 2026.

Jeyaretnam, M. (2025). These u.s. companies are not ditching dei amid trump's crackdown. [link], Accessed: 10 July 2026.

Kayser-Bril, N. (2020). Google apologizes after its vision ai produced racist results. [link], Accessed: 10 July 2026.

Keyes, O. (2018). The misgendering machines: Trans/hci implications of automatic gender recognition. Proc. ACM Hum.-Comput. Interact., 2(CSCW). DOI: https://doi.org/10.1145/3274357.

Klein, G., Hoffman, R. R., Clancey, W. J., Mueller, S. T., Jentsch, F., and Jalaeian, M. (2023). "Minimum necessary rigor" in empirically evaluating human–ai work systems. AI Magazine, 44(3):274–281. DOI: https://doi.org/10.1002/aaai.12108.

Leben, D. (2025). AI Fairness: Designing Equal Opportunity Algorithms. The MIT Press.

Marangon, J. D. (2025). Google cloud vision api for image handling and ocr. [link], Accessed: 10 July 2026.

Martinez-Martin, N. (2019). What are important ethical implications of using facial recognition technology in health care? AMA Journal of Ethics, 21(2):E180–187. DOI: https://doi.org/10.1001/amajethics.2019.180.

McLymore, A. (2025). Amazon cuts reference to diversity from annual report. [link], Accessed: 10 July 2026.

Mohammadi, E., Cai, Y., Novin, A., Vera, V., and Soltanmohammadi, E. (2025). Who is a scientist? gender and racial biases in google vision ai. AI and Ethics, 5(5):4993–5010. DOI: https://doi.org/10.1007/s43681-025-00742-4.

Moon, D. and Ahn, S. (2025). Metrics and algorithms for identifying and mitigating bias in ai design: A counterfactual fairness approach. IEEE Access, 13:59118–59129. DOI: https://doi.org/10.1109/ACCESS.2025.3556082.

Nunes, J. L., Barbosa, G. D. J., de Souza, C. S., and Barbosa, S. D. J. (2024). Using model cards for ethical reflection on machine learning models: an interview-based study. Journal on Interactive Systems, 15(1):1–19. DOI: https://doi.org/10.5753/jis.2024.3444.

Oyěwùmí, O. (2021). A invenção das mulheres: Construindo um sentido africano para os discursos ocidentais de gênero. Bazar do Tempo. [link], Accessed: 14 July 2026.

Paganoni, M. C. et al. (2019). Ethical concerns over facial recognition technology. Anglistica AION an Interdisciplinary Journal, 1:83–92. DOI: https://doi.org/10.19231/angl-aion.201915.

Parks, M. (2025). The ethics of ai at the intersection of transgender identity and neurodivergence. Discover Artificial Intelligence, 5(1):34. DOI: https://doi.org/10.1007/s44163-025-00257-1.

Perilo, M., Valença, G., and Telles, A. (2024). Non-binary and trans-inclusive ai: A catalogue of best practices for developing automatic gender recognition solutions. SIGAPP Appl. Comput. Rev., 24(2):55–70. DOI: https://doi.org/10.1145/3687251.3687255.

Quaresmini, C. and Zanotti, G. (2025). Fairness through feedback: Addressing algorithmic misgendering in automatic gender recognition. arXiv. DOI: https://arxiv.org/abs/2506.02017.

Quinan, C. and Hunt, M. (2022). Biometric bordering and automatic gender recognition: Challenging binary gender norms in everyday biometric technologies. Communication, Culture and Critique, 15:211–226. DOI: https://doi.org/10.1093/ccc/tcac013.

Rafael, R. d. M. R., Santos, H. G. d. S., Caravaca-Morera, J. A., Wilson, E. C., and Breda, K. L. (2023). Inclusão ou ilusão da identidade de gênero no país com o maior número de assassinatos de transgêneros: um ensaio crítico brasileiro. Escola Anna Nery, 27:e20230117. DOI: https://doi.org/10.1590/2177-9465-EAN-2023-0117pt.

Raza, S., Shaban-Nejad, A., Dolatabadi, E., and Mamiya, H. (2024). Exploring bias and prediction metrics to characterise the fairness of machine learning for equity-centered public health decision-making: A narrative review. IEEE Access, 12:180815–180829. DOI: https://doi.org/10.1109/ACCESS.2024.3509353.

Reis, V. Q. d., Rabello, M. E. R., Lima, A. C., Jardim, G. P. S., Fernandes, E. R., and Brefeld, U. (2023). Data practices in apps from brazil: What do privacy policies inform us about? Journal on Interactive Systems, 14(1):1–8. DOI: https://doi.org/10.5753/jis.2023.2954.

Reuters (2025). Amazon ending dei programs. [link], Accessed: 10 July 2026.

Santos, L. G. d. M., Costa, A. B. d., David, J. d. S., and Pedro, R. M. L. R. (2023). Reconhecimento facial: Tecnologia, racismo e construção de mundos possíveis. Psicologia Sociedade, 35:e277141. DOI: https://doi.org/10.1590/1807-0310/2023v35e277141.

Scheuerman, M. K., Paul, J. M., and Brubaker, J. R. (2019). How computers see gender: An evaluation of gender classification in commercial facial analysis services. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–22. Article 144. DOI: https://doi.org/10.1145/3359643.

ScienceDaily (2025). Google's deepfake hunter sees what you can't - even in videos without faces. [link], Accessed: 10 July 2026.

Scott, J. W. (1986). Gender: A useful category of historical analysis. The American Historical Review, 91(5):1053–1075. DOI: https://doi.org/10.2307/1864376.

Silva, H. H. (2021). Algoritmos de reconhecimento facial e as discriminações contra pessoas transexuais. Internet & Sociedade, 2(2):47–66. [link], Accessed: 10 July 2026.

Silva, L. K. M. d., Silva, A. L. M. A. d., Coelho, A. A., and Martiniano, C. S. (2017). Uso do nome social no sistema Único de saúde: elementos para o debate sobre a assistência prestada a travestis e transexuais. Physis: Revista de Saúde Coletiva, 27(3):835–846. Accessed: 10 July 2026. DOI: https://doi.org/10.1590/S0103-73312017000300023.

Silva, T. (2022). Racismo algorítmico: inteligência artificial e discriminação nas redes digitais. Edições Sesc SP. [link], Accessed: 10 July 2026.

Soldatelli, G. D. (2022). Uma avaliação sobre o viés de gênero em reconhecimento facial. Master's thesis, Universidade Federal de Mato Grosso. [link], Accessed: 10 July 2026.

Spizzirri, G., Eufrásio, R., Lima, M. C. P., de Carvalho Nunes, H. R., Kreukels, B. P. C., Steensma, T. D., and Abdo, C. H. N. (2021). Proportion of people identified as transgender and non-binary gender in brazil. Scientific Reports, 11(1):2240. DOI: https://doi.org/10.1038/s41598-021-81411-4.

Tafesse, E. (2025). Tech companies are abandoning diversity to satisfy trump. it will hurt them in the long run. [link], Accessed: 10 July 2026.

Turchi, T., Malizia, A., and Borsci, S. (2024). Reflecting on algorithmic bias with design fiction: The minicode workshops. IEEE Intelligent Systems, 39(02):40–50. DOI: https://doi.org/10.1109/MIS.2024.3352977.

Union, A. C. L. (2018). Amazon's face recognition falsely matched 28 members of congress with mugshots. [link], Accessed: 10 July 2026.