Influence of data stratification criteria on fairer classifications
DOI:
https://doi.org/10.5753/jidm.2025.4677Keywords:
Analysis, Binary Classification, Data Bias, Discriminatory Effects, Fairness, Machine Learning, Supervised LearningAbstract
Data stratification by class is a prominent strategy to enhance the accuracy of model evaluation in unbalanced scenarios. This type of strategy, added to other stratification criteria, can also be effective in a significant issue with machine learning systems, which is their potential to propagate discriminatory effects, harming specific people groups. Therefore, it is crucial to assess whether these systems' decision-making processes are fair across the diversity present in society. This assessment requires stratifying the test set not only by class but also by sociodemographic groups. Furthermore, applying stratification by class and group during the validation step can contribute to developing fairer models. Despite its importance, there is a lack of studies analyzing the influence of data stratification on fairness in machine learning. We address this gap and propose an experimental setup to analyze how different data stratification criteria influence the development of impartial classifiers. Our results suggest that stratifying data by class and group aids develop fairer classifiers, thereby minimizing the spread of discriminatory effects in decision-making processes.
Downloads
References
Alikhademi, K., Drobina, E., Prioleau, D., Richardson, B., Purves, D., and Gilbert, J. E. (2022). A review of predictive policing from the perspective of fairness. Artificial Intelligence and Law, pages 1–17.
Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016). Machine bias: Risk assessments in criminal sentencing.
Barocas, S., Hardt, M., and Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities. MIT Press.
Barocas, S. and Selbst, A. D. (2016). Big data’s disparate impact. Calif. L. Rev., 104:671.
BRASIL (1988). Constituição da República Federativa do Brasil. Brasília, DF: Centro Gráfico.
Buolamwini, J. and Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91.
Calmon, F., Wei, D., Vinzamuri, B., Natesan Ramamurthy, K., and Varshney, K. R. (2017). Optimized pre-processing for discrimination prevention. Advances in Neural Information Processing Systems, 30:3992–4001.
Celis, L. E., Huang, L., Keswani, V., and Vishnoi, N. K. (2019). Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the conference on fairness, accountability, and transparency, pages 319–328.
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153–163.
Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res., 7:1–30.
Dua, D. and Graff, C. (2017). UCI machine learning repository.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. (2012). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214–226.
Gerdon, F., Bach, R. L., Kern, C., and Kreuter, F. (2022). Social impacts of algorithmic decision-making: A research agenda for the social sciences. Big Data & Society, 9(1):20539517221089305.
Goodman, B. and Flaxman, S. (2017). European union regulations on algorithmic decision-making and a “right to explanation”. AI Magazine, 38(3):50–57.
Hanna, A., Denton, E., Smart, A., and Smith-Loud, J. (2020). Towards a critical race methodology in algorithmic fairness. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 501–512.
Hardt, M., Price, E., and Srebro, N. (2016). Equality of opportunity in supervised learning. Advances in neural information processing systems, 29:3315–3323.
Howard, A. and Borenstein, J. (2018). The ugly truth about ourselves and our robot creations: the problem of bias and social inequity. Science and engineering ethics, 24(5):1521–1536.
Kamiran, F. and Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and information systems, 33(1):1–33.
Karimi, H., Akbar Khan, M. F., Liu, H., Derr, T., and Liu, H. (2022). Enhancing individual fairness through propensity score matching. In 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA), pages 1–10. DOI: 10.1109/DSAA54385.2022.10032333.
Larson, J., Mattu, S., Kirchner, L., and Angwin, J. (2016). How we analyzed the compas recidivism algorithm.
Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., and Ntoutsi, E. (2022). A survey on datasets for fairness-aware machine learning. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(3):e1452.
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., and Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6):1–35.
Minatel, D., da Silva, A. C. M., dos Santos, N. R., Curi, M., Marcacini, R. M., and de Andrade Lopes, A. (2023a). Data stratification analysis on the propagation of discriminatory effects in binary classification. In Anais do XI Symposium on Knowledge Discovery, Mining and Learning, pages 73–80. SBC.
Minatel, D., dos Santos, N. R., da Silva, A. C. M., Cúri, M., Marcacini, R. M., and Lopes, A. d. A. (2023b). Unfairness in machine learning for web systems applications. In Proceedings of the 29th Brazilian Symposium on Multimedia and the Web, pages 144–153.
Minatel, D., dos Santos, N. R., da Silva, V. F., Cúri, M., and de Andrade Lopes, A. (2023c). Item response theory in sample reweighting to build fairer classifiers. In Annual International Conference on Information Management and Big Data, pages 184–198. Springer.
Minatel, D., Parmezan, A. R., Cúri, M., and de A. Lopes, A. (2023d). Dif-sr: A differential item functioning-based sample reweighting method. In Iberoamerican Congress on Pattern Recognition, pages 630–645. Springer.
Minatel, D., Parmezan, A. R., Cúri, M., and Lopes, A. D. A. (2023e). Fairness-aware model selection using differential item functioning. In 2023 International Conference on Machine Learning and Applications (ICMLA), pages 1971–1978. IEEE.
Narasimhan, H. (2018). Learning with complex loss functions and constraints. In International Conference on Artificial Intelligence and Statistics, pages 1646–1654.
Parmezan, A. R. S., Lee, H. D., and Wu, F. C. (2017). Metalearning for choosing feature selection algorithms in data mining: Proposal of a new framework. Expert Systems with Applications, 75:1–24.
Pastaltzidis, I., Dimitriou, N., Quezada-Tavarez, K., Aidinlis, S., Marquenie, T., Gurzawska, A., and Tzovaras, D. (2022). Data augmentation for fairness-aware machine learning: Preventing algorithmic bias in law enforcement systems. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 2302–2314, New York, NY, USA. Association for Computing Machinery.
Pessach, D. and Shmueli, E. (2022). A review on fairness in machine learning. ACM Computing Surveys (CSUR), 55(3):1–44. Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., and
Weinberger, K. Q. (2017). On fairness and calibration. Advances in Neural Information Processing Systems, 30:5680–5689.
Valentim, I., Lourenço, N., and Antunes, N. (2019). The impact of data preparation on the fairness of software systems. In 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE), pages 391–401. IEEE.
Vanschoren, J., van Rijn, J. N., Bischl, B., and Torgo, L. (2013). Openml: networked science in machine learning. SIGKDD Explorations, 15(2):49–60. DOI: 10.1145/2641190.2641198.
Xu, T., White, J., Kalkan, S., and Gunes, H. (2020). Investigating bias and fairness in facial expression recognition. In Computer Vision – ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part VI, page 506–523, Berlin, Heidelberg. Springer-Verlag.
Yucer, S., Akcay, S., Al-Moubayed, N., and Breckon, T. P. (2020). Exploring racial bias within face recognition via per-subject adversarially-enabled data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
Zhang, B. H., Lemoine, B., and Mitchell, M. (2018). Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340.
