Co-Training with Active Contrastive Learning and Meta-Pseudo-Labeling on 2D Projections for Deep Semi-Supervised Learning

David Aparco-Cardenas; Jancarlo F. Gomes; Alexandre X. Falcão; Pedro J. de Rezende

doi:10.5753/jbcs.2026.5883

Authors

David Aparco-Cardenas University of Campinas https://orcid.org/0000-0003-2537-7050
Jancarlo F. Gomes University of Campinas https://orcid.org/0000-0002-2466-7315
Alexandre X. Falcão University of Campinas https://orcid.org/0000-0002-2914-5380
Pedro J. de Rezende University of Campinas https://orcid.org/0000-0002-9529-4253

DOI:

https://doi.org/10.5753/jbcs.2026.5883

Keywords:

Semi-supervised learning, Contrastive learning, Pseudo labeling, Deep feature annotation

Abstract

A major challenge that prevents the training of deep learning models is the limited availability of large quantities of accurately labeled data. This shortcoming is particularly acute in areas such as medical and biological sciences where data annotation is an expert-demanding, time-consuming, and error-prone undertaking. In this regard, semi-supervised learning tackles this challenge by capitalizing on scarce labeled and abundant unlabeled data; however, state-of-the-art methods typically depend on pre-trained features and large validation sets to learn effective representations for classification tasks. In addition, the reduced set of labeled data is often randomly sampled, neglecting the selection of more informative samples. Here, we present active Deep Feature Annotation (active-DeepFA), a method that effectively combines contrastive learning, teacher-student-based meta-pseudo-labeling and active learning to train non-pre-trained CNN architectures for image classification in scenarios of scarcity of labeled and abundance of unlabeled data. It integrates deep feature annotation (DeepFA) into a co-training setup that implements two cooperative networks to mitigate confirmation bias arising from pseudo-labels. The method starts with a reduced set of labeled samples by warming up the networks with supervised contrastive learning. Afterward and at regular epoch intervals, label propagation is performed on the 2D projections of the networks' deep features. Next, the most reliable pseudo-labels are exchanged between networks in a cross-training fashion, while the most meaningful samples are annotated and added to the labeled set. The networks independently minimize an objective loss function comprising supervised contrastive, supervised, and semi-supervised loss components, enhancing the representations for image classification. Our approach is evaluated on seven challenging image datasets across three distinct domains with only 5% of labeled samples, surpassing baselines and outperforming seven established benchmarks. In addition, it reduces annotation effort by achieving comparable results to those of its counterparts with only 3% of labeled data.

Downloads

Download data is not yet available.

References

Amorim, W. P., Falcão, A. X., Papa, J. P., and Carvalho, M. H. (2016). Improving semi-supervised learning through optimum connectivity. Pattern Recognition, 60:72-85. DOI: 10.1016/j.patcog.2016.04.020.

Aparco-Cardenas, D., Gomes, J. F., Falcão, A. X., and De Rezende, P. J. (2024). Contrastive learning and iterative meta-pseudo-labeling on 2d projections for deep semi-supervised learning. In 2024 37th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 1-6. IEEE. DOI: 10.1109/SIBGRAPI62404.2024.10716313.

Aparco-Cardenas, D., Gomes, J. F., Falcão, A. X., and de Rezende, P. J. (2025). Consensus-based iterative meta-pseudo-labeling for deep semi-supervised learning. Information Sciences, 692:121671. DOI: 10.1016/j.ins.2024.121671.

Arazo, E., Ortego, D., Albert, P., O’Connor, N. E., and McGuinness, K. (2020). Pseudo-labeling and confirmation bias in deep semi-supervised learning. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE. DOI: 10.1109/IJCNN48605.2020.9207304.

Benato, B. C., Gomes, J. F., Telea, A. C., and Falcão, A. X. (2021a). Semi-automatic data annotation guided by feature space projection. Pattern Recognition, 109:107612. DOI: 10.1016/j.patcog.2020.107612.

Benato, B. C., Telea, A. C., and Falcão, A. X. (2021b). Iterative pseudo-labeling with deep feature annotation and confidence-based sampling. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 192-198. IEEE. DOI: 10.1109/SIBGRAPI54419.2021.00034.

Benato, B. C., Telea, A. C., and Falcão, A. X. (2023). Deep feature annotation by iterative meta-pseudo-labeling on 2d projections. Pattern Recognition, 141:109649. DOI: 10.1016/j.patcog.2023.109649.

Cascante-Bonilla, P., Tan, F., Qi, Y., and Ordonez, V. (2021). Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 6912-6920. DOI: 10.1609/AAAI.V35I8.16852.

Dong, S., Wang, P., and Abbas, K. (2021). A survey on deep learning and its applications. Computer Science Review, 40:100379. DOI: 10.1016/j.cosrev.2021.100379.

Gao, M., Zhang, Z., Yu, G., Arık, S. Ö., Davis, L. S., and Pfister, T. (2020). Consistency-based semi-supervised active learning: Towards minimizing labeling cost. In European Conference on Computer Vision, pages 510-526. Springer. DOI: 10.1007/978-3-030-58607-2_30.

He, K., Zhang, X., Ren, S., and Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 1026-1034. DOI: 10.1109/ICCV.2015.123.

Laine, S. and Aila, T. (2017). Temporal ensembling for semi-supervised learning. In International Conference on Learning Representations. DOI: 10.48550/arxiv.1610.02242.

Lee, D.-H. et al. (2013). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges in Representation Learning, ICML, volume 3, page 896. Atlanta. Available at:[link].

Loshchilov, I. and Hutter, F. (2017). SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations. Available at:[link].

Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. (2018). Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8):1979-1993. DOI: 10.1109/TPAMI.2018.2858821.

Pham, H., Dai, Z., Xie, Q., and Le, Q. V. (2021). Meta pseudo labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11557-11568. DOI: 10.1109/CVPR46437.2021.01139.

Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., and Wang, X. (2021). A survey of deep active learning. ACM Computing Surveys (CSUR), 54(9):1-40. DOI: 10.1145/3472291.

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211-252. DOI: 10.1007/s11263-015-0816-y.

Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., and Raffel, C. (2020). Fixmatch: simplifying semi-supervised learning with consistency and confidence. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20. Curran Associates Inc.. DOI: 10.5555/3495724.3495775.

Song, H., Ji, K., Zhang, Y., Xing, X., and Zou, H. (2016). Sparse representation-based sar image target classification on the 10-class mstar data set. Applied Sciences, 6(1):26. DOI: 10.3390/app6010026.

Steed, R. and Caliskan, A. (2021). Image representations learned with unsupervised pre-training contain human-like biases. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 701-713. DOI: 10.1145/3442188.3445932.

Suzuki, C. T. N., Gomes, J. F., Falcão, A. X., Shimizu, S. H., and Papa, J. P. (2013). Automated diagnosis of human intestinal parasites using optical microscopy images. In IEEE 10th International Symposium on Biomedical Imaging, pages 460-463. DOI: 10.1109/ISBI.2013.6556511.

Tarvainen, A. and Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in Neural Information Processing Systems, 30. DOI: 10.5555/3294771.3294885.

Ulyanov, D. (2016). Multicore-tsne. Available at:[link].

Van Der Maaten, L. (2014). Accelerating t-sne using tree-based algorithms. The Journal of Machine Learning Research, 15(1):3221-3245. DOI: 10.5555/2627435.2697068.

Wang, C., Gu, H., and Su, W. (2021). Sar image classification using contrastive learning and pseudo-labels with limited data. IEEE Geoscience and Remote Sensing Letters, 19:1-5. DOI: 10.1109/LGRS.2021.3069224.

Wang, X., Wu, Z., Lian, L., and Yu, S. X. (2022a). Debiased learning from naturally imbalanced pseudo-labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14647-14657. DOI: 10.1109/CVPR52688.2022.01424.

Wang, Y., Chen, H., Fan, Y., Sun, W., Tao, R., Hou, W., Wang, R., Yang, L., Zhou, Z., Guo, L.-Z., Qi, H., Wu, Z., Li, Y.-F., Nakamura, S., Ye, W., Savvides, M., Raj, B., Shinozaki, T., Schiele, B., Wang, J., Xie, X., and Zhang, Y. (2022b). Usb: A unified semi-supervised learning benchmark for classification. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track. DOI: 10.5555/3600270.3600555.

Xie, Q., Dai, Z., Hovy, E., Luong, T., and Le, Q. (2020). Unsupervised data augmentation for consistency training. Advances in Neural Information Processing Systems, 33:6256-6268. DOI: 10.5555/3495724.3496249.

Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., and Ni, B. (2023). Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41. DOI: 10.1038/s41597-022-01721-8.

Yang, X., Song, Z., King, I., and Xu, Z. (2022). A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, pages 1-20. DOI: 10.1109/TKDE.2022.3220219.

Yuan, X., Li, Z., and Wang, G. (2022). Activematch: End-to-end semi-supervised active representation learning. In 2022 IEEE International Conference on Image Processing (ICIP), pages 1136-1140. DOI: 10.1109/ICIP46576.2022.9898008.

Zhang, Y., Wen, J., Wang, X., and Jiang, Z. (2014). Semi-supervised learning combining co-training with active learning. Expert Systems with Applications, 41(5):2372-2378. DOI: 10.1016/j.eswa.2013.09.035.

Zheng, M., You, S., Huang, L., Wang, F., Qian, C., and Xu, C. (2022). Simmatch: Semi-supervised learning with similarity matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14471-14481. DOI: 10.1109/CVPR52688.2022.01407.

Zhou, Z.-H. (2008). Semi-supervised learning by disagreement. In IEEE International Conference on Granular Computing, pages 93-93. DOI: 10.1109/GRC.2008.4664785.