Retinal Vessel Segmentation in OCTA Images with U-Net: A Comparative Analysis of Training Strategies

Authors

DOI:

https://doi.org/10.5753/jdhbi.2026.8362

Keywords:

Optical Coherence Tomography Angiography, Retinal Vessel Segmentation, U-Net, Data Augmentation, Transfer Learning

Abstract

Optical coherence tomography angiography (OCTA) is a non-invasive, depth-resolved imaging modality that enables visualization of retinal vascular slabs, including superficial and deeper plexuses. Accurate vessel segmentation is essential for deriving biomarkers such as vessel density and foveal avascular zone area, which support the assessment of diabetic retinopathy and age-related macular degeneration. This study investigates whether task-specific data augmentation or ImageNet-based transfer learning provides greater benefit for large-vessel segmentation under limited labeled data. We evaluated four U-Net-based configurations on the OCTA\_6mm subset of OCTA-500 using a 70/15/15 subject-level split: a U-Net without pretraining or augmentation, the same model with augmentation, and two variants with ImageNet-pretrained ResNet34 and EfficientNet-B0 encoders. The augmented U-Net achieved Dice = 0.887 and IoU = 0.799, within the range of previously reported OCTA-500 baselines from 2D projection maps and close to the reported IPN volumetric baseline (DSC = 0.882). The pretrained encoders did not outperform the U-Net without pretraining. To isolate the effect of augmentation from concurrent training-recipe changes, we conducted a controlled multi-seed ablation varying only the augmentation policy. Augmentation yielded a modest but consistent Dice gain of $+0.007$ over a matched no-augmentation baseline, driven almost entirely by geometric transformations; intensity and elastic components added no measurable benefit. These findings suggest that, at this data scale, task-preserving geometric invariances are more effective than natural-image pretraining for 2D large-vessel OCTA segmentation.

Descargas

Los datos de descargas todavía no están disponibles.

Citas

Abtahi, M., Le, D., Lim, J. I., and Yao, X. (2022). MF-AV-Net: An open-source deep learning network with multimodal fusion options for artery-vein segmentation in OCT angiography. Biomedical Optics Express, 13(9):4870–4888. DOI: 10.1364/BOE.468483.

Alam, M., Le, D., Son, T., Lim, J. I., and Yao, X. (2020). AV-Net: Deep learning for fully automated artery-vein classification in optical coherence tomography angiography. Biomedical Optics Express, 11(9):5249–5263. DOI: 10.1364/BOE.399514.

Gao, S. S., Jia, Y., Zhang, M., Su, J. P., Liu, G., Hwang, T. S., Bailey, S. T., and Huang, D. (2016). Optical coherence tomography angiography. Investigative Ophthalmology & Visual Science, 57(9):OCT27–OCT36.

Kashani, A. H., Chen, C.-L., Gahm, J. K., Zheng, F., Richter, G. M., Rosenfeld, P. J., Shi, Y., and Wang, R. K. (2017). Optical coherence tomography angiography: A comprehensive review of current methods and clinical applications. Progress in Retinal and Eye Research, 60:66–100. DOI: 10.1016/j.preteyeres.2017.07.002.

Kim, H. E., Cosa-Linan, A., Santhanam, N., Jannesari, M., Maros, M. E., and Ganslandt, T. (2022). Transfer learning for medical image classification: a literature review. BMC Medical Imaging, 22(1):69.

Li, M., Chen, Y., Ji, Z., Xie, K., Yuan, S., Chen, Q., and Li, S. (2020). Image projection network: 3D to 2D image segmentation in OCTA images. IEEE Transactions on Medical Imaging, 39(11):3343–3354. DOI: 10.1109/TMI.2020.2992454.

Li, M., Huang, K., Xu, Q., Yang, J., Zhang, Y., Ji, Z., Xie, K., Yuan, S., Liu, Q., and Chen, Q. (2023). OCTA-500: A retinal dataset for optical coherence tomography angiography study. IEEE Transactions on Medical Imaging, 42(6):1539–1556. DOI: 10.1109/TMI.2022.3228490.

Raghu, M., Zhang, C., Kleinberg, J., and Bengio, S. (2019). Transfusion: Understanding transfer learning for medical imaging. Advances in Neural Information Processing Systems, 32.

Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 234–241. DOI: 10.1007/978-3-319-24574-4_28.

Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., and Ding, X. (2020). Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Medical Image Analysis, 63:101693.

Xu, Q., Li, M., Ji, Z., Yuan, S., Liu, Q., and Chen, Q. (2022). Foveal avascular zone volume: A new index based on 3D U-Net segmentation for OCTA images. Frontiers in Neuroscience, 16:887420. DOI: 10.3389/fnins.2022.887420.

Zhang, H., Ji, Z., Fan, W., Yuan, S., Liu, Q., and Chen, Q. (2020). Robust layer segmentation against complex retinal abnormalities for en face OCTA generation. In Ophthalmic Medical Image Analysis – MICCAI 2020 Workshop, volume 12437 of Lecture Notes in Computer Science. Springer. DOI: 10.1007/978-3-030-63419-3_32.

Descargas

Published

2026-11-13

Cómo citar

Christoph Bohn, H., da Costa, C. A., Lavinsky, F., Garcia Santos, I., Stein, G. ., & Zeiser, F. A. (2026). Retinal Vessel Segmentation in OCTA Images with U-Net: A Comparative Analysis of Training Strategies. Journal of Digital Health and Biomedical Informatics, 1(1), 35–45. https://doi.org/10.5753/jdhbi.2026.8362

Issue

Section

Artigos de Pesquisa