IRO Journals
Journal of Artificial Intelligence and Capsule Networks

Encoder Selection and Segmentation Quality in U-Net-based Retinal Vessel Analysis

Open Access
Published: 14 October, 2026
Pages: 324-345
  • Jegan Sivaraman , Jegan Sivaraman
    Lecturer, Department of Computer Science and Engineering, University College of Engineering Villupuram, Tamilnadu, India
    Lecturer, Department of Computer Science and Engineering, University College of Engineering Villupuram, Tamilnadu, India
  • Arjun Paramarthalingam , Arjun Paramarthalingam
    Assistant Professor, Department of Computer Science and Engineering, University College of Engineering Villupuram, Tamilnadu, India
    Assistant Professor, Department of Computer Science and Engineering, University College of Engineering Villupuram, Tamilnadu, India
  • Arulnancy Thirunavukkarasu , Arulnancy Thirunavukkarasu
    Assistant Professor, Department of Electronics and Communication Engineering, VRS College of Engineering and Technology, Tamilnadu. India
    Assistant Professor, Department of Electronics and Communication Engineering, VRS College of Engineering and Technology, Tamilnadu. India
  • Regan Rajendran Regan Rajendran
    Assistant Professor, Department of Computer Science and Engineering, University College of Engineering Villupuram, Tamilnadu, India
    Assistant Professor, Department of Computer Science and Engineering, University College of Engineering Villupuram, Tamilnadu, India
view PDF
view PDF

How to Cite

Sivaraman, Jegan, Arjun Paramarthalingam, Arulnancy Thirunavukkarasu, and Regan Rajendran. 2026. “Encoder Selection and Segmentation Quality in U-Net-Based Retinal Vessel Analysis”. Journal of Artificial Intelligence and Capsule Networks 8 (4): 324-45. https://doi.org/10.36548/jaicn.2026.4.003.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

Retinal Vessel Segmentation
U-Net
Encoder Architectures
Deep Learning
Fundus Image Analysis
Transfer Learning
Medical Image Segmentation

Abstract

Retinal vessel segmentation through automation is a very significant area in retinal image processing, which can help in diagnosing many different ocular and systemic diseases. In this research study, a comparative study of the encoder-based U-Net models is presented for retinal vessel segmentation. Five different models including ResNet18+U-Net, ResNet34+U-Net, Traditional U-Net, VGG16+U-Net, and VGG19+U-Net have been developed and trained on the retinal images belonging to the DRIVE and CHASE_DB1 databases. The proposed methodology involves image pre-processing, image normalization, image augmentation, feature extraction from image using the encoder architecture, and U-Net decoding with the skip connection for pixel-wise segmentation of retinal vessels. Among all models evaluated in this paper, the VGG19+U-Net model gives the highest value of Jaccard score of 81.83%, F1 score of 89.53%, Precision of 89.47%, Accuracy of 98.41%, and Specificity of 99.13%, whereas the highest Sensitivity score of 90.32% is achieved by the VGG16+U-Net model.

References

  1. Rodrigues, Erick O., Aura Conci, and Panos Liatsis. "ELEMENT: Multi-modal Retinal Vessel Segmentation based on a Coupled Region Growing and Machine Learning Approach." IEEE Journal of Biomedical and Health Informatics 24, no. 12 (2020): 3507-3519.
  2. Mou, Lei, Li Chen, Jun Cheng, Zaiwang Gu, Yitian Zhao, and Jiang Liu. "Dense Dilated Network with Probability Regularized Walk for Vessel Detection." IEEE Transactions on Medical Imaging 39, no. 5 (2019): 1392-1403.
  3. Almotiri, Jasem, Khaled Elleithy, and Abdelrahman Elleithy. "Retinal Vessels Segmentation Techniques and Algorithms: a Survey." Applied Sciences 8, no. 2 (2018): 155.
  4. Liu, Wentao, Huihua Yang, Tong Tian, Zhiwei Cao, Xipeng Pan, Weijin Xu, Yang Jin, and Feng Gao. "Full-Resolution Network and Dual-Threshold Iteration for Retinal Vessel and Coronary Angiograph Segmentation." IEEE Journal of Biomedical and Health Informatics 26, no. 9 (2022): 4623-4634.
  5. D. Wang, A. Haytham, J. Pottenburgh, O. Saeedi and Y. Tao, "Hard Attention Net for Automatic Retinal Vessel Segmentation," in IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 12, pp. 3384-3396, Dec. 2020.
  6. Li, Kaiqi, Xingqun Qi, Yiwen Luo, Zeyi Yao, Xiaoguang Zhou, and Muyi Sun. "Accurate Retinal Vessel Segmentation in Color Fundus Images via Fully Attention-based Networks." IEEE Journal of Biomedical and Health Informatics 25, no. 6 (2020): 2071-2081.
  7. Tan, Yubo, Kai-Fu Yang, Shi-Xuan Zhao, and Yong-Jie Li. "Retinal Vessel Segmentation with Skeletal Prior and Contrastive Loss." IEEE Transactions on Medical Imaging 41, no. 9 (2022): 2238-2251.
  8. Cheung, Carol Yim-lui, M. Kamran Ikram, Christopher Chen, and Tien Yin Wong. "Imaging Retina to Study Dementia and Stroke." Progress in Retinal and Eye Research 57 (2017): 89-107.
  9. Fraser-Bell, Samantha, Richard Symes, and Anagha Vaze. "Hypertensive eye Disease: a Review." Clinical & Experimental Ophthalmology 45, no. 1 (2017): 45-53.
  10. Akil, Handan, Alex S. Huang, Brian A. Francis, Sirinivas R. Sadda, and Vikas Chopra. "Retinal Vessel Density from Optical Coherence Tomography Angiography to Differentiate Early Glaucoma, Pre-Perimetric Glaucoma and Normal Eyes." PloS one 12, no. 2 (2017): e0170476.
  11. L Srinidhi, Chetan, P. Aparna, and Jeny Rajan. "Recent Advancements in Retinal Vessel Segmentation." Journal of Medical Systems 41, no. 4 (2017): 70.
  12. Moccia, Sara, Elena De Momi, Sara El Hadji, and Leonardo S. Mattos. "Blood Vessel Segmentation Algorithms—Review of Methods, Datasets and Evaluation Metrics." Computer Methods and Programs in Biomedicine 158 (2018): 71-91.
  13. Singh, Nagendra Pratap, and Rajeev Srivastava. "Retinal Blood Vessels Segmentation by using Gumbel Probability Distribution Function based Matched Filter." Computer Methods and Programs in Biomedicine 129 (2016): 40-50.
  14. Karn, Prakash Kumar, Birendra Biswal, and Subhransu Ranjan Samantaray. "Robust Retinal Blood Vessel Segmentation using Hybrid Active Contour Model." IET Image Processing 13, no. 3 (2019): 440-450.
  15. Wang, Weihua, Jingzhong Zhang, Wenyuan Wu, and Shuang Zhou. "An Automatic Approach for Retinal Vessel Segmentation by Multi-scale Morphology and Seed Point Tracking." Journal of Medical Imaging and Health Informatics 8, no. 2 (2018): 262-274.
  16. Tuba, Eva, Lazar Mrkela, and Milan Tuba. "Retinal Blood Vessel Segmentation by Support Vector Machine Classification." In 2017 27th International Conference Radioelektronika (RADIOELEKTRONIKA), pp. 1-6. IEEE, 2017.
  17. Jin, Qiangguo, Zhaopeng Meng, Tuan D. Pham, Qi Chen, Leyi Wei, and Ran Su. "DUNet: A Deformable Network for Retinal Vessel Segmentation." Knowledge-based systems 178 (2019): 149-162.
  18. Sivaraman, Jegan, Arjun Paramarthalingam, Arulnancy Thirunavukkarasu, Asokan Vasudevan, Soon Eu Hui, and Duraimurugan Samiayya. "Optimized U-Net Model for Precise Retinal Blood Vessel Segmentation from Colour Fundus Images." Scientific Reports (2026).
  19. Guo, Song, Kai Wang, Hong Kang, Yujun Zhang, Yingqi Gao, and Tao Li. "BTS-DSN: Deeply Supervised Neural Network with Short Connections for Retinal Vessel Segmentation." International journal of Medical Informatics 126 (2019): 105-113.
  20. Feng, Shouting, Zhongshuo Zhuo, Daru Pan, and Qi Tian. "CcNet: A Cross-Connected Convolutional Network for Segmenting Retinal Vessels using Multi-Scale Features." Neurocomputing 392 (2020): 268-276.
  21. Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully Convolutional Networks for Semantic Segmentation." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431-3440. 2015.
  22. Oliveira, Américo, Sergio Pereira, and Carlos A. Silva. "Retinal Vessel Segmentation based on Fully Convolutional Neural Networks." Expert Systems with Applications 112 (2018): 229-242.
  23. Larxel. “DRIVE Digital Retinal Images for Vessel Extraction.” Kaggle, 2019. https://www.kaggle.com/datasets/andrewmvd/drive-digital-retinal-images-for-vessel-extraction.
  24. Rasha Sarhan Alharthi, “Chase-DB1,” Kaggle, Accessed September 18, 2026. https://chatgpt.com/c/6aacbd68-88ec-83ee-b22e-57654ae85333