Deep Learning and Ocular Image Analysis

Over the past decade, advancements in imaging of the eye has transformed research and clinical decision making. However, methods for image analysis are needed to automatically quantify these medical images and extract useful clinical metrics.

Deep learning (DL) methods, a subfield of artificial intelligence, are techniques that can be used to tackle image analysis problems, which can otherwise be time-consuming and subjective if done manually. A deep learning network contains multiple layers of filters used to progressively extract features at different levels of abstraction/resolution from the input images. The networks are trained in an initial “learning phase” before the method is applied to a task. Over the past ten years, DL methods have become state-of-the-art for to the automatic segmentation and classification of medical images, allowing us to derive rapid and accurate diagnostic information about eye health and morphology.

Our research team applies DL to optical coherence tomography (OCT) imaging which is an imaging modality that provides cross-sectional images of the eye’s tissue structure at a micron scale.  We have developed several image analysis tasks, including segmentation, classification, anomaly detection and synthetic image generation. Segmentation of the layers at the front and back of the eye is commonly performed in the domains of optometry and ophthalmology for both clinical and research purposes. In particular, the quantification of layer thicknesses and the area of lesions are critical metrics for the diagnosis and monitoring of ocular diseases, such as age-related macular degeneration and diabetic retinopathy.

Our OCT image analysis methods allow us to quantify the image and avoid the need for manual segmentation by an expert grader, which often is not feasible due to the significant time requirement and not desirable due to inter-grader variability. Our DL tools have been proven to provide reliable image analysis methods for medical images, thus having a real-world impact in both research and clinical decision making.

Generative adversarial networks (GANs) are a deep learning method used to generate new and realistic images. Their most well-known application is the generation of lifelike photos of people’s faces. An original data set (red) can be augmented, creating additional synthetic images generated by the GAN (blue), and this method is particularly useful when there is limited data.


OCT scans of the posterior eye showing the segmented retina (blue) and choroid (red) for Stargardt disease, an ocular disease affecting the retina.


A graphical example of two OCT images showing probability maps for the raw OCT image, showing ground truth (G.T.), and segmentation outputs when using Mask R-CNN.


OCT scans of the anterior eye showing the segmented corneal layers; corneal epithelium (EP), Bowman’s layer (BL) and endothelium (EN) achieved from patch-based convolutional neural network (CNN) training, a deep learning method.


Recommended Further Readings:

Hamwood, Jared, Schmutz, Beat, Collins, Michael, Allenby, Mark, & Alonso Caneiro, David (2021) A deep learning method for automatic segmentation of the bony orbit in MRI and CT images. Scientific Reports, 11(1), Article number: 13693.  []

Garcia Marin, Yoel, Skrok, Marta, Siedlecki, Damian, Vincent, Stephen J., Collins, Michael J., & Alonso-Caneiro, David (2021) Segmentation of anterior segment boundaries in swept source OCT images. Biocybernetics and Biomedical Engineering, 41(3), pp. 903-915. []

Kugelman, Jason, Alonso-Caneiro, David, Read, Scott A., Vincent, Stephen J., Chen, Fred K., & Collins, Michael J. (2021) Data augmentation for patch-based OCT chorio-retinal segmentation using generative adversarial networks. Neural Computing and Applications, 33(13), 7393–7408. []

Mehdizadeh, Maryam, Macnish, Cara, Xiao, Di, Alonso-Caneiro, David, Kugelman, Jason, & Bennamoun, Mohammed (2021) Deep feature loss to denoise OCT images using deep neural networks. Journal of Biomedical Optics, 26(4), Article number: 046003. []

Kugelman, Jason, Alonso-Caneiro, David, Read, Scott A., Vincent, Stephen J., Chen, Fred K., & Collins, Michael J. (2020) Dual image and mask synthesis with GANs for semantic segmentation in optical coherence tomography. In Proceedings of the 2020 Digital Image Computing: Techniques and Applications (DICTA). Institute of Electrical and Electronics Engineers Inc., United States of America. []

Kugelman, Jason, Alonso-Caneiro, David, Chen, Yi, Arunachalam, Sukanya, Huang, Di, Vallis, Natasha, et al. (2020) Retinal boundary segmentation in stargardt disease optical coherence tomography images using automated deep learning. Translational Vision Science and Technology, 9(11), Article number: 12. []