Generic placeholder image

Current Medical Imaging

Author(s): Mazhar Basyouni Tayel and Azza Mahmoud Elbagoury*

DOI: 10.2174/1573405615666190503142031

DownloadDownload PDF Flyer Cite As
Breast Infrared Thermography Segmentation Based on Adaptive Tuning of a Fully Convolutional Network

Page: [611 - 621] Pages: 11

  • * (Excluding Mailing and Handling)

Abstract

Background: Accurate segmentation of Breast Infrared Thermography is an important step for early detection of breast pathological changes. Automatic segmentation of Breast Infrared Thermography is a very challenging task, as it is difficult to find an accurate breast contour and extract regions of interest from it. Although several semi-automatic methods have been proposed for segmentation, their performance often depends on hand-crafted image features, as well as preprocessing operations.

Objectives: In this work, an approach to automatic semantic segmentation of the Breast Infrared Thermography is proposed based on end-to-end fully convolutional neural networks and without any pre or post-processing.

Methods: The lack of labeled Breast Infrared Thermography data limits the complete utilization of fully convolutional neural networks. The proposed model overcomes this challenge by applying data augmentation and two-tier transfer learning from bigger datasets combined with adaptive multi-tier fine-tuning before training the fully convolutional neural networks model.

Results: Experimental results show that the proposed approach achieves better segmentation results: 97.986% accuracy; 98.36% sensitivity and 97.61% specificity compared to hand-crafted segmentation methods.

Conclusion: This work provided an end-to-end automatic semantic segmentation of Breast Infrared Thermography combined with fully convolutional networks, adaptive multi-tier fine-tuning and transfer learning. Also, this work was able to deal with challenges in applying convolutional neural networks on such data and achieving the state-of-the-art accuracy.

Keywords: AlexNet, breast infrared thermography, fully convolutional networks, fine-tuning, semantic segmentation, transfer learning.

Graphical Abstract

[1]
Siegel RL, Miller KD, Jemal A. Cancer statistics. Cancer J Clin 2018; 4; 68(1): 7-30.
[2]
Resmini R, Conci A, da Silva LF, et al. Application of Infrared Images to Diagnosis and Modeling of Breast. Singapore: Appl Infrared to Biomed Sci Springer 2017; pp. 159-73.
[http://dx.doi.org/10.1007/978-981-10-3147-2_10]
[3]
Etehadtavakol M, Ng EY. An overview of medical infrared imaging in breast abnormalities detection. Singapore. Appl Infrared to Biomed Sci Springer 2017; 2017: 45-57.
[http://dx.doi.org/10.1007/978-981-10-3147-2_4]
[4]
Gogoi UR, Majumdar G, Bhowmik MK, Ghosh AK, Bhattacharjee D. Breast abnormality detection through statistical feature analysis using infrared thermograms. In: International Symposium on Advanced Computing and Communication (ISACC). Silchar, India, New Jersey, IEEE 2015; pp. 258-65.
[http://dx.doi.org/10.1109/ISACC.2015.7377351]
[5]
Tavakol ME, Ng-KE Lucas C, Sadri S. In: Suri JS, Ed. . Diagnostic and therapeutic applications of breast imaging.. Bellingham: SPIE 2012; pp. 373-98.
[6]
Zadeh HG, Masoumzadeh S, Nour S, et al. Breast cancer diagnosis by thermal imaging in the fields of medical and artificial intelligence sciences: review article. TUMJ 2016; 74(6): 377-85.
[7]
Borchartt TB, Conci A, Lima RC, Resmini R, Sanchez A. Breast thermography from an image processing viewpoint: A survey. Sci Prog 2013; 93(10): 2785-803.
[8]
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012; 1097-105.
[9]
Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88.
[http://dx.doi.org/10.1016/j.media.2017.07.005] [PMID: 28778026]
[10]
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition 2014. arXiv preprint arXiv:1409.1556.
[11]
Szegedy C, Liu W, Jia Y, Sermanet P, et al. Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA, New Jersey: IEEE 2015; pp. 1-9.
[12]
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Conference on Computer Vision and Pattern Recognition CVPR. Boston, MA, USA, New Jersey: IEEE 2015; pp. 3431-40.
[13]
Garcia-Garcia A, Orts Escolano, et al. A review on deep learning techniques applied to semantic segmentation 2017. arXiv preprint arXiv:1704.06857.
[14]
Russakovsky O, Deng J, Su H, Krause J, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis 2015; 115(3): 211-52.
[http://dx.doi.org/10.1007/s11263-015-0816-y]
[15]
Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In: International Conference on Computer Vision (ICCV). Santiago, Chile; New Jersey: IEEE 2015; pp. 1520-8.
[16]
Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. In: 13th European conference on computer vision (ECCV). Zurich, Switzerland, Berlin: Springer 2014; pp. 818-33.
[17]
Guo Y, Liu Y, Oerlemans A, et al. Deep learning for visual understanding: A review. Neurocomputing 2016; 187: 27-48.
[http://dx.doi.org/10.1016/j.neucom.2015.09.116]
[18]
Sun C. Shrivastava, et al. Revisiting unreasonable effectiveness of data in deep learning era. In: International Conference on Computer Vision (ICCV). Venice, Italy, New Jersey: IEEE 2017; pp. 843-52.
[19]
Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Miami, FL, USA, New Jersey: IEEE 2009; pp. 248-55.
[20]
Shen D, Wu G, Suk HI. Heung-Il Suk. Deep learning in medical image analysis. Annu Rev Biomed Eng 2017; 19: 221-48.
[http://dx.doi.org/10.1146/annurev-bioeng-071516-044442] [PMID: 28301734]
[21]
Wong SC, Gatt A, Stamatescu V, McDonnell MD. Understanding data augmentation for classification: when to warp? In: International Conference on Digital Image Computing: Techniques and Applications (DICTA). Gold Coast, QLD, Australia, New Jersey: IEEE 2016; pp. 1-6.
[http://dx.doi.org/10.1109/DICTA.2016.7797091]
[22]
Flusser J, Suk T. Pattern recognition by affine moment invariants. Pattern Recognit 1993; 26(1): 167-74.
[http://dx.doi.org/10.1016/0031-3203(93)90098-H]
[23]
Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? Adv Neural Inf Process Syst 2014; 3320-8.
[24]
Long M, Cao Y, Cao Z, Wang J, Jordan MI. Transferable Representation Learning with deep adaptation networks. IEEE 2019; 41(12): 3071-85.
[25]
Erhan D, Bengio Y, Courville A, Manzagol PA, Vincent P, Bengio S. Why does unsupervised pre-training help deep learning? J Mach Learn Res 2010; 2010: 625-60.
[26]
Brostow GJ, Fauqueur J, Cipolla R. Semantic object classes in video: A high-definition ground truth database. Pattern Recognit Lett 2009; 30(2): 88-97.
[http://dx.doi.org/10.1016/j.patrec.2008.04.005]
[27]
Object Recognition in Video Dataset [homepage on the Internet] [Cited 15 October 2018]. Available from:. http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/
[28]
Silva LF, Saade DC, Sequeiros GO, et al. A new database for breast research with infrared image. J Med Imaging Health Inform 2014; 4(1): 92-100.
[http://dx.doi.org/10.1166/jmihi.2014.1226]
[29]
Visual Lab, Database of infrared images and segmentations done by specialists [homepage on the Internet] [Cited January 2018]. Available from:. http://visual.ic.uff.br/en/proeng/software.php
[30]
Digital Infrared Thermal Imaging [homepage on the Internet] [Cited 3 April 2018]. Available from:. http://www.thermography.co.in/index.html
[31]
Farabet C, Couprie C, Najman L, Lecun Y. Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 2013; 35(8): 1915-29.
[http://dx.doi.org/10.1109/TPAMI.2012.231] [PMID: 23787344]
[32]
Menegola A, Fornaciali M, Pires R, Avila S, Valle E. Towards automated melanoma screening: Exploring transfer learning schemes 2016. arXiv preprint arXiv:1609.01228.
[33]
Yap MH, Pons G, Martí J, et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform 2018; 22(4): 1218-26.
[http://dx.doi.org/10.1109/JBHI.2017.2731873] [PMID: 28796627]
[34]
Bengio Y. Practical recommendations for gradient-based training of deep architectures Neural Netw Trick Trade 2012; 2012: 437-78.
[http://dx.doi.org/10.1007/978-3-642-35289-8_26]
[35]
Baffa MF, Cheloni DJ, Lattari LG, Coelho MA. Segmentação Automática de Mamas em Imagens Infravermelhas Utilizando Limiarização com Refinamento Adaptativo em Bases Multivariadas. Revista de Informática Aplicada 2017; 12(2): 1-11.
[36]
Csurka G, Larlus D, Perronnin F, Meylan F. What is a good evaluation measure for semantic segmentation? In: Proceedings of the 24th British Machine Vision Conference (BMVC). Bristol, United Kingdom, BMVA Press. 32.1-32.11.
[http://dx.doi.org/10.5244/C.27.32]
[37]
Marques RS, Resmini R, Conci A, Fontes CAP, Lima RCF. Método para segmentação manual de imagens térmicas para geração de Ground Truth. In: Proceedings of the XII Workshop em Informática Médica. Curitiba, PR, Brasil 2012; pp. 1-9.