Colored Video Analysis in Wireless Capsule Endoscopy: A Survey of State-of-the-Art

Amira    S.    Ashour; Nilanjan       Dey; Waleed    S.    Mohamed; Jolanda    G.    Tromp; R.    Simon    Sherratt; Fuqian       Shi; Luminița       Moraru

Abstract

Wireless Capsule Endoscopy (WCE) is a highly promising technology for gastrointestinal (GI) tract abnormality diagnosis. However, low image resolution and low frame rates are challenging issues in WCE. In addition, the relevant frames containing the features of interest for accurate diagnosis only constitute 1% of the complete video information. For these reasons, analyzing the WCE videos is still a time consuming and laborious examination for the gastroenterologists, which reduces WCE system usability. This leads to the emergent need to speed-up and automates the WCE video process for GI tract examinations. Consequently, the present work introduced the concept of WCE technology, including the structure of WCE systems, with a focus on the medical endoscopy video capturing process using image sensors. It discussed also the significant characteristics of the different GI tract for effective feature extraction. Furthermore, video approaches for bleeding and lesion detection in the WCE video were reported with computer-aided diagnosis systems in different applications to support the gastroenterologist in the WCE video analysis. In image enhancement, WCE video review time reduction is also discussed, while reporting the challenges and future perspectives, including the new trend to employ the deep learning models for feature Learning, polyp recognition, and classification, as a new opportunity for researchers to develop future WCE video analysis techniques.

Keywords: Endoscopy capsule, video analysis, bleeding detection, reviewing time reduction, wireless video gastrointestinal (GI) endoscopy capsule, computer- aided diagnosis.

Graphical Abstract

[1] 
Xin W, Yan G, Wang W. Study of a wireless power transmission system for an active capsule endoscope. Int J Med Robot  2010; 6(1): 113-22.
[http://dx.doi.org/10.1002/rcs.298] [PMID:  20112281] 
[2] 
Iddan G, Meron G, Glukhovsky A, Swain P. Wireless capsule endoscopy. Nature  2000; 405(6785): 417.
[http://dx.doi.org/10.1038/35013140] [PMID:  10839527] 
[3] 
Wang A, Banerjee S, Barth BA, et al. Wireless capsule endoscopy. Gastrointestinal endoscopy   2013; 78(6): 805-15.
[4] 
Triester SL, Leighton JA, Leontiadis GI, et al. A meta-analysis of the yield of capsule endoscopy compared to other diagnostic modalities in patients with obscure gastrointestinal bleeding. Am J Gastroenterol  2005; 100(11): 2407.
[PMID: 16279893] 
[5] 
Mylonaki M, Fritscher-Ravens A, Swain P. Wireless capsule endoscopy: A comparison with push enteroscopy in patients with gastroscopy and colonoscopy negative gastrointestinal bleeding. Gut  2003; 52(8): 1122-6.
[http://dx.doi.org/10.1136/gut.52.8.1122] [PMID:  12865269] 
[6] 
Moglia A, Menciassi A, Dario P. Recent patents on wireless capsule endoscopy. Recent Pat Biomed Eng  2008; 1(1): 24-33.
[http://dx.doi.org/10.2174/1874764710801010024] 
[7] 
Liedlgruber M, Uhl A. Computer-aided decision support systems for endoscopy in the gastrointestinal tract: A review. IEEE Rev Biomed Eng  2011; 4: 73-88.
[http://dx.doi.org/10.1109/RBME.2011.2175445] [PMID:  22273792] 
[8] 
Eliakim R, Fischer D, Suissa A, et al. Wireless capsule video endoscopy is a superior diagnostic tool in comparison to barium follow-through and computerized tomography in patients with suspected Crohn’s disease. Eur J Gastroenterol Hepatol  2003; 15(4): 363-7.
[http://dx.doi.org/10.1097/00042737-200304000-00005] [PMID:  12655255] 
[9] 
Dey N, Ashour AS, Shi F, Sherratt RS. Wireless capsule gastrointestinal endoscopy: Direction-of-arrival estimation based localization survey. IEEE Rev Biomed Eng  2017; 10: 2-11.
[http://dx.doi.org/10.1109/RBME.2017.2697950] [PMID:  28459696] 
[10] 
Ciuti G, Caliò R, Camboni D, et al. Frontiers of robotic endoscopic capsules: a review. J Microbio Robot  2016; 11(1): 1-18.
[http://dx.doi.org/10.1007/s12213-016-0087-x] [PMID:  29082124] 
[11] 
Koulaouzidis A, Dabos KJ. Looking forwards: not necessarily the best in capsule endoscopy? Ann Gastroenterol  2013; 26(4): 365-7.
[PMID: 24714324] 
[12] 
Zhou M. On the accuracy of wireless capsule endoscope RF and visual localization Doctoral thesis, Worcester Polytechnic Institute,USA  2015.
[13] 
Iakovidis DK, Maroulis DE, Karkanis SA. An intelligent system for automatic detection of gastrointestinal adenomas in video endoscopy. Comput Biol Med  2006; 36(10): 1084-103.
[http://dx.doi.org/10.1016/j.compbiomed.2005.09.008] [PMID:  16293240] 
[14] 
Szeliski R. Computer vision: Algorithms and applications. Springer Science & Business Media [2010 Sep 30]. 
[15] 
Ojala T, Pietikäinen M, Mäenpää T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell  2002; (7): 971-87.
[http://dx.doi.org/10.1109/TPAMI.2002.1017623] 
[16] 
Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vis  2004; 60(2): 91-110.
[http://dx.doi.org/10.1023/B:VISI.0000029664.99615.94] 
[17] 
Dalal N, Triggs B. Histograms of oriented gradients for human detection. Proc CVPR San Diego.  CA, USA. 2005; pp. 2005 vol . 1. 886-93.
[http://dx.doi.org/10.1109/CVPR.2005.177] 
[18] 
Berens J, Finlayson GD, Qiu G. Image indexing using compressed colour histograms. IEE Proc Vis Image Signal Process  2000; 147(4): 349-55.
[http://dx.doi.org/10.1049/ip-vis:20000630] 
[19] 
Mackiewicz M, Berens J, Fisher M. Wireless capsule endoscopy color video segmentation. IEEE Trans Med Imaging  2008; 27(12): 1769-81.
[http://dx.doi.org/10.1109/TMI.2008.926061] [PMID:  19033093] 
[20] 
Gong Y, Chuan CH, Xiaoyi G. Image indexing and retrieval based on color histograms. Multimedia Tools Appl  1996; 2(2): 133-56.
[http://dx.doi.org/10.1007/BF00672252] 
[21] 
Manjunath BS, Ohm JR, Vasudevan VV, Yamada A. Color and texture descriptors. IEEE Trans Circ Syst Video Tech  2001; 11(6): 703-15.
[http://dx.doi.org/10.1109/76.927424] 
[22] 
Seguí S, Drozdzal M, Pascual G, et al. Deep learning features for wireless capsule endoscopy analysis.In Iberoamerican Congress on Pattern Recognition 2016 Nov 8; 326-33.. 
[23] 
Connah D, Finlayson GD. Using local binary pattern operators for colour constant image indexing. Conference on Colour in Graphics, Imaging, and Vision 2006 Jan 1 2006; 1: 60-4.. 
[24] 
Ojala T, Pietikäinen M, Mäenpää T. Gray scale and rotation invariant texture classification with local binary patterns. European Conference on Computer Vision. 2000 Jun 26; 404-20.. 
[http://dx.doi.org/10.1007/3-540-45054-8_27] 
[25] 
Lee J, Oh J, Shah SK, Yuan X, Tang SJ. Automatic classification of digestive organs in wireless capsule endoscopy videos 2007.
[http://dx.doi.org/10.1145/1244002.1244230] 
[26] 
Fisher L, Krinsky ML, Anderson MA, et al. The role of endoscopy in the management of obscure GI bleeding. Gastrointestinal endosc opy  2010; 72(3): 471-9.
[http://dx.doi.org/10.1016/j.gie.2010.04.032] 
[27] 
Dey N, Ashour A. Classification and clustering in biomedical signal processing. Hershey: IGI global 2016.
[28] 
AlShahrani AM, Al-Abadi MA, Al-Malki AS, Ashour AS, Dey N. Automated system for crops recognition and classification.In Computer Vision: Concepts,Methodologies, Tools, and Applications  2018; 1208-23.
[29] 
Saba L, Dey N, Ashour AS, et al. Automated stratification of liver disease in ultrasound: An online accurate feature classification paradigm. Comput Methods Programs Biomed  2016; 130: 118-34.
[http://dx.doi.org/10.1016/j.cmpb.2016.03.016] [PMID:  27208527] 
[30] 
Ahmed SS, Dey N, Ashour AS, et al. Effect of fuzzy partitioning in Crohn’s disease classification: A neuro-fuzzy-based approach. Medical & biological engineering & computing  2017.
[http://dx.doi.org/10.1007/s11517-016-1508-7] 
[31] 
Virmani J, Dey N, Kumar V. PCA-PNN and PCA-SVM based CAD systems for breast density classification Applications of intelligent optimization in biology and medicine 2016; 159-80.. 
[32] 
Wang P, Krishnan SM, Kugean C, Tjoa MP. Classification of endoscopic images based on texture and neural network 2001.
[http://dx.doi.org/10.1109/IEMBS.2001.1019637] 
[33] 
Figueiredo IN, Prasath S, Tsai YH, Figueiredo PN. Automatic detection and segmentation of colonic polyps in wireless capsule images ICES REPORT 2010 Sep 22; 10-36. 
[34] 
Karargyris A, Bourbakis N. Detection of small bowel polyps and ulcers in wireless capsule endoscopy videos. IEEE Trans Biomed Eng  2011; 58(10): 2777-86.
[http://dx.doi.org/10.1109/TBME.2011.2155064] [PMID:  21592915] 
[35] 
Zhao Q, Meng MQ. Polyp detection in wireless capsule endoscopy images using novel color texture features. 2011 9th World Congress on Intelligent Control and Automation 2011 Jun 21; 948-52.. 
[36] 
Bourbakis N, Makrogiannis S, Kavraki D. A neural network-based detection of bleeding in sequences of WCE images 2005.
[http://dx.doi.org/10.1109/BIBE.2005.6] 
[37] 
Jung YS, Kim YH, Lee DH, Kim JH. In: Active blood detection in a high resolution capsule endoscopy using color spectrum transformation, 2008: International conference on biomedical engineering informatics, IEEE; Sanya, China.. 
[http://dx.doi.org/10.1109/BMEI.2008.216] 
[38] 
Li B, Meng MQ. Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments. Comput Biol Med  2009; 39(2): 141-7.
[http://dx.doi.org/10.1016/j.compbiomed.2008.11.007] [PMID:  19147126] 
[39] 
Li B, Meng MQ. Small bowel tumor detection for wireless capsule endoscopy images using textural features and support vector machine.	2009 IEEE/RSJ International Conference on Intelligent Robots and Systems 2009.. 
[http://dx.doi.org/10.1109/IROS.2009.5354726] 
[40] 
Yu L, Yuen PC, Lai J. Ulcer detection in wireless capsule endoscopy images. Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).  2012 Nov 11; . 45-8.
[41] 
Yuan Y, Meng MQ. Deep learning for polyp recognition in wireless capsule endoscopy images. Med Phys  2017; 44(4): 1379-89.
[http://dx.doi.org/10.1002/mp.12147] [PMID:  28160514] 
[42] 
Kumar R, Zhao Q, Seshamani S, Mullin G, Hager G, Dassopoulos T. Assessment of Crohn’s disease lesions in wireless capsule endoscopy images. IEEE Trans Biomed Eng  2012; 59(2): 355-62.
[http://dx.doi.org/10.1109/TBME.2011.2172438] [PMID:  22020661] 
[43] 
Sekuboyina AK, Devarakonda ST, Seelamantula CS. A convolutional neural network approach for abnormality detection in wireless capsule endoscopy 2017.
[http://dx.doi.org/10.1109/ISBI.2017.7950698] 
[44] 
Zou Y, Li L, Wang Y, Yu J, Li Y, Deng WJ. Classifying digestive organs in wireless capsule endoscopy images based on deep convolutional neural network. 2015 IEEE International Conference on Digital Signal Processing (DSP) 2015 .  1274-8.
[http://dx.doi.org/10.1109/ICDSP.2015.7252086] 
[45] 
Cunha JS, Coimbra M, Campos P, Soares JM. Automated topographic segmentation and transit time estimation in endoscopic capsule exams. IEEE Trans Med Imaging  2008; 27(1): 19-27.
[http://dx.doi.org/10.1109/TMI.2007.901430] [PMID:  18270058] 
[46] 
Vilarino F, Spyridonos P, Deiorio F, Vitria J, Azpiroz F, Radeva P. Intestinal motility assessment with video capsule endoscopy: Automatic annotation of phasic intestinal contractions. IEEE Trans Med Imaging  2010; 29(2): 246-59.
[http://dx.doi.org/10.1109/TMI.2009.2020753] [PMID:  19423434] 
[47] 
Yagi Y, Vu H, Echigo T, et al. A diagnosis support system for capsule endoscopy. Inflammopharmacology  2007; 15(2): 78-83.
[http://dx.doi.org/10.1007/s10787-006-0010-5] [PMID:  17450447] 
[48] 
Szczypiński PM, Sriram RD, Sriram PV, Reddy DN. A model of deformable rings for interpretation of wireless capsule endoscopic videos. Med Image Anal  2009; 13(2): 312-24.
[http://dx.doi.org/10.1016/j.media.2008.12.002] [PMID:  19157954] 
[49] 
Appleyard M, Fireman Z, Glukhovsky A, et al. A randomized trial comparing wireless capsule endoscopy with push enteroscopy for the detection of small-bowel lesions. Gastroenterology  2000; 119(6): 1431-8.
[http://dx.doi.org/10.1053/gast.2000.20844] [PMID:  11113063] 
[50] 
de Iorio F, Radeva P, Vitria J, Pujol O, Spyridonos P, Vilarino F. Automatic detection of intestinal juices in wireless capsule video endoscopy. 18th International Conference on Pattern Recognition (ICPR’06).   2006 Aug 20; 4. 719-22.
[51] 
Bashar MK, Kitasaka T, Suenaga Y, Mekada Y, Mori K. Automatic detection of informative frames from wireless capsule endoscopy images. Med Image Anal  2010; 14(3): 449-70.
[http://dx.doi.org/10.1016/j.media.2009.12.001] [PMID:  20137998] 
[52] 
Wang C, Luo Z, Liu X, Bai J, Liao G. Detection of protruding lesion in wireless capsule endoscopy videos of small intestine. Medical Imaging 2018. Computer-Aided Diagnosis  2018;  Feb 27;. 105751057513
[53] 
Vu H, Echigo T, Sagawa R, et al. Contraction detection in small bowel from an image sequence of wireless capsule endoscopy 2007.
[http://dx.doi.org/10.1007/978-3-540-75757-3_94] 
[54] 
Vu H, Echigo T, Sagawa R, et al. Detection of contractions in adaptive transit time of the small bowel from wireless capsule endoscopy videos. Comput Biol Med  2009; 39(1): 16-26.
[http://dx.doi.org/10.1016/j.compbiomed.2008.10.005] [PMID:  19061993] 
[55] 
Malagelada C, De Iorio F, Azpiroz F, et al. New insight into intestinal motor function via noninvasive endoluminal image analysis. Gastroenterology  2008; 135(4): 1155-62.
[http://dx.doi.org/10.1053/j.gastro.2008.06.084] [PMID:  18691579] 
[56] 
Spyridonos P, Vilariño F, Vitrià J, Azpiroz F, Radeva P. Anisotropic feature extraction from endoluminal images for detection of intestinal contractions. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin,Heidelberg: Springer 2006; pp. 161-8.. 
[http://dx.doi.org/10.1007/11866763_20] 
[57] 
Vilariño F, Spyridonos P, Vitrià J, Malagelada C, Radeva P. Linear radial patterns characterization for automatic detection of tonic intestinal contractions.In: Iberoamerican Congress on Pattern Recognition.  Berlin, Heidelberg: Springer 2006; pp. 178-87.
[http://dx.doi.org/10.1007/11866763_20] 
[58] 
Drozdzal M, Seguí S, Vitrià J, Malagelada C, Azpiroz F, Radeva P. Adaptable image cuts for motility inspection using WCE. Comput Med Imaging Graph  2013; 37(1): 72-80.
[http://dx.doi.org/10.1016/j.compmedimag.2012.09.002] [PMID:  23098835] 
[59] 
Szczypinski PM, Sriram PV, Sriram RD, Reddy D. Computerized
	image analysis of wireless capsule endoscopy videos using a dedicated
	web-like model of deformable rings-a feasibility study. 12th
	United European Gastroenterology Week, Prague 2004 Sep;
	36(Suppl I): A76.. 
[60] 
Okuhata H, Nakamura H, Hara S, Tsutsui H, Onoye T. Application of the real-time Retinex image enhancement for endoscopic images 2013.
[http://dx.doi.org/10.1109/EMBC.2013.6610273] 
[61] 
Ramaraj M, Raghavan S, Khan WA. Homomorphic filtering techniques for WCE image enhancement 2013.
[http://dx.doi.org/10.1109/ICCIC.2013.6724282] 
[62] 
Li B, Meng MQ. Wireless capsule endoscopy images enhancement via adaptive contrast diffusion. J Vis Commun Image Represent  2012; 23(1): 222-8.
[http://dx.doi.org/10.1016/j.jvcir.2011.10.002] 
[63] 
Gopi VP, Palanisamy P. Capsule endoscopic image denoising based on double density dual tree complex wavelet transform. Int J Imaging Robot  2013; 9(1): 48-60.
[64] 
Karargyris A, Bourbakis N. An elastic video interpolation methodology for wireless capsule endoscopy videos 2010.
[http://dx.doi.org/10.1109/BIBE.2010.16] 
[65] 
Häfner M, Liedlgruber M, Uhl A. POCS-based super-resolution for HD endoscopy video frames. Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems 2013 Jun 20;.  185-90.
[66] 
Hai V, Echigo T, Sagawa R, et al. Adaptive control of video display for diagnostic assistance by analysis of capsule endoscopic images. 18th International Conference on Pattern Recognition (ICPR’06).  2006 Aug 20; 3: . 980-3.
[67] 
Lewis BS, Swain P. Capsule endoscopy in the evaluation of patients with suspected small intestinal bleeding: Results of a pilot study. Gastrointest Endosc  2002; 56(3): 349-53.
[http://dx.doi.org/10.1016/S0016-5107(02)70037-0] [PMID:  12196771] 
[68] 
Chu X, Poh CK, Li L, et al. Epitomized summarization of wireless capsule endoscopic videos for efficient visualization 2010.
[http://dx.doi.org/10.1007/978-3-642-15745-5_64] 
[69] 
Szeliski R. Image alignment and stitching: A tutorial Foundations and Trends® in Computer Graphics and Vision 2007 Jan 2; 2(1): 1-4.. 
[70] 
Jia X, Meng MQ. A deep convolutional neural network for bleeding detection in wireless capsule endoscopy images 2016.
[http://dx.doi.org/10.1109/EMBC.2016.7590783] 
[71] 
Li B, Meng MQ. Texture analysis for ulcer detection in capsule endoscopy images. Image Vis Comput  2009; 27(9): 1336-42.
[http://dx.doi.org/10.1016/j.imavis.2008.12.003] 
[72] 
Charisis VS, Katsimerou C, Hadjileontiadis LJ, Liatsos CN, Sergiadis GD. Computer-aided capsule endoscopy images evaluation based on color rotation and texture features: An educational tool to physicians. Proceedings of the 26th IEEE International Symposium On Computer-Based Medical Systems.  2013 Jun 20. 203-8.
[73] 
Boulougoura M, Wadge E, Kodogiannis V, Chowdrey HS. Intelligent systems for computer-assisted clinical endoscopic image analysis. Second International Conference on Biomedical Engineering 2004.
[74] 
Spyridonos P, Vilariño F, Vitria J, Radeva P. Identification of intestinal motility events of capsule endoscopy video analysis 2005.
[http://dx.doi.org/10.1007/11558484_67] 
[75] 
Dey N. Uneven illumination correction of digital images: A survey of the state-of-the-art. Optik (Stuttg)  2019; 183: 483-95.
[http://dx.doi.org/10.1016/j.ijleo.2019.02.118] 
[76] 
Yuan Y, Meng MQ. Hierarchical key frames extraction for WCE video 2013.
[http://dx.doi.org/10.1109/ICMA.2013.6617922] 
[77] 
Kwack WG, Lim YJ. Current status and research into overcoming limitations of capsule endoscopy. Clin Endosc  2016; 49(1): 8-15.
[http://dx.doi.org/10.5946/ce.2016.49.1.8] [PMID:  26855917] 
[78] 
Razmjooy N, Mousavi BS, Soleymani F. A hybrid neural network Imperialist Competitive Algorithm for skin color segmentation. Math Comput Model  2013; 57(3-4): 848-56.
[http://dx.doi.org/10.1016/j.mcm.2012.09.013] 
[79] 
Razmjooy N, Sheykhahmad FR, Ghadimi N. A hybrid neural network–world cup optimization algorithm for melanoma detection. Open Med (Wars)  2018; 13(1): 9-16.
[http://dx.doi.org/10.1515/med-2018-0002] [PMID:  29577090] 
[80] 
Mirjalili S, Hashim SZ, Sardroudi HM. Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput  2012; 218(22): 11125-37.
[http://dx.doi.org/10.1016/j.amc.2012.04.069] 
[81] 
Toth E, Marthinsen L, Bergström M, et al. Colonic obstruction caused by video capsule entrapment in a metal stent. Ann Transl Med  2017; 5(9): 199.
[http://dx.doi.org/10.21037/atm.2017.03.79] [PMID:  28567379] 

Cite As

Current Medical Imaging

Colored Video Analysis in Wireless Capsule Endoscopy: A Survey of State-of-the-Art

Abstract

Graphical Abstract