On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule Endoscopy
V. B. Surya Prasath
Computational Imaging and Visualization Analysis Lab Department of Computer Science
University of Missouri-Columbia Columbia MO 65211 USA
Abstract
Advances in image and video processing algorithms and the availability of computational resources has paved the way for real-time medical decision support sys- tems to be a reality. Video capsule endoscopy is a di- rect imaging method for gastrointestinal regions and produces large scale color video data. Fuzzification of color spaces can improve contextual description based tasks that are required in medical decision support. We consider abnormalities detection in video capsule en- doscopy using fuzzy sets and logic theory on different colorspaces. Application in retrieval of bleeding detec- tion and polyp vascularization are given as examples of the methodology considered here and preliminary re- sults indicate we obtain promising retrieval results.
Introduction
Video capsule endoscopy (VCE) is a revolutionary imag- ing technique which paved the way for unprecedented di- rect visualization of the gastrointestinal tract without much discomfort to patients. A typical colon VCE exam produces around 8 hours of color (RGB) video data. For example, Pill- cam Colon capsule endoscope (Given Imaging, Yoqneam, Israel) produces approximately 30,000 frames per patient and more than 1.6 million patients worldwide have used capsule endoscopy over the past10years. Automatic algo- rithms can help augment computer aided diagnosis (CAD) in VCE medical decision support systems and can help reduce the burden on gastroenterologists (Rey 2008; Niwa et al.
2008). For example, polyp detection (Figueiredo et al. 2010;
2011), mucosa surface identification (Prasath, Figueiredo, and Figueiredo 2011; Prasath et al. 2012; Prasath and Del- hibabu 2015b), contrast enhancement (Prasath and Del- hibabu 2015a). Nevertheless, designing automatic methods for automatically analyzing the VCE imagery via image processing and computer vision techniques pose significant challenges as we are dealing with big data.
Image processing involves uncertainty quantification and fuzzy techniques are effective in handling various tasks (Kerre and Nachtegael(eds) 2000; Vlachos and Ser- giadis 2006; Shamoi, Inoue, and Kawanaka 2014b; 2014a).
Recently, Shamoi et al (Shamoi, Kawanaka, and Inoue 2014) used fuzzification of HSI color space for apparel coordina- tion. In (Shamoi, Kawanaka, and Inoue 2014) a case ex-
Figure 1: Each VCE exam involves a human operator (gas- troenterologist) looking at various abnormal frames and classifying the impressions using linguistic terms. Auto- matic classification using a mapping of such linguistic terms to colors using fuzzy logic can be used for various tasks such as image retrieval.
ample in HSI (Hue, Saturation, Intensity) space is provided for obtaining a correspondence between colors and human impressions. In this work, we adapt the ground-work done in (Shamoi, Kawanaka, and Inoue 2014) to the VCE color space fuzzification for abnormality classifications. We pro- vide an overview of different color spaces (RGB, CMYK, HSV, La*b*) and their fuzzifications to organize all pos- sible human operator color perceptions of abnormalities in VCE images. Using operator defined impressions expressed in terms of linguistic terms we provide retrieval examples.
We note the overall framework is general in the sense that it can be expanded with domain knowledge for various related tasks.
The rest of the paper is organized as follows. Next section introduces different color spaces useful in VCE imagery and fuzzification techniques for representing color perceptions of human operators using fuzzy sets. Next, we provide some example classification results on VCE images for bleeding regions.
Figure 2: Fuzzy logic and mass assignment theory based mapping of different colors and human (operator) impres- sions of them. Image adapted from (Shamoi, Kawanaka, and Inoue 2014).
Color Fuzzification for Abnormalities in video Capsule Endoscopy
Fuzzification
Appearance of different abnormal regions such as polyps, adenomas, and bleeding in VCE videos under different color spaces provide different linguistic terms for description.
This can be utilized in the fuzzy logic framework advocated in (Shamoi, Kawanaka, and Inoue 2014), see Figure 2. All three components of a medical decision system, namely, dif- ferent color spaces, impressions based on linguistic terms, and mapping between them, are all interpretable using fuzzy logic. In particular we consider an example of bleeding de- tection in VCE, see Figure 3. Note that various color spaces can be utilized in the fuzzification framework and we con- sider standard color spaces such as RGB, CMYK, HSV and La*b*, we refer to (Wyszecki and Stiles 1982) for corre- sponding definitions and formulae. Each give a different per- spective of an abnormality, see Figure 3 for an example of bleeding region in RGB, CMYK, HSV and La*b* spaces.
Advantages of color (spectral) information can be exploited for different diagnostic decision purposes (Figueiredo et al. 2011; Prasath and Delhibabu 2015a). In this particu- lar case of bleeding detection, gastroenterologists tend to mark bleeding regions using linguistic terms such as dark red/medium red or pale red. For example, in RGB color space (see Figure 3(a)) the bleeding region is darker in green and blue channels and the overall appearance can be charac- terized as dark red in the RGB spectrum. Thus, fuzzy logic and mass assignment theory based mapping of different col- ors and human (operator) oriented impressions can be uti- lized in making a medical decision support system.
Figure 4 shows RGB and HSV fuzzy sets which are used to fuzzify different bleeding regions. In contrast to the apparel coordination application considered in (Shamoi, Kawanaka, and Inoue 2014), here we use only RGB color space and the Value (from HSV) fuzzy membership func- tions. Hence, the{Red, Green, Blue}and{Dark, Medium, Pale}are the spectral and linguistic terms, respectively. We utilized a ground truth marked histogram from two expe- rienced gastroenterologists for various bleeding regions and Figure 6 shows some examples with Red and three linguistic terms. Context dependent color impressions in the bleeding scenario arelight, strongwhich indicate lighter or stronger
(a) RGB (b) CMYK (c) HSV (d) La*b*
Figure 3: Different color spaces can be utilized in identifying abnormalities in VCE exams. In this example, a bleeding region (from a PillcamR Colon capsule image) is visualized in different color spaces and the proposal in this paper can play a crucial rule in medical decision support system for VCE. First row shows RGB, CMYK, HSV, La*b* images and next rows their corresponding individual channels. For CMYK space, we only show magenta, yellow, key images and omit cyan as it did not provide any visual information in this example.
bleeding regions respectively. Note that for different tasks these context dependent color impressions are utilized as a query, for example light bleeding (see Figure 5(a)).
Retrieval
Following (Shamoi, Kawanaka, and Inoue 2014) we utilized a taxonomy of color impressions adapted for VCE imagery based medical decision support systems. Here we describe it for bleeding regions and Table 1 provides the taxonomy of color impressions in the RGB - Value case, and a similar table is generated for the polyp vascularization with RGB - Density. Using these taxonomy we follow basic formulae in fuzzy logic such as the intersection (minimum), union (max- imum) of two membership functions,
(µA∩µB)(x) = min{µA(x), µB(x)}
(µA∪µB)(x) = max{µA(x), µB(x)}
andα-cut
fα={x:µf(x)≥α}.
These basic fuzzy formulae are used to fuzzify color spaces and interprets linguistic impressions of colors for composite cases. We used these formulae along with a map between
(a) RGB
(b) HSV - Value
Figure 4: Example fuzzy sets for color spaces RGB and HSV based on linguistic labels in bleeding regions classification in VCE. Note for the HSV space we only show the Value channel.
color impressions and color in RGB, Value in ranking simi- lar images for bleeding region identification.
Similar interpretations are done in the polyp vasculariza- tion and density, tortuosity are used as context dependent impressions. Figure 5 provides example results in bleed- ing and polyp vascularization using linguistic queries alone.
Figure 6 shows the corresponding ranking mechanism based on linguistic impressions for the bleeding the vasculariza- tion cases. As can be seen, histograms are utilized to iden- tify the top three ranked nearest images matching theDark redinterpretation and the retrieval results are accurate as per gastroenterologist ground truth markings. All the retrieved bleeding regions are from the jejunum area of the gastroin- testinal tract and we only show top three results according to color histogram matching, see Figure 6(a). A similar analy- sis with RGB space and polyp vascularizationdensityis un- dertaken and the the query image (Figure 5(d)) is described asPale Red Denseand the ranking given in Figure 6(b) ranks the resultant images according to the density of vasculariza- tion. All the retrieved vascular regions contain dense ves- sels and are malignant. We utilized 400 PillcamR Colon capsule images for bleeding detection, these were obtained from 5 different patients, and marked by two gastroenterol- ogists who provided ground truth regions along with bound- aries separating bleeding from normal mucosa tissue. For polyp vascularization based retrieval we used 100 images which are benchmarked against an automatic segmentation method (Prasath, Pelapur, and Palaniappan 2014) for calcu- lating the density of vascularization in polyps.
(a) Input (b) Groundtruth
(c) Bleeding
(d) Input (e) Groundtruth
(f) Polyp vasularization
Figure 5: Two example results in bleeding and polyp vas- cularization. (a): For a given input image of bleeding spot which is described as (b)Dark Redthe system retrieves sim- ilar images containing bleeding regions with (c) decreasing ranks (matching). (d): For a given input polyp image which is described as (e)Dense tortuousthe system retrieves sim- ilar polyp images with (f) decreasing density. Note we only the top three retrieval results and the ground truth regions are marked by human (operators) and used in training the impressions and their mappings.
Table 1: Taxonomy of color impressions in the RGB case for bleeding regions in VCE images.
Level Impressions Comment
I Red, Black Atomic
context independent colors II Dark, Medium, Pale Atomic
context independent, dependent colors III Combinations of I & II Composite
context independent, dependent colors
Conclusion
In this paper, we considered fuzzification of different color spaces for medical decision support systems in gastroin- testinal diagnosis using video capsule endoscopy. Follow- ing (Shamoi, Kawanaka, and Inoue 2014) we utilized fuzzy sets and logic, color space theory for VCE imagery inter- pretation and used it for retrieval tasks. Our preliminary re- sults in bleeding region detection and polyp vascularization in various VCE images indicate promise for using fuzzifica- tion techniques in a medical decision support system. Future works include introducing shape (e.g. polyp appearance), texture (e.g. pit patterns) features along with fuzzification framework studied here for different VCE videos. Moreover, increasing the number of experts (in our case study gastroen- terologists) and quantifying/enlarging the linguistic impres- sions is an important avenue to be explored. We also believe the framework considered here will help in identifying trash, bubbles for uninformative frame classification.
Acknowledgment
The author thanks the Gastroenterologists Dr. R. Shankar, Dr. A. Sebastian from Vellore Christian Medical College Hospital, India for their help in interpreting VCE imagery and Radhakrishnan Delhibabu (Kazan Federal University
& Innopolis, Kazan, Russia) in helping with data collec- tion/organization. This work was done while the author was visiting the Center for Scientific Computation and Mathe- matical Modeling (CSCAMM) at the University of Mary- land, MD, USA.
References
Figueiredo, I. N.; Prasath, S.; Tsai, Y.-H. R.; and Figueiredo, P. N. 2010. Automatic detection and segmentation of colonic polyps in wireless capsule imagess. Technical Re- port 10–37, University of Coimbra, Portugal.
Figueiredo, P. N.; Figueiredo, I. N.; Surya Prasath; and Tsai, R. 2011. Automatic polyp detection in pillcam colon 2 cap- sule images and videos: Preliminary feasibility report. Di- agnostic and Therapeutic Endoscopy2011:7pp. Article ID 182435.
Kerre, E., and Nachtegael(eds), M. 2000. Fuzzy techniques in image processing. Heidelberg: Physica-Verlag.
Niwa, H.; Tajiri, H.; Nakajima, M.; and Yasuda, K. 2008.
New Challenges in Gastrointestinal Endoscopy. Springer.
(a) Bleeding
(b) Polyp vascularization
Figure 6: Ranking based on linguistic terms and (a) color histograms in the bleeding region identification (b) Density of vascularization in the polyps in decreasing order.
Prasath, V. B. S., and Delhibabu, R. 2015a. Automatic con- trast enhancement for wireless capsule endoscopy videos with spectral optimal contrast-tone mapping. In Compu- tational Intelligence in Data Mining - Volume 1, 243–250.
Springer SIST (eds.: L. Jain, H. S. Behera, J. K. Mandal, D.
P. Mohapatra).
Prasath, V. B. S., and Delhibabu, R. 2015b. Automatic im- age segmentation for video capsule endoscopy. InCompu- tational Intelligence in Medical Informatics, SpringerBriefs in Applied Sciences and Technology, 73–80. Springer CIMI (eds.: N. B. Muppalaneni, V. K. Gunjan).
Prasath, V. B. S.; Figueiredo, I. N.; Figueiredo, P. N.; and Palaniappan, K. 2012. Mucosal region detection and 3D reconstruction in wireless capsule endoscopy videos using active contours. In34th IEEE/EMBS International Confer- ence, 4014–4017.
Prasath, V. B. S.; Figueiredo, I. N.; and Figueiredo, P. N.
2011. Colonic mucosa detection in wireless capsule endo- scopic images and videos. InCongress on Numerical Meth- ods in Engineering (CMNE 2011).
Prasath, V. B. S.; Pelapur, R.; and Palaniappan, K. 2014.
Multi-scale directional vesselness stamping based segmen- tation for polyps from wireless capsule endoscopy. In Figshare.
Rey, J.-F. 2008. Future perspectives for esophageal and
colorectal capsule endoscopy: Dreams or reality? InNew Challenges in Gastrointestinal Endoscopy. Springer. 55–64.
Shamoi, P.; Inoue, A.; and Kawanaka, H. 2014a. Fuzzy color space for apparel coordination.Open Journal of Information Systems1(2):20–28.
Shamoi, P.; Inoue, A.; and Kawanaka, H. 2014b. Percep- tual color space: Motivations, methodology, applications. In Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Sympo- sium on Advanced Intelligent Systems (ISIS), 1354–1359.
Shamoi, P.; Kawanaka, H.; and Inoue, A. 2014. Fuzzifica- tion of HSI color space and its use in apparel coordination.
InProc. of 25th Modern Artificial Intelligence and Cognitive Science Conference, 11–17. Spokane, WA, USA: CEUR- WS.org. online CEUR-WS.org/Vol-1144/paper2.pdf.
Vlachos, I. K., and Sergiadis, G. D. 2006. A heuristic ap- proach to intuitionistic fuzzification of color images. World Scientific. chapter 108, 767–774.
Wyszecki, G., and Stiles, W. 1982.Color Science: Concepts and Methods, Quantitative Data and Formulas. New York, NY, USA: Wiley.
