Color Processing
3.7 Image Segmentation for Hybrid Color Spaces: Vec- Vec-tor Gradient
In [L. Busin N. Vandenbroucke 2004], a color segmentation in hybrid spaces is proposed. The paradigm lies on multi-thresholds of 1D histograms. The main drawback of this approach is its marginal nature. In fact, each thresh-old is calculated individually for each color component. Others systems like [N. Vandenbroucke L. Macaire 2003] and [J. D. Rugna P. Colantoni 2004] only per-form a pixel classification without considering the spatial inper-formation. To tackle these problems, a vector gradient segmentation is adopted. Once the source image is transferred into a suitable hybrid color space, an edge detection algorithm is pro-cessed. This contour image is generated thanks to a vectorial gradient according to the following formalism. The gradient or multi-component gradient takes into account the vectorial nature of a given image considering its representation space (In our case a hybrid color space). The vectorial gradient is calculated from all com-ponents seeking direction for which variations are the highest. This is done through maximization of a distance criterion according to the L2 metric, characterizing the vectorial difference in a given color space. The approaches proposed by DiZenzo [Dizenzo 1986] first, and then by Lee and Cok under a different formalism are meth-ods that determine multi-components contours by calculating a color gradient from the marginal gradients. Given 2 neighbour pixels P and Q characterizing by their color attribute A, the color variation is given by the following equation:
4A(P, Q) =A(Q)−A(P) (3.6) The pixels P and Q are neighbors, the variation can be calculated for the infinitesi-mal gap: dp= (dx, dy)
dA= ∂A
∂xdx+∂A
∂ydy (3.7)
This differential is a distance between pixels P and Q. The square of the distance is given by the expression below:
Where, E ={e1, e2, e3} can be seen as a set of color components representing the three primaries of the hybrid color model. And whereGmn can be expressed as the marginal gradient in the directionnfor themth color components of the set E. The calculation of gradient vector requires the computation at each site (x, y): the slope
direction of A and the norm of the vectorial gradient. This is done by searching the extrema of the quadratic form above that coincide with the eigen values of the matrix M. Finally the contour force for each pixel (x,y) is given by the following relation:
Edge(x, y) =p
λ+−λ− (3.12)
This segmentation algorithm is assessed in the next section.
3.8 Experiments
In the idea to assess our system, we perform two evaluation stages. The first one is a color classification step to test if the color representation found by our framework is interesting in term of color distinction. The second step is a segmentation phase.
Indeed, a better representation system should give better segmentation results. Note that this software, called Best Color Space Finder, can be found on the L3i-ALPAGE website1.
3.8.1 Color classification
Test image descriptions: Our approach is applied on three different types of images. An image of: natural scene, document and a synthetic image, this is de-picted in table 3.3and figure3.9. The "Lenna" image is a conventional data source widely used in image processing. The image of cadastral map represents the prob-lem we attempt to tackle and finally, the synthetic image is a chessboard-like image composed of 64 squares; this image is of special interest since 2 adjacent squares can be distinguished by their saturation level. For this artificial image the number of clusters is set 65 (64 boxes and the surrounding border).
Protocol: Considering the full set of attributes a clustering algorithm (EM) is applied on each image. The number of color clusters per image is reported in table 3.3. The final purpose is to find the hybrid color space which provides the most similar color partition compared to with the one discovered using the whole set of features. The merit of a color space is evaluated under this consideration.
For each image, each color cluster is divided in two sub-sets, one for training and one for validation purpose. At this stage, each pixel is labeled according its cluster
1http://alpage-l3i.univ-lr.fr/ -> Best Colour Space Finder
3.8. Experiments 43
|Xtraining|pixels |Xtest|pixels
IM1 130107 130107
IM2 100951 100951
IM3 110424 110424
Table 3.4: Training and Test Databases
number. Feature selection methods are performed on the training database and each color space is evaluated thanks to the test data set. The color space performance evaluation consists in a classification stage. Both test and training databases are projected into the color space we want to evaluate. Each color vector from the test database is classified using a nearest neighbor rule (One nearest neighbor classifier, 1-NN for short). The test object is labeled with the cluster number of the most similar color instance from the training database; the underlying vector comparison is based on a euclidean distance (L2 norm). Table3.4reports the number of elements handled during the classification process.
Results: A fully detailed example is reported in table 3.5. The complete list of selected attributes by feature selection methods is provided for the image 3.9(b).
The selected components by the color space finder methods are quite heterogeneous, likely the reason lies on the variability of the considered approaches. In fact, they do not rely on the same principles; they all differ either from their search method or from their selection mechanism. Nevertheless, all feature selection methods adopted the saturation component to describe the content of this image. This demonstrates the pertinence of feature selection algorithms. For each real-world image, classification errors are gathered in tables 3.6, 3.7. The color space minimizing the confusion rate is elected to be the most discriminating feature space. Over the two images, color spaces built by the GACS perform the best. Consequently, generally speaking HCS outperform standard spaces. However, a closer look to the results denotes a poor achievement of selected attributes by OneRs, DHCS and CFS, they failed to overcome standard spaces. The "top 5" is often dominated by one HCS follows by four standard spaces.
(a) (b)
(c)
Figure 3.9: Images in use.
3.8. Experiments 45
Table 3.5: Hybrid color Spaces found on the Image IM2
Attributes CFS GACS DHCS OneRs
R 0 0 0 0
G 0 0 1 0
B 0 0 0 0
I1 0 0 0 0
I2 0 0 0 0
I3 0 0 0 0
T 0 0 0 1
S 1 1 1 1
I 0 0 0 0
L* 0 0 0 0
a* 0 0 0 0
b* 0 1 0 0
L* 0 0 0 0
u* 1 0 0 0
v* 0 0 0 0
A 0 0 0 0
C1 0 0 0 0
C2 0 0 0 0
X 0 0 0 0
Y 0 0 1 0
Z 1 0 0 0
Y 0 0 0 0
I 0 0 0 1
Q 0 0 0 0
Y 0 0 0 0
U 0 1 0 0
V 0 0 0 0
# of attributes 3 3 3 3
Table 3.6: Confusion rate on Image 1 IM1
Color Spaces Error Color Spaces Error
GACS 0.2868 OnRS 0.3558
L*u*v* 0.29785 La*b* 0.3578
YUV 0.32764 I1I2I3 0.3683
YIQ 0.3345 XYZ 0.4650
HSI 0.3394 CFS 0.5877
AC1C2 0.3435 DHCS 0.7067
RGB 0.3529
Table 3.7: Confusion rate on Image 3 IM2
Color Spaces Error Color Spaces Error
GACS 0.14065 RGB 0.1561
YIQ 0.1445 OnRS 0.1615
I1I2I3 0.1478 La*b* 0.1650
HSI 0.1488 XYZ 0.2093
L*u*v* 0.1533 CFS 0.3043
YUV 0.15387 DHCS 0.349
AC1C2 0.1557
3.8. Experiments 47
3.8.2 Application to segmentation and evaluation
In this section, we evaluate the worth of HCSvs the standard RGB space in a seg-mentation context. Firstly, we describe two datasets for these experiments: (1) the well-known and publicly available Berkley database, this later allows an evaluation on a large ground-truthed corpus; (2) an image of document on which a segmen-tation algorithm was applied. Next, the question of the segmensegmen-tation evaluation is discussed and finally, the segmentation performance is evaluated to figure out which color space offers the best image representation.
Database description
• Ancient color cadastral map
– In the context of the ALPAGE project, we are considering the digital-ization of ancient maps on which objects are drawn by using color to distinguish parcels for instance. We believe that such a problem would take advantages of a dedicated color space. The color segmentation of cadastral maps relies on the edge values defined in Eq.3.12. These edge values are then filtered using a two class classifier based on an entropy principle in order to get rid off low gradient values. At the end of this clustering stage a binary image is generated. This image will be called as contour image through the rest of this chapter. Finally, regions are ex-tracted by finding the white areas outlined by black edges. The gradient and the binary images are displayed in figure3.10.
• Berkeley segmentation data set
– Research on early vision problems such as edge detection and image seg-mentation has traditionally been critiqued on the grounds that quanti-tative measurements of performance are rare. It is therefore difficult to evaluate the effect of different design choices and the superiority (or in-feriority) of various novel heuristics that have been proposed in the liter-ature. Recently the availability of the Berkeley Segmentation DataSet [Martin 2001], [Segmentation 2002] has allowed the quantitative mea-surement of performance on boundary finding and the relative power of various pairwise similarity cues. While this is, of course, not the first example of quantitative measurement in segmentation the availability of this large data set containing a wide variety of images and segmentations by multiple human observers (11,0000 segmentations of 1000 images), allows one to draw conclusions with greater "statistical confidence". A sample of the kind of images that composes the database is shown in figure3.11along with a human segmentation.
(a) Original RGB image (b) Color Gradient Image
(c) Binary image
Figure 3.10: Map segmentation Segmentation evaluation method
• Berkeley data set (with ground-truth)
– The human segmented images provide our ground truth boundaries. We consider any boundary marked by a human subject to be valid. Since we have multiple segmentations of each image by different subjects, it is the collection of these human-marked boundaries that constitutes the ground truth. In figure 3.11, the output of our algorithm is presented for a given image. Let us assume that this output is a soft boundary map with one pixel wide boundaries, valued from zero to one where high values signify greater confidence in the existence of a boundary. The task is to determine how well this soft boundary map approximates the ground truth boundaries.
• Cadastral map subset (without ground-truth).
– Another way to assess a segmentation process is to compute the Levin and Nazif (LN) criterion. Without ground-truth for our images, a un-supervised evaluation is required. LN criterion combines intra-class and inter-class disparities. Inter-class disparity score computes the sum of contrasts of the regions balanced by their surfaces while the intra-class uniformity score computes the sum of the normalized standard deviation
3.8. Experiments 49
Figure 3.11: Boundary detection: Machine vs Human. Precision and Recall curve.
of each region. It takes into parameters the segmented image and the original image and returns a score, the higher the better. This com-parison is carried out on 50 pairs of maps. Levin and Nazif criterion [Rosenberger 2006] is the union of two principles, the variability intra and inter regions.
Results
• Berkeley data set (with ground-truth)
– The vector gradient, defined in section 3.7, is performed as a bound-ary detector on the Berkeley Dataset. Segmentation evaluation in RGB space and HCS space are illustrated in figure 3.12. In figure 3.12, Pre-cision and Recall (PR) curves are presented. In this graph, Recall is a measure of how well the detector performs in finding relevant contours while Precision is a measure of how well the detector performs in not returning nonrelevant contours. The F-measure is the harmonic mean of precision and recall. One remarkable consideration extracted from the Precision/Recall curves is the fact that the PR curves in HCS is always above or equal to the RGB PR curve. Consequently, the F-measure is slightly higher when performing in HCS space. This is an encouraging observation, it means that at least a HCS will not degrade the segmenta-tion results and compare to RGB, HCS will tend to improve the precision when the recall is low.
• Cadastral map subset (without ground-truth)
– In table 3.8LN criterion for the segmentations based on RGB and HCS is reported. As mentioned in the prior paragraph, HCS reveal to be slightly better than RGB, it means that regions found when processing the color gradient in HCS are more uniform and more contrasted than in RGB space. However, this remark must be tempered by the fact that the improvement is not significant. Somehow the approach reminds of the ”killing butterflies with missiles” paradigm, a complex framework to achieve a bit better than the original image.
3.9 Conclusion
The quality of a color model is judged by two decisive factors: "Robustness" and
"Distinction". The robustness of the color representation is an indication of the sensitivity of color values to illumination and brightness variations. The “Distinc-tion” capacity of a color model is directly linked to its capacity to separate one color from the others. The color space minimizing the error rate classification is the most discriminating space for a given image (Tables 3.6,3.7). The space generating the
3.9. Conclusion 51
(a) On RGB space
(b) On HCS space
Figure 3.12: Overall results: Precision/Recall curves on Berkeley benchmark
Table 3.8: Comparison HCS and RGB spaces on a segmentation process using LN criterion.
Dizenzo segmentation color
cadastral maps LN Criterion on 50
images
Average Std deviation
RGB 0.4770375 0.005396543
HCS 0.480325 0.007211647
least mistake will be retained to continue treatments on the image. The chosen space is minimizing the distance intra-class, within the same unit chromatic while maximizing the distance inter-classes. Such properties are helpful in post-processing stages such as segmentation, or graphics recognition. The LN criterion results lead to the same conclusion, showing that the contrast inter regions and the homogene-ity intra-region are slightly better in HCS than in the RGB case. These results are encouraging and they demonstrate how important it is to choose a "good" color model. To take the stock, in this chapter, we have presented a color space selection framework. Our contribution focuses on a "all-in-one" system to find a suitable color space. Our tool can be seen as a pprocess to any color information re-trieval application (Segmentation, graphic recognition . . . ). Our approach aims at maximizing one criterion which is the color recognition rate to unleash the color information. Each image is like no other, so a dedicated color representation is re-quired. We believe, it is hardly possible to model a unique color space from a given image set and then to apply this “mean model” individually, that’s why our method computes independently a dedicated model to each image. Our framework relies on a wise use of different feature selection methods in order to take advantages of their diverse ways to reach a single goal. Finally, Hybrid Color Spaces are particularly well suited while dealing with very specific images, such as medical images, images of documents where CIE spaces are not particularly well designed. We believe that much color image software would get profit to the use of an adapted color space.
A future work is envisaged by comparing Hybrid Color Spaces to Support Vector Machine approaches such as Multidimensional scaling (MDS).