2 Comparing Images Using Color Histograms 2.1 Color Histograms for Images

n²n

distances have to be calculated. The high number of required distances is one more reason to search for a faster metric for the area of image clustering.

The rest of this paper is organized as follows: Sect.2will briefly summarize the theoretical foundations for our study. In Sect.3the mathematical background of the original EMD and our own approximation algorithm will be presented. The results of our experiments are illustrated and declared in Sect.4. Finally we conclude our findings and particularize our plans for future work.

2 Comparing Images Using Color Histograms 2.1 Color Histograms for Images

In the following we assume that an imageA(orB; C; : : :) can be summarized as a collection ofT^Apixels with color measurementsC^AD fc^A₁; : : : ;c^A_TAg. The measure-ment c^A_t .t D 1; : : : ; T^A/takes values fromXD fx1; : : : ;xMg R^Kwhich reflects the possible intensities in a underlying prespecifiedK-dimensional color space. In digital image processing,K D3has proven to be useful e.g. when the 8-bit-coded RGB (Red-Green-Blue) color spaceXD f.0; 0; 0/; .0; 0; 1/; : : : ; .255; 255; 255/gis used withM D256³D16;777;216different colors (DRGB intensity triples).

For characterizing the distribution of these measurements across the image (and for comparing images), (color) histograms can be used. A histogram is a fixed-size discrete distributional function with an a priori declared numberN of disjoint color rangesXi X (i D 1; : : : ; N), the so-called bins. So, e.g., if each possible color of the 8-bit-coded RGB space would be declared as a bin, we would have N DM D16;777;216bins. Alternatively, if 8 subsequent intensities in each of the K D 3dimensions would be summarized, one would receive onlyN D₂₅₆

8

3

D 32³D32;768bins. For each biniof histogram h^AD f.p^A₁; h^A₁/; : : : ; .p^A_N; h^A_N/gwe can calculate its number of colors in its rangeN_i, its position (mean color) in the color space pi and the corresponding number of pixelsh^A_i in imageAaccording to

Ni D

Usually, to make images comparable with different numbers of pixels (T^A; T^B; : : :), the histogram values are transformed into shares of pixels, i.e. the frequencyh^A_i is replaced by the weight (share) w^A_i D h^A_i=T^A. Thus we get the normalized histogram w^AD f.p^A₁;w^A₁/; : : : ; .p^A_N;w^A_N/g. Also, instead of histograms w.r.t. to the same color ranges across all images, so-called signatures can be used,

Comparing Earth Mover’s Distance and its Approximations for Clustering Images 71

Fig. 1 The CIE L*a*b* color space (left) and an example of a color intensity histogram (right) of a macaw picture (middle)

which describe the color distribution of an image via image-specific non-empty bins. An adequate coding for a signature for image A with N^A bins could be q^A D f.p^A₁;w^A₁/; : : : ; .p^A_NA;w^A_NA/g.N^A N /. The positions and weights are calculated in the same way as above. This is especially useful when parts of the color space are expected to have no weights, e.g. when comparing underwater-images without red or yellow colored pixels. However, in the following, we restrict our discussions to histograms and weights.

2.2 The CIE L*a*b* Color Space

In our experiments we use distances between color histograms to compare images.

For humans, the similarity of colors has proven to be very important (seeSwain and Ballard 1991). So, e.g. humans use colors to interpret traffic signs and to decide whether food is healthy or not, animals use colors to send messages or warnings.

There are many different color spaces. Probably the best known is the above mentioned RGB space which uses an additive mixing of primary color intensities.

Initially it was used for photographic experiments by James Clerk Maxwell in 1861 using three color-filtered separate takes. Later it became popular for TV, computer displays and other electronic devices. It bases on the human processing of color stimuli via three types of retinal cones in the eye. But it was not designed to produce color codings whose distances reflect perceived color differences. Here, the CIE (Commission Internationale de l’Eclairage) L*a*b* color space – also a three-dimensional color space – has proven to be more adequate (see, e.g. the experiments conducted bySchwarz et al. (1987)). The first dimension represents the lightness (L*), the second and third are opponent red-green (a*) and yellow-blue (b*) color-axes (see Fig.1). Every visible color is represented in this space, thus no output-device can display all colors of the L*a*b* color space. The color

72 S. Frost and D. Baier

Table 1 Overview of the six traditional distance measures used in our experiments. The distance is calculated on the basis of the normalized histograms w^Aand w^BwithmiD.w^A_i Cw^B_i/=2and

O w_iDPi

jD1w_j. The computational complexity (abbreviated: CC) relates to the number of binsN

Distance Type Formula CC

similarity functions developed by the CIE are based on so called ‘supplementary standard colorimetric observers’ to get a relation between the perceived color and the physical reason for a color-stimulus (Wyszecki and Stiles 1982). It is one of the common color feature spaces in computer vision (see:Rubner et al.(1997) or Puzicha et al.(1999)) and even professional image processing software like Adobe Photoshop uses the L*a*b* color space for internal color processing. The L*a*b*

coordinates can be obtained by transforming RGB coordinates into the XYZ system, which is able to normalize the colors with respect to the source of illumination. The second step is a nonlinear transformation from the normalized XYZ-system into the L*a*b*-system. In this special color space the calculated Euclidean distance between two colors corresponds to the human perception of color dissimilarity. As a result the ground distance (Sect.3.1) for the EMD will be the Euclidean distance, while using this color space in our experiments.

2.3 Traditional Distance Measures

Puzicha et al. (1999) compared the EMD with seven other distance measures on histograms w.r.t. image retrieval. They compared the number of correct retrieved images relative to total number of retrieved images. Unfortunately, their results don’t solve our problem. For example, Kullback-Leibler divergence yielded a small percentage of classification errors but because of its asymmetry it is not usable for hierarchical clustering approaches. Table 1lists the traditional distance measures we used in our evaluation. All distances between imageAandB make use of the corresponding normalized histograms w^A;w^B w.r.t. the L*a*b* color space and the same color ranges (bins). The quadratic-form distance additionally uses the

Comparing Earth Mover’s Distance and its Approximations for Clustering Images 73

symmetric similarity matrix S2R^NN withsijD 1_max^gij_gij (Rubner et al. 2000), wheregijis the distance (we used Euclidean dist.) between the positions p^A_i and p^B_j of the binsiandj.