Asymmetric Generalized Gaussian Mixtures for Radiographic Image Segmentation

(1)

Asymmetric Generalized Gaussian Mixtures for Radiographic Image Segmentation

Nafaa Nacereddine and Djemel Ziou

Abstract In this paper, a parametric histogram-based image segmentation method is used where the gray level histogram is considered as a finite mixture of asymmetric generalized Gaussian distribution (AGGD). The choice of AGGD is motivated by its flexibility to adapt the shape of the data including the asymmetry. Here, the method of moment estimation combined to the expectation–maximization algorithm (MME/EM) is originally used to estimate the mixture parameters. The proposed image segmentation approach is achieved in radiographic imaging where the image often presents an histogram with a complex shape. The experimental results provided in terms of histogram fitting error and region uniformity measure are comparable to those of the maximum likelihood method (MLE/EM) with the advantage that MME/EM method reveals to be more robust to the EM initialization than MLE/EM.

1 Introduction

The segmentation is a key step in any image analysis system where, its quality has indisputably a direct effect on the results of the next stage, i.e., the feature extraction on which are dependent the image interpretation and retrieval, the object recognition and classification, etc. For delicate applications such as radiographic imaging in both fields of industry and medicine, an accurate segmentation is more than ever required since a bad interpretation or a false diagnosis lead sometimes to irreparable harm to the human patient and the industrial plant in question. On the other hand, an accu-

N. Nacereddine (

B

⁾

Research Center in Industrial Technologies CRTI, P.O.Box 64, 16014 Algiers, Algeria e-mail: [email protected]; [email protected]

N. Nacereddine

Lab. des Math. et leurs Interactions, Centre Universitaire de Mila, Mila, Algeria D. Ziou

Dpt. de Math. et Informatique, Université de Sherbrooke, Québec, Canada e-mail: [email protected]

R. Burduk et al. (eds.),Proceedings of the 9th International Conference

on Computer Recognition Systems CORES 2015, Advances in Intelligent Systems and Computing 403, DOI 10.1007/978-3-319-26227-7_49

521

(2)

rate modeling of unknown probability density functionspdfs of data, encountered in practical applications, can play an important role in the direction of simulation and design of modern signal processing systems [1]. The latter include the image gray level histogram which is, in general, multimodal and which can be approximated by a finite mixture model (FMM) [2] for the purpose of image segmentation. The most popular method to estimate FMM parameters is the expectation–maximization (EM) algorithm [3]. In most of applications, including image segmentation, the Gaussian pdf is used for FMM but the analyzed signals often present complex and non-Gaussian shapes. This is why, in this work, the asymmetric generalized Gaussian distribution (AGGD) has been chosen because it is shown to not only model a wide range of statistical distributions (e.g., impulsive, Laplacian, Gaussian) but also include the asymmetry [4]. This last property permits to an asymmetric generalized Gaussian mixture model (AGGMM) [5] to portray successfully a large class of signals (e.g., image gray level histogram, speech data). However, in case of maximum likelihood estimation (MLE) using EM algorithm, all the AGGD parameters in the mixture are expressed by high nonlinear equations [5] which makes the numerical solution cumbersome and sensitive to EM initial values, as we can see later in experiments. As an alternative, in this work, the moment matching estimation (MME) method [6, 7] in combination with the EM algorithm is used and newly tested on the AGG mixture model. For the aim of segmentation of images issued from the real-world applications mentioned above, MME/EM is applied and compared with MLE/EM in terms of histogram fitting, segmentation quality, and robustness to EM initialization. The remainder is organized as follows. In Sect.2, the analytical expression of AGGD and its mixture model for image segmentation are given. Section3deals with the computing of AGGD moments and presents the algorithm to combine them with EM algorithm. The experiments are carried out and commented in Sect.4. Finally, the concluding remarks are drawn in Sect.5.

2 AGGD and Its Mixture Model

Thepdf of a one-dimensional AGGD is defined as

f(x|θ)=

⎧⎪

⎪⎪

⎨

⎪⎪

⎪⎩

β

(α1+α2)Γ (1/β)e^−[(−^x^+μ)/α¹^]^β ifx< μ β

(α1+α2)Γ (1/β)e^−[(^x^−μ)/α²^]^β ifx≥μ

(1)

forx(x∈R), and whereθ=(μ, α1, α2, β)^T (μ∈R,{α1, α2, β} ∈R^∗₊) is the distribution parameter vector with the components are the left and the right scale parameters and the shape parameter, respectively. The gamma function is defined by Γ (ξ)=_∞

0 e⁻^tt^ξ−1dt. The parameterβ dictates the exponential rate of decay: the larger theβthe flatter thepdf; the smaller theβthe more peaked thepdf, as shown in

(3)

Fig. 1 Thepd fof AGGDs forμ=5,α1=2,α2=4, β= {1,2,5}

Fig.1. Special cases of AGGD can be obtained with some values ofβ: asymmetric Laplacian distribution (β=1), asymmetric Gaussian (β=2), impulse distribution (β →0), and uniform distribution (β → ∞). Also, the asymmetry is avoided if α1=α2. Note thatα1andα2which express the distribution width are linked to the standard deviationsσ1andσ2by

αi =σi

Γ (1/β)

Γ (3/β), i =1,2 (2)

For the purpose of the image segmentation, let us assume that the normalized gray level histogramhg(g∈ [0,L−1]withLthe number of gray levels) can be approximated by an AGG mixture model as

f(g|)= M m=1

πmf(g|θm) (3)

where =(πm,θm) is a vector of parameters to estimate with θm=(πm, μm, α1,m, α2,m, βm), m=1, . . . ,M (M is the number of the regions to be segmented) andπm themth mixing parameter satisfyingπm>0 and _m^M₌₁πm=1. Thus, the complete log-likelihood function is given by

Lc()=

L−1

g=1

M m=1

zg,mhglog [πmf(g|θθθm)] (4)

Using the MLE/EM method, the posterior probabilities zg,t and the mixture parameters estimatesˆ are given in [5]. It is noted [5] that all the derived equations in M-step (AGGD parameters estimation) are highly nonlinear. In fact, there are many

(4)

types of nonlinear functions, such as logarithm, power, digamma, and especially the piecewise-defined function for which, the numerical solution based on Newton–

Raphson method could be instable due to its sensitivity, among other, to differen- tiation involved at the function breakpoint, i.e.,μm. To map the segmented image, the optimal realization S(u,v) for each pixel with (u,v)coordinates is assigned according to the Bayes decision rule

S(u,v)=

⎧⎪

⎪⎨

⎪⎪

⎩

ω1(tf) if p1(tf)= max

m:1,...,Mpm(tf) ωM...(tf) if pM(tf)= max

m:1,...,Mpm(tf)

(5)

where,tf is the final number of iterations for the EM algorithm convergence,pm(t)= πmf(g|θθθm(t))andωm(t)is the estimated real mean parameter given byωm(t+1)=

g zg,m(t)hgg

g zg,m(t)hg.

3 Moment Matching Estimation (MME) on AGGMM

Proposition 1 If X is a random variable with the AGGD density and, for any r ∈N, the r -order noncentral moment can be written as

E X^r = 1

(α1+α2)Γ (1/β) r k=0

r

k (−1)^kα₁^k⁺¹+ α₂^k⁺¹ Γ

k+1 β

μ^r⁻^k (6)

Proof

E X^r = β

(α1+α2)Γ (1/β)

⎡

⎣ μ

−∞

x^re⁻

−x+μ α1

_β

dx+ ∞ μ

x^re⁻

x−μ α2

_β

dx

⎤

⎦

By taking into account the following: (1) set the change of variabley=(−x+μ)/α1

ifx≤μandy=(x−μ)/α2ifx > μ, (2) use the power series formula(a+b)ⁿ=

n k=0

_n

k

aⁿ⁻^kb^kand the definite integral_∞

0 x^ν−1exp(−ηx^p)dx= ¹_pη⁻^ν^pΓ

ν p

and (3) replace temporarilyβ/[(α1+α2)Γ (1/β)]byΩ, we obtain

E X^r =Ω

⎡

⎣(−1)^rα^r₁⁺¹ ∞ 0

y− μ

α1

r

e⁻^y^βdy+α₂^r⁺¹ ∞

0

y+ μ

α2

r

e⁻^y^βdy

⎤

⎦

(5)

=Ω r

k=0

r

k (−1)^rα^r₁⁺¹

−μ α1

r−k

+α^r₂⁺¹ μ

α2

r−k∞ 0

y^ke⁻^y^βdy

= 1

(α1+α2)Γ (1/β) r

k=0

r

k (−1)^kα^k₁⁺¹+α^k₂⁺¹ Γ

k+1 β

μ^r⁻^k

Proposition 2 Using the Proposition 1, the first four moments (mean, variance, skewness, and kurtosis) of X are deduced as

Mean

me=μ−(α1−α2)Γ (2/β)

Γ (1/β) (7)

Variance

σ² =α³₁+α₂³

α1+α2 ×Γ (3/β)

Γ (1/β)−(α1−α2)²Γ (2/β)²

Γ (1/β)² (8)

Skewness

γ1= α1−α2

σ³

×

3α₁³+α₂³ α1+α2

Γ (2/β)Γ (3/β)

Γ (1/β)² −2(α1−α2)²Γ (2/β)³

Γ (1/β)³ −(α₁²+α₂²)Γ (4/β) Γ (1/β)

(9) Kurtosis

γ2= 1 σ⁴

α₁⁵+α₂⁵

α1+α2 ×Γ (5/β)

Γ (1/β)−(α1−α2)²

4(α²₁+α₂²)Γ (2/β)Γ (4/β) Γ (1/β)²

−6α³₁+α₂³ α1+α2

×Γ (2/β)²Γ (3/β)

Γ (1/β)³ +3(α1−α2)²Γ (2/β)⁴ Γ (1/β)⁴

−3 (10)

Knowing that{x₁, . . . ,xn}is a set of n realizations of random variable X , the empir- ical or sample moments denoted mes, σs², γ1s, andγ2sare computed as

mes = ¯x= ⁿi=1xi/n ; σs²= ⁿi=1(xi− ¯x)²/n

γ1s= ⁿi=1(xi− ¯x)³/(nσs³) ; γ2s = ⁿi=1(xi− ¯x)⁴/(nσs⁴) (11) In this section, the EM algorithm and the moment matching estimation method are combined for a mixture of AGGDs. In fact, here, MME has as role to estimate the AGGD parameters by equating the analytical expressions of moments given by

(6)

Algorithm 1Pseudo-code of EM-based AGGMM moment matching method Data: Number of modesM; initial mixture parameters⁽⁰⁾; convergence threshold Result:, the estimate ofˆ

t←0

Initialization:^(t)=⁽⁰⁾ repeat

E-step:Compute the a posteriori probabilitieszˆ^(t)_g_,_m z^(t)_g_,_m=πmf(g|θθθ^(t)m) M

l=1πlf(g|θθθ^(t)l ) M-step:Maximization ofL_c(⁽^t⁾)

πm^(t)= gz^(t)_g_,_mhg and θˆ⁽m^t⁾=θ^(t)mme,m

t←t+1 until E

L_c(),ˆ⁽^t⁾

−E

L_c(),ˆ⁽^t⁻¹⁾< ;

Eqs. (7)–(10) with the sample moment equations given in (11) and then to solve numerically the system of nonlinear equations

me=mes; σ²=σs²; γ1=γ1s; γ2 =γ2s (12) Thus, the estimated parametersθmme=(μmme, α1mme, α2mme, βmme)^T are obtained.

So, such an estimation replaces the log-likelihood function differentiations in M-step of MLE/EM method. However, E-step remains as it is since it permits to evolve the algorithm so that, the likelihood function is maximized iteratively in order to better fit the estimated mixture to the real histogram. The combination of EM algorithm with MME for an univariate asymmetric generalized Gaussian mixture model is summarized in Algorithm 1.

4 Experiments

With the purpose to compare the performances of MME/EM with those of MLE/EM in histogram clustering for image segmentation, the experiments are carried out on real X-ray images. The first image shown in Fig.2 represents a knee X-ray image with respectively, the lowest and the highest parts of the femur and the tibia bones. This image may be labeled a priori in three regions: the bone, the flesh, and the background. The second tested image (see Fig.3) represents an X-ray image of a welded joint [8] and the region of interest (ROI) subjected to the segmentation. In this case, the task is to extract the weld defect indications from the regions representing other radiogram parts such as the welded joint and the base metal. We initialize the EM algorithm as follows: (1) all the mixing

(7)

Fig. 2 Knee X-ray image. Histogram fitting and segmented regions for MME/EM ina,cand MLE/EM inb,d(withΔμ0= {0 0 0}); and for MME/EM ine,gand MLE/EM inf,h(withΔμ0= {10 0 0})

(8)

Fig. 3 Weld X-ray image. Histogram fitting and segmented regions for : (1) MME/EM ina,cand MLE/EM inb,d(withΔμ0= {0 0 0}), (2) MME/EM ine,f (withΔμ0= {−20 −50 0}) and (3) MME/EM ing,iand MLE/EM inh,j(withΔμ0= {−20 −50 0}andΠσ1²,0=Πσ2²,0= {5 5 5})

(9)

parametersπm(0)are taken equal to 1/M, (2) the pseudo-meansμm(0)are chosen so thatμm(0)=H⁻¹(0.05)+(2m−1)[H⁻¹(0.95)−H⁻¹(0.05)]/(2M)whereH denotes the cumulative histogram, (3) the initial left and right variancesσ_1,²m(0)and σ₂²_,m(0)are equally set to(μ2(0)−μ1(0))²/10 (recall thatσ₁²_,₂is linked toα_1,2by Eq. (2)), and (4) all initial shape parametersβm(0)are taken equal to 1.5. Two comparison criteria, the sum of squared error (SSE) and the region uniformity measure U [9], are used to measure the histogram curve fitting and to evaluate the ensued segmentation results, respectively. They are given by

S S E=

g

h(g)− f(g,)ˆ ₂

(13)

wherehis the original histogram and f(g,)ˆ its estimation by AGGD mixture

U=1− M m=1

πmσm²/σT² (14)

whereπmis the area ratio of themth segmented region,σm²its variance andσ_T²the total image variance. The highest isU, the better is the segmentation. In order to examine further, the influence of the EM algorithm initialization change on the estimation scores of both methods, we have introduced three parametersΔμ0,Πσ₁²_,₀andΠσ₂²_,₀ which are, respectively, equal toμ0−μ^new₀ ,σ₁²_,^,₀^new/σ₁²_,₀ andσ₂²_,^,₀^new/σ₂²_,₀. For both methods MME/EM and MLE/EM and for all initializations, the fitted histograms and the images segmented accordingly, are illustrated in Figs.2 and3. Also, the estimated AGGMM parameters, the values ofˆ S S EandUare reported in Table1.

When the above-mentioned first EM initialization is used, i.e.,Δμ0 =0 andΠσ₁₀² = Πσ₂₀² =1, for each of the tested Knee and Weld images, the segmentation results are given in Figs.2a–d and3a–d and their evaluation scores are provided by the two first rows of Table1. In fact, the MME/EM method is as performing as the MLE/EM method in the knee X-ray image classification in bone, flesh, and background regions whereUreaches 0.92 for the both. The same successfulness is observed for the weld radiographic image for both methods (U∼0.83 and 0.81) where, in spite of the bad quality of such images, characterized usually by weak contrast, blurry contours, weld overthickness and lot of artifacts and noise, the major part of the weld defects named slag inclusions, contained in the selected ROI, are well extracted. Due to the nature of the nonlinearities found in the derived AGGMM parameters by MLE/EM in [5], especially the piecewise-defined functions, the algorithm implementation works in the limits of instability and becomes, among other, very sensitive to initialization.

The conclusions given above are argued by the examples given in Figs.2e–h and 3e–j and summarized in Table1(Knee image: 2nd row and Weld image: 2nd and 3rd rows). In fact, when the initial mean values μm,0 given at the beginning of this section are moved by amountsΔμm,0, unlike the MME/EM method for Knee image, the MLE/EM method fails to recover correctly all the distribution modes

(10)

Table1AGGMMparameterestimatesandfittingerrorviaMME/EMandMLE/EM Δμ0Πσ 2 1Πσ,0 2 2πμωααβSSEU12,0 KneeX-ray image

MME{000}{111}{111}{0.320.390.29}{12.366.6156}{15.577.9164.3}{6.917.724.4}{12.233.339.1}{1.61.42}2.7×10−30.92 MLEid.id.id.{0.420.290.29}{963160}{28.281.1161.8}{0.14.626.7}{0.828.328.9}{0.41.31.5}1.8×10−30.92 MME{1000}{111}{111}{0.320.390.29}{12.366.4156}{15.477.7164.2}{7.117.724.5}{12.333.439.2}{1.71.42.1}2.7×10−30.92 MLEid.id.id.{0.520.180.30}{21.777160.7}{32.394.2161.4}{18.34.628.1}{55.923.429}{17.61.11.5}4.4×10−30.87 WeldX-ray image MME{000}{111}{111}{0.210.300.49}{54107.3164.4}{50.299.4168.5}{44.337.939}{36.523.647.5}{6.326.1}7.7×10−50.83 MLEid.id.id.{0.130.350.52}{34103.9170.4}{35.991.3166.9}{25.448.147.2}{29.221.643.1}{5.3618.8}7.4×10−50.81 MME{−20 −500}

{111}{111}{0.110.330.56}{34.489.0159}{3285.5163.2}{2636.743.8}{2129.552.5}{5.65.46.6}6.8×10−50.79 MLEid.id.id.FailsFailsFailsFailsFailsFails–– MME{−20 −500}

{555}{555}{0.090.290.62}{2682.7163.8}{28.878.8158.6}{17.73558.6}{23.326.947.7}{44.35.8}6.7×10−50.76 MLEid.id.id.{0.090.270.64}{2791.4161.5}{29.477157.3}{18.542.859.7}{23.412.452.1}{4.15.423.6}8.0×10−50.74

(11)

as seen in Fig.2f, h and which produces, as consequence, greaterS S E, and lower U in comparison to MME/EM. That said, because the left part of the 2nd mode is assigned wrongly to 1st mode, we are in presence of under-segmentation where some parts of flush are confused with background. It is also noted from the fitted histogram in Fig.2b. that MLE/EM method tracks better the 1st side of the 2nd mode despite its pronounced verticalness. However, whenαi,m,i =1,2 get close to 0 or the new mean initial valuesμ^new_1,0 andμ^new_2,0 (see 2nd row of Table1for Weld and Fig.3e) are very close to each other, the numerical solutions for EM algorithm become instable causing computation breaking as it can be seen in Fig.3 where, MLE/EM method fails completely. Nevertheless, still for weld X-ray image, with the same new initial mean values (−20−50 0); but now, if the initial values of the left and right variances, given in the beginning of the experiments section, are multiplied by 5 (Πσ₁₀² =Πσ₂₀² =5) implying, according to Eq. (2), an increase ofαi,mby√

5, the MLE/EM method becomes again stable and converges correctly with slight lower scores than MME/EM (see last row of Table1for Weld image).

5 Conclusion

In this paper, MME/EM method is applied on finite mixture of AGGDs to fit the histograms of images obtained from radiographic imaging. In addition to its good behavior and the reasonable agreement of its segmentation results, compared to those obtained by the MLE/EM method, the solutions given by the moment matching method, although they handled nonlinear equation system, are stable and more robust to the EM algorithm initialization. In fact, for MLE/EM, due to the nature of the nonlinearity of the derived parameter equations, especially the piecewise-defined function, the numerical solutions given by methods such as Newton–Raphson could be in the limits of stability when the scale parameters tends toward zero and in addition, very sensitive to EM initialization as shown by the experiments. That is why, in the light of the obtained results, the MME/EM method could be an interesting alternative to fit histograms approximated by this kind of distributions.

References

1. Kokkinakis, K., Nandi, A.K.: Exponent parameter estimation for generalized Gaussian probability density functions with application to speech modeling. Signal Process.85(9), 1852–1858 (2005)

2. McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000)

3. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. Ser. B39(1), 1–38 (1977)

4. Lee, J.-Y., Nandi, A.K.: Parameter Estimation of the Asymmetric Generalised Gaussian Family of Distributions. Stat. Signal Process., IEE Colloquium, pp. 9/1–9/5 (1999)

(12)

5. Nacereddine, N., Tabbone, S., Ziou, D., Hamami, L.: Asymmetric generalized Gaussian mixture models and EM algorithm for image segmentation. In: Proceedings of 20th International Conference on Pattern Recognition (ICPR’2010), pp. 4557–4560. Istanbul (2010)

6. Lindsay, B.G., Pilla, R.S., Basak, P.: Moment-based approximations of distributions using mixtures: Theory and application. Ann. Inst. Stat. Math.52(2), 215–230 (2000)

7. Delicado, P., Goria, M.N.: A small sample comparison of maximum likelihood, moments and L-moments methods for the asymmetric exponential power distribution. Comput. Stat. Data Anal.52(3), 1661–1673 (2008)

8. Nacereddine, N., Ziou, D., Hamami, L.: Fusion-based shape descriptor for weld defect radiographic image retrieval. Int. J. Adv. Manuf. Tech.68(9–12), 2815–2832 (2013)

9. Ng, W.S., Lee, C.K.: Comment on using the uniformity measure for performance measure in image segmentation. IEEE Trans. PAMI18(9), 933–934 (1996)