IMPROVED SOS BLIND IDENTIFICATION ALGORITHMS BASED ON SYMMETRIC EVD/SVD

There is a current trend in ICA/BSS to investigate the “average eigen-structure” of a large set of data matrices which are functions of available data (typically, covariance or cumulant matrices for different time delays). In other words, the objective is to extract reliable information (like for example, estimation of sources and/or the mixing matrix) from the eigen-structure of a possibly large set of data matrices [160]. However, since in practice only a finite number of samples of signals corrupted by noise is available, the data matrices do not exactly share the same eigen-structure. Furthermore, it should be noted that determining the eigen-structure on the basis of one or even two data matrices leads usually to poor or unsatisfactory results because such matrices, based usually on arbitrary choice, may have some degenerate eigenvalues and they usually discard information contained in other data matrices. Therefore, from a statistical point of view, in order to provide robustness and accuracy it is necessary to consider the average eigen-structure by taking into account simultaneously a possibly large set of data matrices [158, 159, 160, 1222]. In this and the next section, we will describe several approaches that exploit average eigen-structure in order to estimate reliable sources and mixing matrix.

4.3.1 Robust Orthogonalization of Mixing Matrices for Colored Sources Let us consider the standard mixing model:

x(k) =H s(k) +ν(k), (4.97) where x(k)∈ IR^m is the available vector of sensor signals, H ∈ IR^m×n is the full column rank mixing matrix ands(k)∈IRⁿ is the vector of temporally correlated sources.

IMPROVED BLIND IDENTIFICATION ALGORITHMS BASED ON EVD/SVD 149 We formulate the robust orthogonalization problem as follows: Find a linear transforma-tionx(k) =Q x(k)∈IRⁿ such that the global mixing matrix, defined asA=Q H∈IR^n×n, will be orthogonal and unbiased by the additive white noise ν(k).

Such robust orthogonalization is an important pre-processing step in a variety of BSS methods. It ensures that the global mixing matrix is orthogonal. The conventional whiten-ing exploits the zero time-lag covariance matrix Rxx =Rx(0) =E{x(k)x^T(k)}, so that the effect of the additive white noise can not be removed if the covariance matrix of the noise can not be precisely estimated, especially in the case when the number of sensors is equal to the number of sources.

The idea of the robust orthogonalization is to search for such a linear combination of several (typically, from 5 to 50) symmetric time-delayed covariance matrices, i.e.,

Rx(α) = XK

i=1

αiR˜x(pi), (4.98)

that matrix Rx is positive definite and moreover, it is not sensitive to the additive white noise [1156]. A proper choice of coefficients{αi} induces that the matrixRx will be sym-metric positive definite. It should be noted, that matrices ˜Rx(pi) = [Rx(pi) +R^T_x(pi)]/2 are symmetric but not necessarily positive definite, especially for a large time delaypi.

The practical implementation of the algorithm for data corrupted by white noise, is given below [1156,90,236].

Algorithm Outline: Robust Orthogonalization

1. Estimate a set of time-delayed covariance matrices of sensor signals for preselected time delays (p1, p2, . . . , pK) and construct anm×mKmatrixR= [ ˜Rx(p1)· · ·R˜x(pK)], where ˜Rx(pi) = (­

x(k)x^T(k−p)® +­

x(k−p)x^T(k)® )/2.

Then compute the singular value decomposition (SVD) ofR, i.e.,

R=U Σ V^T, (4.99)

whereU= [Us,Uν]∈IR^m×m (withUs = [u1, . . .un]∈IR^m×n) and V∈IR^mK×mK are orthogonal matrices, andΣ is an m×mK matrix whose leftn columns contain diag{σ1, σ2, . . . , σn}(with non increasing singular values) and whose right (mK−n) columns are zero. The number of unknown sourcesn can be detected by inspecting the singular values as explained in the previous section under the assumption that the noise covariance matrix is modelled as Rν =σ_ν²Im and the variance of noise is relative low, i.e.,σ²_ν¿σ_n².

2. Fori= 1,2, . . . , K, compute

Ri=U^T_sR˜x(pi)Us. (4.100) 3. Choose any non-zero initial vector of parametersα= [α1, α2, . . . , αK]^T.

4. Compute

R= XK

i=1

αiRi. (4.101)

5. Compute the EVD decomposition ofRand check if Ris positive definite or not. If Ris positive definite, go to Step 7. Otherwise, go to Step 6.

6. Choose an eigenvectorucorresponding to the smallest eigenvalue⁸ ofRand update αvia replacing αbyα+δ, where

δ=

£u^TR1u· · ·u^TRKu¤T

k[u^TR1u· · ·u^TRKu]k. (4.102) Go to step 4.

7. Compute symmetric positive definite matrix Rx(α∗) =

XK

i=1

αiR˜x(pi), (4.103)

and perform SVD or symmetric EVD ofRx, Rx(α∗) = [US,UN]

· ΣS 0 0 ΣN

¸

[VS,VN]^T, (4.104) where (α∗) is the set of parametersαi after the algorithm achieves convergence, i.e., positive definiteness of the matrixR,US contains the eigenvectors associated withn principal singular values ofΣS = diag{σ1, σ2, . . . , σn}.

8. The robust orthogonalization transformation is performed by

x(k) =Q x(k), (4.105) whereQ=Σ⁻_S¹²U^T_S.

Some remarks and comments are now in order:

• The robust orthogonalization algorithm converges globally for any non-zero initial condition ofαunder assumption that all sources have different autocorrelation func-tions which are linearly independent or equivalently they have distinct power spectra.

Moreover, it converges in a finite number of steps [1156].

8If the smallest eigenvalues has some multiplicity, take any vectorucorresponding to the smallest eigenvalue.

IMPROVED BLIND IDENTIFICATION ALGORITHMS BASED ON EVD/SVD 151

• In the ideal noiseless case, the last (m−n) singular values ofRx(α) are equal to zero, thusΣN =0.

• In the case of m= n(equal numbers of sources and sensors), step 1 and 2 are not necessary. Simply, we letRi= ˜Rx(pi) = (Rx(pi) +R^T_x(pi))/2.

• Form > nthe linear transformationx(k) =Q x(k) besides orthogonalization enables us to estimate the number of sources, i.e., the orthogonalization matrix reduces the array of sensor signals to ann-dimensional vector, thus the number of sources can be estimated, under conditions that the SNR is relative high.

• By defining a new mixing matrix as A = Q HD^1/2, where D = P_L

i=1αiR˜s(pi) is a diagonal (scaling) matrix with positive entries, it is straightforward to show that AA^T =In, thus the matrixAis orthogonal. This orthogonality condition is necessary for performing separation of signals using the symmetric EVD or Joint Diagonaliza-tion approaches. It should be noted that in contrast to the standard prewhitening procedure for our robust orthogonalization generally E{x x^T}=Dx 6=In. We have x=A˜s+Q n, where ˜s=D^−1/2s, but due to the scaling indeterminacy of the sources, we may write in the sequel thatx=A s+ν˜ (˜ν=Qν). The diagonal elements of Dare positive, due to the positive definiteness ofRx(α) [458].

Several extensions and improvements of the above presented robust orthogonalization algorithm are possible, especially, if the noise is not completely white (i.e., the noise has white and colored components) and/or the number of available samples is relatively small.

First of all, instead of the simple shift (time delay) operator, we can use generalized delay operators or more generally suitably designed filters. In other words, instead of the standard time-delayed sampled covariance matricesRb_x(p_i) =­

x(k)x^T(k−p_i)®

, we can use the generalized sampled covariance matrices of the form

Rb_xxe(B_i) = 1 N

XN

k=1

x(k)ex^T_Bi(k), (i= 1,2, . . . , K) (4.106) where vectorxeBi(k) =Bi(z)[x(k)] =P

pbipx(k−p) is a filtered version of the vectorx(k) and Bi(z) denotes transfer function of a suitably designed filter or generalized time-delay operator⁹. It should be noted that in general any set of FIR (finite impulse response) or stable IIR (infinite impulse response) filter may be used in the preprocessing stage. However, we propose to use banks of bandpass filters possibly with overlapping band-passes covering a bandwidth of all source signals but with different central frequencies as is illustrated by Fig. 4.5. For example, we can use simple second-order IIR bandpass filters with transfer characteristics

Bi(z) =z^−qi(1−ri)(ωciz⁻¹/(ri+r²_i))−1

1−ωciz⁻¹+r²_iz⁻² , (4.107)

9In the simplest caseBi(z) = z⁻ⁱ. Generalized delay operator of the first-order has the following form B1(z) =^α+βz_1+γz−1⁻¹, whereα,βandγare suitably chosen coefficients.

(a) (b)

Fig. 4.5 Illustration of processing of signals by using a bank of bandpass filters: (a) Filtering a vector x of sensor signals by a bank of sub-band filters, (b) typical frequency characteristics of bandpass filters.

where ωci(k) = 2ricos(2πfcik) with the center frequency fci and the parameterri related to the frequency bandwidth by relationshipBwi = (1−ri)/2. The suitably designed bank of bandpass filters enables us to remove efficiently wide-band noise, which is out of the band-width of the source signals.¹⁰ If the bandwidth of source signals is known approximately, we can use onlyK=nbandpass filters with bandwidths consistent with the bandwidth of the sources.

In order to ensure the symmetry of the generalized covariance matrices, we will use the matrices defined as: Rexex(Bi) = (E{x(k)ex^T_Bi(k)}+E{exBi(k)x^T(k)})/2, where the vectors e

xB_i(k) = [ex1(k), . . . ,xem(k)]^T represent sub-band filtered versions of the vectorx(k).

For nonstationary source signals and stationary noise and/or interference, we can adopt an alternative approach, based on the concept of the differential correlation matrices, defined as [225]

δRx(Ti, Tj, pl) =Rx(Ti, pl)−Rx(Tj, pl), (4.108) where Ti and Tj are two not overlapping time windows of the same size and Rx(Ti, pl) denotes the time-delayed correlation matrix for the time window Ti. It should be noted that such defined differential time-delayed correlation matrices are insensitive to stationary signals. In order to perform robust orthogonalization for nonstationary sources, we divide the sensor datax(k) intoKnon-overlapping blocks (time windowsTi) and estimate the set of differential matrices δR˜x(Ti, Tj, pl) for i= 1, . . . , K, j > iand l = 1, . . . , M (typically, M = 5 andK= 10 and the number of samples in each block is 100).

10It is important that bandpass of the filters possibly match bandwidths corresponding to the highest energy of the individual sources.

IMPROVED BLIND IDENTIFICATION ALGORITHMS BASED ON EVD/SVD 153

In the next step, we formulate the composite differential matrix defined as δRx(α) =X

ijl

αijlδR˜x(Ti, Tj, pl) (i= 1,2, . . . , K; j > i; l= 1,2, . . . , M) (4.109) and using the approach described above, we can estimate the set of coefficients αijl for which the matrix δRx(α) is positive definite. In the last step, we perform the symmetric EVD of the positive definite matrixδRx(α∗) and compute the orthogonalization matrixQ (cf. Eqs. (4.101)-(4.105)).

For sensor signals corrupted by any Gaussian noise, instead of the time-delayed covariance matrices, we can use the fourth order quadricovariance matrices defined as:

Cx(p,Eq) = Cx{(x^T(k−p)Eqx(k−p))x(k)x^T(k)}

= E{(x^T(k−p)Eqx(k−p))x(k)x^T(k)} −Rx(p)EqR^T_x(p)

−tr(EqRx(0))Rx(0)−Rx(p)E^T_q R^T_x(p), (4.110) where Rx(0) =E{x(k)x^T(k)}=E{x(k−p)x^T(k−p)}, Rx(p) =E{x(k)x^T(k−p)} and Eq ∈ IR^n×n is any matrix, typically,Eq = Ior Eq =uqu^T_q, where up is the p-th vector of some orthogonal matrix U. ¹¹ Our objective is to find such a set of matrices Eq and time-delaypthat the quadricovariance matrix (4.110) (or linear combination of several such matrices) is positive definite.

4.3.2 Improved Algorithm Based on GEVD

On basis of robust matrix orthogonalization, we can develop several improved and extended algorithms based on the EVD/SVD or GEVD. In this section, we will discuss an improved algorithm based on GEVD or the matrix pencil proposed by Choi et al. [223,235].

The set of all matrices of the formR1−λR2(with some parameterλ) is said to be amatrix pencil. Frequently, we encounter the case whereR1is symmetric andR2 is symmetric and positive definite. Pencils of this variety are referred to as symmetric definite pencils[501].

Theorem 4.2 IfR1−λR2is a symmetric definite pencil (i.e. both matrices are symmetric andR2 is positive definite), then there exists a nonsingular matrixV= [v1, . . . ,vn] which performs simultaneous diagonalization of R₁ andR₂:

V^TR1V = D1, (4.111)

V^TR2V = D2, (4.112)

if the diagonal matrixD1D⁻¹₂ has distinct entries. Moreover, the problem can be converted to the GEVD:R1V=R2V Λ, whereΛ= diag{λ1, λ2, . . . , λn}=D1D⁻¹₂ (or equivalently R1vi=λiR2vi fori= 1, . . . , n), if all eigenvaluesλi= ^d_d^i(R1)

i(R2) are distinct.

11The matrixUcan be estimated by the EVD of the simplified contracted quadricovariance matrix forp= 0 andEq=IasCx(0,I) =Cx{(x^T(k)x(k))x(k)x^T(k)}=E{(x^T(k)x(k))x(k)x^T(k)} −2Rx(0)Rx(0)− tr(Rx(0))Rx(0) =U ΛIU^T.

It is apparent from Theorem 4.2 that R1 should be symmetric andR2 should be sym-metric and positive definite so that the generalized eigenvector V can be a valid solution (in the sense that Hb = (V^T)⁻¹) on the condition that all the generalized eigenvalues λi

are distinct. Unfortunately, for some time delays the covariance matricesRx(p1) =R1and Rx(p2) =R2cannot be positive definite. Moreover, due to some noise and numerical error they cannot be symmetric. Thus, we might have a numerical problem in the calculation of the generalized eigenvectors, which can be complex-valued in such cases [235,236].

Remark 4.6 In fact, the positiveness of matrix R2 =Rx is not absolutely necessary. If R1 andR2 are symmetric and R2 is not positive definite, then we can try to construct a positive definite matrix R3=β1R1+β2R2 for some choice of real coefficients β1, β2, and next to solve equivalent generalized symmetric eigen problem R1V=R3VΛ in the sense that the eigenvectors of the pencils R1−λR2 andR1−λR3are identical. The eigenvalues λi of R1−λR2 and the eigenvaluesλ˜i of R1−λR3 are related byλi=β2˜λi/(1−β1λ˜i).

Let us consider two time-delayed covariance matricesRx(p1) andRx(p2) for non-zero time lagsp1andp2. For the requirement of symmetry, we replaceRx(p1) andRx(p2) by ˜Rx(p1) and ˜Rx(p2) that are defined by

R˜x(p1) = 1 2

©Rx(p1) +R^T_x(p1)ª

, (4.113)

R˜x(p2) = 1 2

©Rx(p2) +R^T_x(p2)ª

. (4.114)

Then the pencil ˜Rx(p1)−λR˜x(p2) is a symmetric pencil. In general, the matrix ˜Rx(p2) is not positive definite. Therefore, instead of ˜R_x(p₂) for a single time delay, we consider a linear combination of several time-delayed covariance matrices:

Rx(α) = XK

i=1

αiR˜x(pi). (4.115)

The set of coefficients {αi} is chosen in such a way that the matrix Rx(α) is positive definite, as described in the previous section. Hence, the pencilRx(p1)−λRx(α) is a sym-metric definite pencil and its generalized eigenvectors are calculated without any numerical problem.

This method referred to asImproved GEVD (Matrix Pencil) Methodis summarized below [223,235].

Algorithm Outline: Improved GEVD (Matrix Pencil) Algorithm

1. Compute R1 = ˜Rx(p1) for some time lag p1 6= 0 (typically, p = 1) and calculate a symmetric positive definite matrix R2 =Rx(α) = P_K

i=1αiR˜_xex(pi) by using the robust orthogonalization method (employing a time-delay operator, bank of bandpass filters or differential correlation matrices).

IMPROVED BLIND IDENTIFICATION ALGORITHMS BASED ON EVD/SVD 155 2. Find the generalized eigenvector matrixVfor the generalized eigen value

decomposi-tion (GEVD)

R˜x(p1)V=Rx(α)V Λ. (4.116)

3. The mixing matrix is given byHb = (V^T)⁻¹ on the condition that all eigenvalues are real and distinct.

4.3.3 Improved Two-stage Symmetric EVD/SVD Algorithm

Instead of the GEVD approach, we can use the standard symmetric EVD or SVD in a two-stage (or more) procedure. Let us assume, that the sensor signals are corrupted by the additive white noise and the number of sources is generally unknown (with the number of sensors larger or equal to the number of sources).

Using a set of covariance matrices and the robust orthogonalization described above, we can implement the following algorithm.

Algorithm Outline: Robust EVD/SVD Algorithm

1. Perform a robust orthogonalization transformation x(k) = Q x(k) using one of the method described in previous section, such that the global mixing matrix A=Q H is orthogonal.

2. Compute the linear combination of a set of the time-delayed covariance matrices of the vectorx(k) for a set of time delayspi6= 0 (or alternatively using a bank of bandpass filters)

Rx(β) = XM

i=1

βiR˜x(pi), (4.117)

where a set of coefficientsβi can be randomly chosen.

3. Perform SVD (or equivalently EVD) as

Rx(β) =UxΣxU^T_x (4.118)

and check whether for the specific set of parametersβi and pi all singular values of the diagonal matrixΣx are distinct. If not, repeat step 2 and 3 for different set of parameters. If the singular values are distinct and sufficiently far away from each other then, we can estimate (unbiased by white noise) the mixing matrix as

Hb =Q⁺Ux (4.119)

and/or if necessary-estimate (noisy) colored source signals as

y(k) =bs(k) =U^T_xx(k) =U^T_xQ x(k). (4.120)

4.3.4 Blind Separation and Identification Using Bank of Bandpass Filters and Robust Orthogonalization

Instead of using the linear combination of a set of covariance matrices for various time delays, we can use a single generalized covariance matrixRe_xex= (E{x(k)x(k)e ^T}+E{x(k)e x^T(k)})/2, where the vector ex(k) = [xe1(k), . . . ,exn(k)]^T represents a filtered version of the vector x.

More precisely, each signal exj(k) =B(z)xj(k) =P

pbpxj(k−p) is a sub-bandpass filtered version of signalxj. It should be noted that all filters have identical transfer functionB(z) for each channel and each source signal should have a frequency range located, at least par-tially, in the bandwidth of the filters. Detailed implementation of the algorithm are given below.

Algorithm Outline: Robust SVD with Bank of Band-Pass Filters

1. Perform the robust orthogonalization transformation (x(k) =Q x(k)), for example, by computing the SVD of a symmetric positive definite matrix

Rxex(α) = XK

i=1

αiR˜xex(Bi) =USΣSV^T_S, (4.121) such that the global mixing matrix (A=Q H∈IR^n×n) is orthogonal.

2. Generate the vectorx(k) = [e ex1(k), . . . ,xen(k)]^T, defined as ex(k) = B(z)x(k) = P_L

p=1bpx(k−p), by passing the signalsxj(k) through the bandpass filterB(z). Es-timate next the symmetric generalized covariance matrix defined as

Re_xex= 1

2(E{x(k)x(k)e ^T}+E{ex(k)x^T(k)}). (4.122) 3. Perform the SVD (or equivalently the EVD) of the symmetric covariance matrixRe_xex

Re_xex=UxΣxU^T_x (4.123)

and check whether for a specific set of parameters of the filter (B(z) =P_L

p=1bpz⁻¹) all singular values of the diagonal matrixΣx are distinct. If not, repeat step 2 and 3 for the different set of parameters of filters. If the singular values are distinct and sufficiently far away from each other then, we can estimate (unbiased by the noise) the mixing matrix as

Hb =Q⁺Ux=US(ΣS)^1/2Ux (4.124) and/or the noisy source signals as

y(k) =bs(k) =U^T_xx(k) =U^T_x(ΣS)^−1/2U^T_Sx(k). (4.125)

JOINT DIAGONALIZATION - ROBUST SOBI ALGORITHMS 157

Dans le document Adaptive Blind Signal and Image Processing (Page 180-189)