On AoA estimation in the presence of mutual coupling: Algorithms and performance analysis

EURECOM

Department of Mobile Communications Campus SophiaTech

CS 50193

06904 Sophia Antipolis cedex FRANCE

Research Report RR-16-326

On AoA Estimation in the Presence of Mutual Coupling:

Algorithms and Performance Analysis

November 1^st, 2016

Ahmad Bazzi and Dirk. T.M. Slock

Tel : (+33) 4 93 00 81 00 Fax : (+33) 4 93 00 82 00 Email :{bazzi, slock}@eurecom.fr

1EURECOM’s research is partially supported by its industrial members: BMW Group Research and Technology, IABG, Monaco Telecom, Orange, Principaut de Monaco, SAP, ST Microelectron- ics, Symantec.

On AoA Estimation in the Presence of Mutual Coupling:

Algorithms and Performance Analysis

Ahmad Bazzi and Dirk. T.M. Slock

Abstract

This paper presents two novel methods for Angle-of-Arrival (AoA) es- timation in the presence of unknown mutual coupling. We mainly focus on the first method, where we show that this method could estimate AoAs of multiple sources when more mutual coupling parameters are included in the model, compared to existing methods. Furthermore, we derive a closed form Mean-Squared-Error (MSE) expression of the proposed method and com- pare it to the MSE of MUSIC with known coupling parameters. In addition, the derived MSE expression is also compared to theCram´er-Raobound of joint AoA and mutual coupling estimation in the asymptotic regime, i.e. at high signal-to-Noise ratio or large number of antennas. The second method serves as a refinement of the first one. Simulation results have demonstrated the potential of the proposed methods as they enjoy better performance than existing methods. A better description of the paper could be found in the Conclusions section.

Index Terms

Angle-of-Arrival estimation, Mutual Coupling, Mean-Squared-Error anal- ysis, MUSIC.

Contents

1 Introduction 1

1.1 Localization and Array Signal Processing . . . 1

1.2 Literature Review . . . 2

1.3 Contributions . . . 3

2 System Model 4 2.1 Problem Formulation . . . 4

2.2 Assumptions . . . 6

2.3 Problem Statement . . . 6

3 The MUSIC Algorithm and Related Work 7 3.1 Preliminaries . . . 7

3.2 Mutual Coupling in the sense of MUSIC . . . 7

3.3 Related Work . . . 9

4 The Proposed Mutual Coupling Agnostic Algorithm 10 4.1 Introductory Theorems . . . 10

4.2 Algorithm Derivation . . . 12

4.3 Properties off(θ) . . . 14

5 MSE Analysis 16 6 Comparison with the Cram´er-Rao Bound 20 6.1 Large Number of Antennas . . . 20

6.2 High SNR . . . 21

7 Refining the AoA estimates by alternating minimisation 21 8 Simulation Results 22 9 Conclusions 26 10 Appendix 26 10.1 Appendix A: Proof of Theorem 2 . . . 26

10.2 Appendix B: Proof of Theorem 3 . . . 27

10.3 Appendix C: Proof of Consequence 1 . . . 31

10.4 Appendix D:1^stand2^ndorder derivatives off(θ) . . . 32

10.5 Appendix E: Perturbation ofbλ1 andbvbvvb1 . . . 32

10.6 Appendix F: MSE Expression . . . 34

10.7 Appendix G: Proof of Theorem 6 . . . 34

List of Figures

1 Eigenvalues ofBBB^H(θ)BBB(θ)as a function ofθ for different values ofN andp. . . 43 2 The behaviour ofγ_kfor fixedp= 3as a function ofN. . . 44 3 The behaviour ofγkfor fixedN as a function ofp. . . 45 4 MSE of the proposed method in equation (49) for different values

ofpandθ₁. . . 46 5 Different normalized spectra (in dB) of methods that estimate AoAs

in the presence of mutual coupling. . . 47 6 Bias and MSE of the AoA estimates as a function of SNR for Ex-

periment 1 . . . 48 7 Bias and MSE of the AoA estimates as a function of SNR for Ex-

periment 2 . . . 49 8 Bias and MSE of the AoA estimates as a function of SNR for Ex-

periment 3 . . . 50 9 Error on coupling parameters as a function of SNR for the three

experiments . . . 51

1 Introduction

1.1 Localization and Array Signal Processing

Localisation, also known as location estimation, of multiple emitting sources by observing measurements at the output of an array of antennas arises frequently in signal processing applications, such as sonar, seismology, radar, and astron- omy [1, 2]. Indeed, locating multiple sources could be determined by processing the different signals at the output of the receiving array. This is so because the phase shifts across different antennas is a function of the angular position of the emitter [3, 4], known as the Angle-of-Arrival, or AoA for short. Once the AoA is known, say by two different array receivers, then one could apply techniques, like triangulation, to locate the source [5, 6]. Of course, there exists other signal pa- rameters that could determine the position of a source, such as the Time-of-Arrival (ToA) and Recieved Signal Strength (RSS). For more information on this topic, we encourage the reader to refer to [7]. Our main focus in this paper is on Angle-of- Arrival estimation using an array of antennas.

Throughout several decades, the problem of Angle-of-Arrival estimation has been seen as a specific problem of array signal processing and parameter estima- tion [8]. The Maximum Likelihood (ML) approach was one of the first to be inves- tigated [9]. The ML is optimal, in a sense that it minimizes the mean squared error, but it requires a multidimensional search over different AoAs. As a response, this issue attracted a number of researchers in order to find computationally efficient methods and, at the same time, optimise the ML cost function, such as the Iterative- Quadratic-ML (IQML) algorithm [10], the Alternating Projection (AP) method [11], the Expectation-Maximisation (EM) method [12], the Method-Of-Direction- Estimation (MODE) [13], and Space-Alternating-Generalised-EM (SAGE) [14].

These methods are ”much faster” than a multidimensional search, but might suffer from convergence issues. For instance, see [15, 16] for convergence properties on IQML, MODE, EM and SAGE.

Subspace methods are another class of algorithms, such as MUltiple SIgnal Classification, MUSIC [17], and Estimation of Signal Parameters via Rotational Invariance Techniques, ESPRIT [18]. These class of algorithms have attracted a numerous amout of researchers because they are computationally more attractive than the algorithms listed above, however this comes at the price of performance.

More specifically, MUSIC is a large sample realization of an ML estimator [19, 20]. The ESPRIT algorithm was also analyzed in [21, 22] and was reported to be statistically less efficient than MUSIC in [21]. Moreover, the performance of both MUSIC and ESPRIT deteriorate when the sources are highly correlated. As a result, spatial smoothing [23] was proposed as a remedy of highly correlated sources. In this paper, we focus on the MUSIC algorithm to solve the problem we are interested in.

1.2 Literature Review

In practical scenarios, an ideal model is, rarely, satisfied and modeling errors are the main reason behind this non-ideality. Moreover, it is well-known that differ- ent modeling errors could degrade the performance of direction-finding algorithms, when not taken into consideration [24, 25]. More specifically, these modeling er- rors induce a bias in the MUSIC estimator and could result in large mean-squared errors of AoA estimates, when the modeling errors are large enough [25]. These modeling errors include: antenna position uncertainty [26], unknown gain/phases between different antennas [27], and mutual coupling between antennas [28].

Mutual coupling is a popular problem in array signal processing. This phe- nomenon arises when antennas are close to each other [29], and thus the current developed in an antenna element depends on its own excitation and on the contri- butions from adjacent antennas. Methods that aim on solving the mutual coupling problem are sometimes referred to as calibration methods, which are of two types:

Offline and Online. In an offline calibration approach, one estimates the mutual coupling parameters using known locations [30–34]. For the sake of fairness, we should distinguish the paper in [34], since this method could calibrate the anten- nas in the presence of multipath. Offline calibration methods could estimate all modeling errors, thanks to the aid of known positions. However, one might argue that this approach is time-consuming and, in some scenarios, impossible to imple- ment. Other methods assume partly calibrated antennas [35–38] or partly known positions [39–41].

The other type of calibration methods, which is our main interest, are known as online (or auto-calibration) methods. One of the first papers to appear that deal with online calibration is [42]. This method is iterative and performs alternating min- imisation steps between the AoAs and mutual coupling parameters to optimize the MUSIC cost function. However, this method [42] doesn’t admit a unique solution (See [43] for the case of linear arrays and [44] for non-linear arrays). Moreover, an initial estimate of the coupling parameters are required. Other iterative methods we could mention are [45–48]. In [45], the convergence rate may be slow. Also, convergence is not always guaranteed. In [46], initial estimates are needed and this is not always available. The paper in [48] assume a diagonal source covariance ma- trix, i.e. the sources have no correlation. This is not always true. In addition, they treat the mutual coupling matrix and its conjugate-transpose as independent ma- trices. This might lead to some sub-optimality. Furthermore, they have reported, through simulations, that their algorithm needs 12 iterations to converge, and that global convergence is not guaranteed. Moreover, the methods in [49–52] add aux- iliary, or ”dummy” antennas, to solve the mutual coupling issue. For instance, in [52], antennas were added on the left and right edges of the array. Then, the main array, or the ”middle sub-array” of the total array, was used to estimate the AoAs. However, adding antennas is not possible in some scenarios. Also, the al- gorithms in [53–55] focus on processing the ”middle sub-array” of the main array, but this is sub-optimal since not all the antennas are being processed.

A RAnk-REduction estimator, also known as RARE, was first proposed in [56]

in the context of partly calibrated arrays. The same idea was used for totally un- calibrated Uniform Circular Arrays (UCA) in [57, 58] and Uniform Linear Arrays (ULA) in [59, 60]. This method makes use of the MUSIC algorithm to estimate AoAs in the presence of mutual coupling via rank reduction of an appropriate ma- trix. The method in [60] is a Recursive-RARE (R-RARE), which was shown to achieve better performance than the traditional RARE. A similar rank-reduction approach was adopted in [61]. Moreover, the algorithm in [62] is based on a min- imum eigenvalues instead. In addition, the method in [63] formulates the problem through a quadratic minimisation problem. We shall elaborate on those algorithms in Section III.

1.3 Contributions

Our main contributions are summarized as follows.

• We present some introductory algorithms that turn out to be essential for the bulk of this paper. The theorems lead to a consequence for ULAs that suffer from mutual coupling.

• Using these theorems, we derive an algorithm that is able to estimate AoAs when more mutual coupling parameters are present in the model.

• We derive an asymptotic Mean-Squared-Error expression and compare it to the appropriateCram´er-Raobound.

• Another algorithm is introduced to further refine the AoA estimates by solv- ing a multidimensional problem by alternating minimisation.

• Simulation results have demonstrated the potential of the proposed algo- rithms, which perform better than existing algorithms.

The remainder of the paper is divided as follows: Section 2 presents the system model used throughout the paper. Section 3 revises the MUSIC algorithm, with and without mutual coupling. Also, we summarize some related methods that could estimate AoAs in the presence of mutual coupling. We derive and state some properties of the proposed algorithm that could estimate the AoAs in the presence of mutual coupling in Section 4. The Mean-Squared-Error (MSE) of the proposed algorithm is derived in Section 5, where we compare the MSE expression to the MSE of MUSIC withknownmutual coupling. We also compare the derived MSE expression to theCram´er-Raobound in Section 6. Furthermore, Section 7 presents a refinement of the proposed method that could further enhance the AoA estimates.

Some simulation results are demonstrated in Section 8. We conclude the paper in Section 9. The Appendix of the paper is given in Section 10.

Table 1: Nomenclature

C^M×N Class of allM×N complex-valued matrices C^∗ Class of all complex-valued numbers except zero A

AA^T Transposition of matrixAAA

AAA^∗ Conjugatation of matrixAAA

A

AA^H Conjugate-transposition of matrixAAA

A

AA⁺ Moore-Penrosepseudo-inverse of matrixAAA Re

AA

A Real part of matrixAAA

rankAAA Rank of matrixAAA

kAAAk Frobeniusnorm of matrixAAA

AAA

i,j The(i, j)^thelement of matrixAAA III The identity matrix with appropriate dimensions J

JJ_k An all-zero matrix inC^k×kexcept ones at its anti-diagonal 0

00 The zero matrix with appropriate dimensions AAABBB TheHadamardproduct ofAAAandBBB N AAA) The null-space ofAAA, i.e.xxx∈ N AAA)ifAAAxxx= 000

eee_i Thei^thcolumn ofIII

X=⇒Y IfXis true, thenY is true

X⇐⇒Y StatementsXandY are equivalent

E{X} Expectation of a random variableX

R AAA, xxx

Rayleigh quotient,^xxx_xxx^HH^AAAxxxx^xx

1k k×1vector of all-ones

δ_i,j TheDiracdelta (Ifi=j, thenδ_i,j = 1, else0)

|z| Magnitude of complex numberz

CN(ννν,ΣΣΣ) Complex Gaussian of meanνννand covariance matrixΣΣΣ

2 System Model

This section formulates the signal model and presents the assumptions used throughout the paper. Then, the problem of mutual coupling is addressed.

2.1 Problem Formulation

Consider a Uniform Linear Array (ULA) ofN antennas. Furthermore, assume q < Nnarrow-band sources, centered around a known frequency, sayf_c, attacking the array from different angles,ΘΘΘ = [θ1. . . θq]. Since narrow-bandedness in the context of array processing means that the propagation delays of the signals across the array are much smaller than the reciprocal of the bandwidth of the signals, it follows that these propagation delays translate into phase shifts that depend on the location.

Now following [1], the received analog signal across all antennas, in the ab- sence of mutual coupling, could be written as

x x x(t) =

q

X

i=1

a

aa(θ_i)s_i(t) +www(t) (1) where

xxx(t) =

x₁(t). . . x_N(t)T

(2) is the received vector across all antennas at timet. Moreover, the vectoraaa(θ) is referred to as the ”steering vector” of the array towards angleθ. It is this vector that allows us to perform Angle-of-Arrival (AoA) estimation, and is given by

a a a(θ) =

1, z_θ, . . . z_θ^N−1T

(3) wherezθ =e^{−j2πdλsin(θ)},dis the inter-element spacing andλis the signal’s wave- length. Moreover, the signalsi(t)is the signal emitted by thei^thsource at timet and

ww w(t) =

w1(t). . . wN(t)T

(4) is background noise across all antennas at timet. Equation (1) could be written in a more compact way as follows

x x

x(t) =AAA(ΘΘΘ)sss(t) +www(t) (5) whereAAA(ΘΘΘ)∈C^N×qis referred to as ”steering matrix” and is given as

A

AA(ΘΘΘ) =

aaa(θ₁). . . aaa(θ_q)

(6) andsss(t)∈C^q×1is the vector of transmitted signals, viz.

ss s(t) =

s1(t). . . sq(t)T

(7) Finally, sampling (5) atLtime instances, sayt={0, T, . . . ,(L−1)T}, whereT is the sampling period, we get

XXX =AAA(ΘΘΘ)SSS+WWW (8) whereXXX = [xxx(0), xxx(T), . . . , xxx (L−1)T

] ∈ C^N×L is the data collected over the observed interval of time. MatricesSSS ∈ C^q×LandWWW ∈ C^N×Lare similarly defined.

Equation (8) assumes an ideal model, in the sense that each antenna acts inde- pendently of all the others. In reality, the current developed in an antenna element depends on its own excitation and on the contributions from adjacent antennas.

As a consequence, an ideal model is no longer valid. This phenomenon is called

”Mutual Coupling” between array elements, and it enters the model as follows [42]

X¯¯ X¯

X =TTT(ccc)AAA(ΘΘΘ)SSS+WWW (9)

whereTTT(ccc)∈ C^N×N captures the effect of mutual coupling, and is known as the

”Mutual Coupling Matrix” (MCM). Due to the linear and uniform configuration of the different elements of the array, the MCMTTT(ccc) is given by a symmetric Toeplitz matrix. Letc_i be the coupling coefficient between two elements placedi inter-element spacings apart. Since the amplitude of the coupling parameters tend to decay as a function of increasing distance, namely

1>|c₁|> . . . >|c_N−1| (10) then a well-approximation ofTTT(ccc)is a banded symmetric Toeplitz matrix [66] with bandwidthp, i.e.

TT T(ccc)

i,j =

(c_|i−j| if|i−j|< p

0 else (11)

In other words, antennas that are placed at least p inter-element spacings apart do not interfere, i.e. c_i = 0for alli ≥ p. In what follows, the MCM of a ULA configuration is modelled as banded symmetric Toeplitz matrix of bandwidthpand denoted asTTT(ccc), whereccc= [1, c₁. . . cp−1]^Tis the vector of coupling parameters.

2.2 Assumptions

Before the problem is clearly addressed, some assumptions have to be made so as to proceed. They are given as follows:

• Assumption 1: The Angles-of-Arrival of theq sources are distinct, namely θ_i 6=θ_j fori6=j.

• Assumption 2: The processessss(t)andwww(t)are ergodic and stationary over the observed interval of time.

• Assumption 3: The signalssss(t)are not coherent, i.e. they are not fully cor- related.

• Assumption 4: The number of signalsqand the bandwidth ofTTT(ccc), which is p, are known.

• Assumption 5: The noise www(lT) is modelled as a white circular complex Gaussian process of zero mean and covarianceσ²III and independent from the source signals.

2.3 Problem Statement

We are now ready to address our Mutual Coupling problem:

”GivenXXX,¯¯¯ q, andp, estimate the angles-of-arrivalΘΘΘof the incoming signals in the presence of mutual couplingTTT(ccc).”

3 The MUSIC Algorithm and Related Work

3.1 Preliminaries

This subsection serves as a review of the MUSIC algorithm, in the absence of mutual coupling. In other words, the model in equation (8) is assumed. The covariance matrix of the received data could be written as

R_xx Rxx

R_xx,E{xxx(t)xxx^H(t)}

=AAA(ΘΘΘ)RRR_ssss_ssAAA^H(ΘΘΘ) +σ²III (12) where the second equality is due toAssumption 5and

R_ss

RR_ssss,E{sss(t)sss^H(t)} (13) is the source covariance matrix. Using spectral decomposition, the matrixRRR_xxxx_xx is expressed as

RRR_xxxxxx = U_s Us

U_s UUU_nn_n

ΣΣΣsss 000 000 ΣΣΣn_nn

UUU_ss_s UUU_nn_n H

=UUUs_ssΣΣΣs_ssUUU_sss^H+UUUn_nnΣΣΣn_nnUUU_nnn^H

(14) The partitioning in equation (14) is done becauseRRR_xxxx_xx is composed of two ma- jor parts: Signal and Noise. The signal part AAA(ΘΘΘ)RRRss_ssssAAA^H(ΘΘΘ) is rank q, under Assumptions 1and 3. Therefore, due to the noise part σ²III, one can say thatΣΣΣs_ss

is aq ×q diagonal matrix composed of eigenvalues strictly greater thanσ² and Σn

ΣΣ_nn = σ²IIIN−q. The eigenvectorsUUUs_ss andUUUn_nnare, often, referred to as the signal andnoise subspaces, respectively.

In the absence of mutual coupling, the key to MUSIC is the following observation:

kUUU_nnn^Haaa(θ)k² = 0 =⇒θ∈ΘΘΘ (15) In practice, one could estimate the covariance quantities through sample averaging, viz.

RRRbbbxxxxxx = 1

LXXXXXX^H=UUUbbbsssΣΣbΣbbsssUUUbbb_sss^H+UUUbbbnnnΣΣΣbbbnnnUUUbbb_nnn^H (16) MUSIC estimates the angles-of-arrivalΘΘΘthrough peak finding, as follows

{bθi}^q_i=1= arg max

θ

1

aaa^H(θ)UUUbbb_nn_nUUUbbb_nnn^Haaa(θ) (17) 3.2 Mutual Coupling in the sense of MUSIC

The previous subsection tells us that one can estimate the angles-of-arrival in the absence of mutual coupling by performing a 1D-search according to equa- tion (17). Now, for the ease of exposition, let¯a¯a¯a(θ)denote the steering vector in the presence of mutual coupling, i.e.

¯ a

¯ a¯

a(θ) =TTT(ccc)aaa(θ) (18)

Similarly, defineAAA(Θ¯¯¯ ΘΘ)as follows A¯¯

AA(Θ¯ ΘΘ) =TTT(ccc)AAA(ΘΘΘ) = [¯aa¯¯a(θ1). . .¯a¯aa(θ¯ q)] (19) Taking into account mutual coupling, i.e. the model in equation (9), one could follow the same steps from equation (12) till (16) in order to say that the angles- of-arrival could be estimated as follows

{bθi}^q_i=1= arg max

θ

1

¯

a¯a¯a^H(θ)UUUbbb_nn_nUUUbbb_nnn^H¯a¯a¯a(θ) (20) whereUUUbbb_nn_nis the sample estimate ofUUU_nn_n. Throughout the rest of this paper,UUU_nn_nis the noise subspace, namelyUUUn_nnUUU_nnn^H=PPP_AAA^⊥⊥⊥_¯¯¯ =III−PPPAAA¯¯¯, where

PA¯

PPAA¯¯= ¯AAA( ¯¯¯AAA¯¯^HAAA)¯¯¯ ⁻¹AAA¯¯¯^H (21) However, applying MUSIC directly as in equation (20) to the problem in hand is not possible, since the functional form of the steering vector is not known. In other terms, we have partial knowledge of vector¯aa¯¯a(θ), which is that it is a knownVan- dermondevectoraaa(θ) pre-multiplied by an unknown banded symmetric Toeplitz matrixTTT(ccc), as in equation (18). Nevertheless, MUSIC implies the following

kUUU_nnn^HTTT(mmm)aaa(θ)k² = 0 =⇒ {θ∈ΘΘΘandmmm=ccc} (22) In order to proceed, we find the following theorem useful:

Theorem 1: Let ααα = [α₀, α₁. . . αp−1]^T ∈ C^p×1 andaaa ∈ C^N×1. Define the corresponding matrixTTT(ααα). Then the following is true for any1≤p≤N

T

TT(ααα)aaa=BBB_pααα (23) where

B B

Bp ,G_p(aaa) =

aaa SSS1aaa . . . SSSp−1aaa

(24) andSSS_k∈C^N×N is an all-zero matrix except at thek^thsub- and super-diagonals, which are set to 1.

Proof. See [42, 63].

Using this theorem, we can say that

¯

a¯a¯a(θ) =TTT(ccc)aaa(θ) =BBB(θ)ccc (25) where

BBB(θ) =G_p(aaa(θ)) (26) Therefore, equation (22) could be re-written as

kUUU_nnn^HBBB(θ)mmmk² = 0 =⇒ {θ∈ΘΘΘandmmm=ccc} (27)

Said differently and in a more compact way, equation (27) also means







UUU_nnn^HBBB(θ1) ... UUU_nnn^HBBB(θq)





mmm

2

= 0 =⇒mmm=ccc (28)

Therefore, one way to formulate the problem is (P₁) : min

mmm,θ¯1...θ¯q

mmm^HSSS(¯bbb θ1. . .θ¯q)mmm (29) where

SSS(θbbb ₁. . . θ_q) =





 Ubb

UUb_nnn^HBBB(θ1) ... UUUbbb_nnn^HBBB(θq)







H



 Ubb

UUb_nnn^HBBB(θ1) ... UUUbbb_nnn^HBBB(θq)





=

q

X

j=1

Kbb Kb

K(θ_j) (30)

where

Kbb

KK(θ) =b BBB(θ)UUUbbb_nn_nUUUbbb_nnn^HBBB(θ) (31) Assuming true subspaces (i.e. UUUbbbn_nn =UUUn_nn) and excluding the trivial solutionmmm = 000, it is clear that one solution of problem (P₁) is attained when mmm = ccc and [¯θ₁. . .θ¯_q] = [θ₁. . . θ_q] = ΘΘΘ. Said differently, S(ΘΘΘ)admits a null space of di- mension 1 spanned by the vector of coupling parameters,ccc.

In any case, this is a multidimensional problem in the AoA parameters, and a num- ber of papers have resorted to an alternative and sub-optimal problem, namely

(P₂) : min

mmm,θmmm^HKKK(θ)mbbb mm (32) The sub-optimality here has a nice interpretation: It is ”as if” the coupling pa- rameters are treated to be angular-dependent and therefore, one does not acknowl- edge that the vector of coupling parametersccc is fixed for anyθ. Consequently, the objective function in(P₂)would have been a reasonable choice if the coupling parameters are a function ofθ, i.e. ccc = ccc(θ). Surprisingly, a problem involving angular-dependent coupling parameters suggests a computationally less optimisa- tion problem in terms of the AoA parameters.Indeed, this approach is sub-optimal whencccis independent ofθ.

3.3 Related Work

As mentioned in the introduction, some methods make use of the MUSIC algo- rithm to estimate the AoAs of multiple sources in the presence of mutual coupling.

We provide a brief review of these algorithms in this subsection. The RARE algo- rithm in [56, 59] maximizes the following cost function through peak searching

f_RARE(θ) = 1 det

Kbb Kb

K(θ) (33)

The peaks of the RARE cost function are estimates of the AoAs. Indeed, this is a reasonable cost function, since equation (27) tells us thatKKK(θ)admits a null space whenθ∈ΘΘΘ. The identifiability conditions of RARE is:{p+q ≤N andp≤ ^N₂} (See [60]). This condition should be distinguished from the one in [56], since the RARE method developed there was for partly calibrated arrays. The paper in [60]

develop a Recursive-RARE algorithm to optimize problem(P₁)via multiple 1-D searches. They also assumep ≤ ^N₂. Furthermore, the method in [61] is similar to the RARE cost function, in the sense that a determinant criterion is used, however it involves a different type of matrix. Nevertheless, the identifiability conditions of the method in [61] is: {2p+q ≤ N + 1}. It should be noted that this condition also implies thatpcould not exceed ^N₂. In addition, the method in [62] propose to maximize the following cost function

f_MinEig(θ) = 1 λmin

KKK(θ)bbb (34) whereλ_min stands for the minimum eigenvalue. Whether stated in [62], or not, the above cost function is a solution of(P₂)under a norm constraint onmmm. More- over, a linear constraint (LC) was imposed onmmm, i.e.mmm^Heee1₁1 = 1to optimize(P₂) in [63], which gives the following cost function

f_LC(θ) =eee^H₁^H₁^H₁KKKbbb⁻¹(θ)eee1₁1 (35) The identifiability conditions for this method is similar to the one in RARE. To the best of our knowledge, the reason of conditionp ≤ ^N₂ is not so clear through the literature. One aspect of this paper is to have a clear understanding on why this inequality has to be satisfied for all the stated algorithms (seeTheorem 3and Consequence 1). Furthermore, we propose an algorithm that is able to estimate the AoAs of multiple sources in the presence of mutual coupling, even whenp > ^N₂.

4 The Proposed Mutual Coupling Agnostic Algorithm

This section presents the algorithm that could estimate the angles-of-arrival of multiple sources, in the presence of mutual coupling. The algorithm is based on optimising the cost function given in (P₂), under a suitable constraint. The constraint is based on the following theorems:

4.1 Introductory Theorems

Theorem 2: Letααα_p = [α₀, α₁. . . αp−1]^Tandaaa= [1, z . . . z^N−1]^T. Define the corresponding matrixTTT(αααp). Then for any1≤p≤N, the following holds

T

TT(ααα_p)aaa=g(z, ααα_p)aaa−MMM_pαααe_p (36)

where the polynomialg(z, ααα)is given by

g(z, αααp) =α0+

p−1

X

k=1

α_k(z^k+z^−k) (37) The matrixMMM_p ∈C^N×(p−1)is defined as

M MM_p =

UUUp

0 0 0

+

000 z^N−1JJJp−1UUU^∗_p

(38) whereUUUp ∈C(p−1)×(p−1)is written as

UUUp

i,j =

(z^−(j−i+1) ifj ≥i

0 else (39)

andαααe_p = [α₁, α₂. . . αp−1]^T. Proof. See Appendix A.

Theorem 2is key toTheorem 3, which comes next:

Theorem 3: Letaaa = [1, z . . . z^N−1]^T andBBB_p = G_p(aaa). Then, BBB_p has the fol- lowing spectral characteristics:

1. Ifp≤ ^N+1₂ , thenBBB_pis full column rank.

2. Ifp = ^N+2₂ andzis an N^thunit root (i.e. z^N = 1) thenrank(BBB_p) = ^N₂. The null space is given in equation(121). Otherwise, it is full column rank.

3. We distinguish 2 cases whenp >^N₂⁺²: (a) N is even:

i. Ifz^N 6=±1, thenBBB_pis full column rank.

ii. Ifz^N =−1, thenrank(BBBp) = ^N₂ + 1. The null space is given in equation(140).

iii. Ifz^N = 1, thenrank(BBB_p) = ^N₂. The null space is given in equa- tion(144).

(b) N is odd:

i. Ifz^N = ±1, thenrank(BBB_p) = ^N+1₂ . The null space is given in equation (145).

ii. Otherwise,BBBpis full column rank.

Proof. See Appendix B.

For ULA configurations, a direct consequence ofTheorem 3is the following:

Consequence 1: For ULA type configurations, i.e. aaa(θ) = [1, z_θ, . . . z_θ^N−1]^T with z_θ =e^{−j2πλdsin(θ)}. Define the following sets

ΘΘΘ₊+₊=n

sin⁻¹(kλ

N d), k=−N 2 . . .N

2 o

(40) ΘΘΘ−−−=n

sin⁻¹((k+ ¹₂)λ

N d ), k=−N 2 . . .N

2 o

(41) ΘΘΘ±±±=n

ΘΘΘ₊+₊∪ΘΘΘ−−−

o

(42) The matrixBBB(θ) =G_p(aaa(θ))has the following characteristics:

• Ifp < ^N+2₂ , the matrixBBB(θ)is full column rank.

• Whenp≥ ^N+2₂ , we distinguish the following cases:

– IfN is even andθ∈ΘΘΘ+₊+, thenrank(BBB(θ)) = ^N₂. – IfN is even andθ∈ΘΘΘ−−−, thenrank(BBB(θ)) = ^N₂ + 1.

– IfN is odd andθ∈ΘΘΘ±±±, thenrank(BBB(θ)) = ^N₂⁺¹. – ElseBBB(θ)is full column rank.

Proof. See Appendix C.

One purpose of this paper is to understand the behaviour of matrixBBB(θ)as function ofθ. Let ν₁ ≤ ν₂ ≤ . . . ≤ ν_p be the eigenvalues ofBBB^H(θ)BBB(θ). In order to partially verifyConsequence 1, we have depicted two figures wherep > ^N+2₂ . In Fig. 1a, we fixN = 8(even) and p= 7. The red and green dashed vertical lines correspond to angles inΘΘΘ₊+₊andΘΘΘ−−−, respectively. Observe that whenθapproaches angles inΘΘΘ+₊+, we have three eigenvalues, i.e.ν1,ν2, andν3, dropping to zero. This implies that, whenθ∈ΘΘΘ+++, the rank ofBBB(θ)isp−3 = 4 = ^N₂. However, when whenθ∈ΘΘΘ−−−, only two eigenvalues, namelyν₁andν₂, go to zero. In this case, the rank ofBBB(θ)isp−2 = 5 = ^N₂ + 1. Also note thatν4is strictly positive. In Fig. 1b, we fixN = 9(odd) andp= 8. Again,ν4 is strictly positive. Whenθ∈ΘΘΘ±±±, three eigenvalues go to zero, implying that the rank ofBBB(θ)isp−3 = 5 = ^N+1₂ . 4.2 Algorithm Derivation

The previous subsection reveals an important phenomenon of matrix BBB(θ).

According toConsequence 1, ifθ_k∈ΘΘΘ±±±andp≥ ^N₂⁺², thenBBB(θ_k)admits a null- space. Therefore, optimising the cost function given in (P₂), without choosing an appropriate constraint, gives false AoAs. Mathematically speaking, the cost

function in(P₂)is exactly zero for allθ_k ∈ΘΘΘ±±±whenp ≥ ^N₂⁺². To circumvent this issue, we form the following optimisation problem







minimize

m m

m,θ mmm^HKKK(θ)mbbb mm subject to eee₁^HBBB(θ)mmm= 1

(43) It is easy to see that, for anyθ, the trivial solutionmmm = 000and the vectors that lie in the null space ofBBB(θ)(i.e. BBB(θ)mmm = 0) are not feasible solutions because they do not satisfy the constraint. Therefore, optimising the above problem will exclude the latter false solutions.

TheLagrangianfunction corresponding to the optimisation problem in (43) is the following:

L(mmm, ν) =mmm^HKKK(θ)mbbb mm−ν eee₁^HBBB(θ)mmm−1

(44) Setting the derivative ofL(mmm, ν)with respect tommmto 0, we get

∂

∂mmmL(mmm, ν) = 2KKK(θ)mbbb mm−νBBB^H(θ)eee₁= 0 (45) Equation (45) gives the optimal coupling parameters,mmm^ooo, for a givenθ, in terms of the optimal Lagrangian multiplierν^oas

m^o mm^oo = ν^o

2KKKbbb⁻¹(θ)BBB^H(θ)eee1 (46) It is easy to prove that

B B

B^H(θ)eee₁ =aaa^∗_p(θ) (47) whereaaa_p(θ) is a p×1 vector defined as in equation (3). The expression of ν^o is obtained by plugging equations (46) and (47) in the constraint of the problem in (43), viz.

ν^o= 2

aaa^T_p(θ)KKKbbb⁻¹(θ)aaa^∗_p(θ) (48) Therefore,mmm^ooois now given as

m^o

mm^oo = KKKbbb⁻¹(θ)aaa^∗_p(θ) a

aa^T_p(θ)KKKbbb⁻¹(θ)aaa^∗_p(θ) (49) Substitutingmmm^ooo in the objective function of (43), theq AoAs could be estimated as follows

θbk

q

k=1 = arg min

θ

1

f(θ) (50a)

where

f(θ) =aaa^T_p(θ)KKKbbb⁻¹(θ)aaa^∗_p(θ) (50b)

Note thatKKKbbb(θ) is not invertible for the cases given inConsequence 1and when θ∈ΘΘΘat infinite SNR. For that, we adopt diagonal loading as done in [64], namely

KK

K(θ) +III−1

, where >0is small. Additionally, it has been mentioned in [64]

that there is generally no known method for determining the optimal value of, and it is usually determined experimentally. We have found that= 10⁻¹⁴serves as a good value.

4.3 Properties off(θ)

For a better understanding of the behaviour of the cost function given in equa- tion (50), we reveal some of its properties

Property 1: Forp= 1, i.e. no mutual coupling, the functionf(θ)”boils down”

to the traditional MUSIC cost function in equation(17).

Proof. Trivial.

Property 2: This property characterizes the null space ofKKK(θ)forp+q ≤N as a function ofθ

N(KKK(θi)) =















{000}, ifθ_i6∈ΘΘΘ∪ΘΘΘ±

{000∪ccc}, ifθ_i∈ΘΘΘandθ_i 6∈ΘΘΘ±

N BBB(θi)

, ifθi6∈ΘΘΘandθi ∈ΘΘΘ±

N BBB(θ_i)

∪ {ccc}, ifθ_i∈ΘΘΘ∩ΘΘΘ±

(51)

Note that ifp < ^N₂⁺², thenN BBB(θi)

={000}. Also note that this property assumes true subspaces, i.e.KKK(θ) =bbb KKK(θ) =BBB^HHH(θ)UUU_nn_nUUU_nnn^HBBB(θ).

Proof.

• The first two cases are a direct consequence of equation (27).

• The third case is a result ofConsequence 1and equation (27).

• As for the fourth case, assume that the setsΘΘΘandΘΘΘ± overlap and ^N+2₂ ≤ p < N. Letθ_i ∈ΘΘΘ∩ΘΘΘ±. Therefore,KKK(θ_i)ααα= 000only whenααα∈ N BBB(θ_i) orααα =ccc. It suffices to prove that the setcccis linearly independent from the span ofN BBB(θi)

.

Let∆be the dimension ofN BBB(θi)

. Furthermore, letγγγ ∈C^(∆+1)×1be an arbitrary vector andEEE ∈ C^p×(∆+1)be a matrix where the first∆columns spanN BBB(θ_i)

and the last column is the vectorccc. It remains to show that EEEγγγ = 000⇒γγγ= 000. Under the assumption thatp < Nand using the structure

of the null space ofBBB(θ_i)given in equations (121), (140), (144), and (145), one could easily verify that the second row ofEEEis given as

[0. . .0

| {z }

∆

, c1] (52)

which implies that the last element ofγγγ is0, since by constructionc₁ 6= 0, for p ≥ 2. Hence, γγγ = 000 because the first ∆columns ofEEE are linearly independent.

Property 3: Assuming true subspaces (i.e. UUUbbbn_nn = UUUn_nn), the function f(θ) is bounded whenθ6∈ΘΘΘand unbounded whenθ∈ΘΘΘ.

Proof. Let the functionf(θ)be defined as follows:

f(θ) =aaa^T_p(θ) KKK(θ) +III−1

aaa^∗_p(θ) (53) and therefore

lim→0f(θ) =f(θ) (54) By spectral decomposition,

KKK(θ) =VVVΦΦΦVVV^H (55) where thek^thcolumn ofVVV is thek^thnormalized eigenvector¹ofKKK(θ), denoted as vvvkand its corresponding eigenvalue is thek^thsmallest eigenvalue found in thek^th diagonal entry ofΦΦΦ, denoted asλ_k. We could then expressf(θ)as

f(θ) =aaa^T_p(θ)VVV ΦΦΦ +III−1

V V

V^Haaa^∗_p(θ) (56)

• Whenθ6∈ΘΘΘ, we distinguish two sub-cases:

– Ifθ 6∈ΘΘΘ±, thenKKK(θ)is full rank according toProperty 2and hence λ_k>0for allk, so

f(θ) =kΦΦΦ^−1/2VVV^Haaa^∗_p(θ)k² <∞ (57) – Ifθ ∈ ΘΘΘ±±±, thenKKK(θ)is full rank (ifp < ^N₂⁺²) and the preceeding argument holds. However, ifp ≥ ^N₂⁺², thenKKK(θ) admits the same null-space as that ofBBB(θ)according toProperty 2. As before, let∆be the dimension ofKKK(θ), thereforef(θ)behaves as

f(θ)v

∆

X

k=1

1

λ_k+kaaa^T_p(θ)vvv_kk²=

∆

X

k=1

1

keee₁^TBBB(θ)vvv_kk² (58) Note that{vvv_k}^∆_k=1span the null space ofBBB(θ)and thereforeBBB(θ)vvv_k= 000. So,f(θ) =f(θ) = 0<∞.

1Indeed,VVV andΦΦΦare functions ofθ. This is omitted for the sake of compact exposition.

• Whenθ∈ΘΘΘ, we also distinguish the same sub-cases:

– If θ 6∈ ΘΘΘ±, then there is only one singularity in KKK(θ) according to Property 2, i.e.λ1 = 0,vvv1= _kccck^ccc , andλ_k>0for allk≥2. Hence

f(θ)v 1

kaaa^T_p(θ)ccck²

kccck² (59)

Notice that the termaaa^T_p(θ)ccc is a polynomial of degree p−1 evalu- ated at the unit circle. For a polynomial with non-zero coefficients to have zeros on the unit-circle, the coefficient vectorcccmust be conjugate- symmetric [68], which is not the case according to equation (10). There- fore,aaa^T_p(θ)ccc6= 0and thus

f(θ) = lim

→0f(θ) =∞ (60)

– Ifθ ∈ΘΘΘ±±±, then the null space ofKKK(θ) is spanned by∆ + 1 vectors given inProperty 2, and we have

f(θ)v

∆

X

k=1

1

keee₁^TBBB(θ)vvv_kk²+1

kaaa^T_p(θ)ccck²

kccck² (61) Using the same argument as before, asgoes to zero, the1^st term of the above expression goes to zero, whereas the2^nd term goes to∞.

Property 4: The condition so that f(θ) uniquely identifies the AoAs is that p+q≤N.

Proof. This is so because the cost function in equation (50) depends on the inver- sion ofKKK(θ). Hence, in the case whereθ 6∈ΘΘΘ∪ΘΘΘ±±±, the matrixUUU_nnn^HBBB(θ)is full column rank whenp ≤ N −q. As for the case when θ ∈ΘΘΘ∪ΘΘΘ±±±, we have the argument inProperty 3.

Remark: The existing methods in equations (33), (34), and (35) can not iden- tify the true AoAs, when the number of coupling parametersp > ^N₂. According to Property 2, the cost functions of these exsiting methods would yield peaks when- everθ∈ΘΘΘ±±±andp > ^N₂. One could not, simply, remove these peaks because they would affect the estimation, when the true AoAs are close to those inΘΘΘ±±±.

5 MSE Analysis

It is well known that the noise subspace could be decomposed into two parts:

Ubb_n Ubn

U_n=UUU_nn_n+UUUeee_nn_n (62)

where the first part,UUU_nn_n, is the true noise subspace and the second one,UUUeee_nn_n, is the er- ror term. Using this decomposition and other asymptotic properties (i.e. for large Lor high SNR) which will appear in this section, we wish to derive an asymp- totic MSE expression for the AoA estimates obtained from equation (50). In other words, we seek an asymptotic expression ofE

(θek)² , whereeθkis the error part θb_k =θ_k+θe_k (63) Since

θb_k ^q_k=1are minimum points off⁻¹(θ), then

∂f⁻¹(θbk)

∂θ , ∂f⁻¹(θ)

∂θ θ=bθk

= 0 (64)

As done in [19], sinceθb_kis an estimate ofθ_k, we could, asymptotically, expand the above derivative in the neighborhood of the trueθkusingTaylorseries

∂f⁻¹(bθ_k)

∂θ = ∂f⁻¹(θ_k)

∂θ + ∂²f⁻¹(θ_k)

∂θ² (bθ_k−θ_k) +. . . (65) which gives an approximate expression of the errorθek =θbk−θk

θe_kw−

∂f⁻¹(θk)

∂θ

∂²f⁻¹(θk)

∂θ²

=− f⁰(θ_k) f⁰⁰(θk)−2^(f_f(θ^0(θk))2

k)

(66)

wheref⁰(θk)andf⁰⁰(θk)are the1^stand2^ndorder derivatives off(θ)evaluated at pointθ_k, respectively.

Property 5: The derivativesf⁰(θ)andf⁰⁰(θ)are given as

f⁰(θ) =g₁(θ) +g₂(θ) (67) f⁰⁰(θ) =h₁(θ) +h₂(θ) +h₃(θ) (68) where g1(θ) andg2(θ) are given in equation(151) andh1(θ), h2(θ), andh3(θ) are given in equation(152).

Proof. See Appendix D.

The expressions of f⁰(θ) and f⁰⁰(θ) in equations (67) and (68), respectively, turn out to be too complicated to analyze the error in equation (66). However, some simplifications could be done, asymptotically, thanks to the following theo- rem

Theorem 4: Let λ_j andvvv_j be thej^th smallest eigenvalue and its corresponding

normalized eigenvector of KKK(θ_k). Similarly, define bλ_j andbvbvbv_j for KKK(θbbb _k). The smallest eigenvaluebλ1and its eigenvectorvbbvbv1could be approximated as

bλ1 = 1

kccck²ccc^HBBB^H(θk)UUUeeen_nnPPP_k^⊥⊥⊥UUUeee_nnn^HBBB(θk)ccc+O(kUUUeeen_nnk³) (69) bv

bvbv₁ = 1 kccck

ccc−

p

X

i=∆+2

vv

v^H_iBBB^H(θ_k)UUUn_nnUUUeee_nnn^HBBB(θ_k)ccc λ_i vvv_i

+O(UUUeee_nnn²) (70) wherePPP_k^⊥⊥⊥=III−PPP_kand

P

PP_k =UUU_nnn^HBBB(θ_k)KKK⁺(θ_k)BBB^H(θ_k)UUU_nn_n (71) and∆is the dimension of N BBB(θ_k)

, which is0 whenp ≤ ^N₂⁺² or{p > ^N₂⁺² andθ_k 6∈ΘΘΘ±±±}and non-zero otherwise (according toConsequence 1). Note that O(keUUUee_nn_nk^k)andO(UUUeee_nnn^k)are scalar and vector terms, respectively, in whichUUUeee_nn_nap- pearsktimes in each term.

Proof. See Appendix E.

This theorem reveals a behaviour ofbλ₁, i.e. it acts asO(kUUUeee_nn_nk²). UsingThe- orem 4, and some straightforward algebra, we have the following asymptotic ap- proximations off(θ),f⁰(θ), andf⁰⁰(θ)

f(θ) = 1

bλ1kccck²µk+O keUUUeen_nnk⁻¹

(72) f⁰(θ) =− 2

bλ²₁kccck⁴µkRe{ρek}+O kUUUeeen_nnk⁻²

(73) f⁰⁰(θ) = 2

bλ³₁kccck⁴µ_k 4

kccck²(Re{ρe_k})²−bλ₁υ_k

+O kUUUeee_nn_nk⁻³

(74) where

µ_k=kccc^Haaa^∗_p(θ_k)k² (75) ρe_k=ccc^HBBB^H(θ_k)UUUeeen_nnPPP_k^⊥⊥⊥UUU_nnn^HDDD(θ_k)ccc (76) υ_k=ccc^HDDD^H(θ_k)UUU_nn_nPPP_k^⊥⊥⊥UUU_nnn^HDDD(θ_k)ccc (77) Substituting these expressions in equation (66), we arrive at

θe_k' Re{ρe_k}

υ_k (78)

In order to proceed, we use the following lemma, which gives the probabilistic dis- tribution of the columns ofUUUeee_nn_n

Lemma 1: Letenneen_i be thei^th column ofUUUeee_nn_n. Asymptotically, the vectorsUUU_ss_sUUU_sss^Henneen_i are jointly Gaussian distributed with zero means and covariance matrices given by

E n

U_s UUs_sUUU_sss^Henneen_i

U_s

UUs_sUUU_sss^Henneen_jHo

= σ²

LUUU δ_i,j (79) E

n

UUU_sssUUU_sss^Hneenen_i

UUU_sssUUU_sss^Hneenne_jTo

= 000 (80)

where

UUU =UUU_ss_sΣΣΣ_ss_s ΣΣΣ_ss_s−σ²III−2

UUU_sss^H (81) Proof. See [19].

This lemma is key to the following theorem, which gives the MSE expression E

(eθ_k)²

Theorem 5: The estimates

θb_k ^q_k=1 estimated throughf(θ)by equation(50) are asymptotically unbiased. Furthermore, the MSE expressionE

(θe_k)² is given as

E

(eθ_k)² ,var^(p)_f (bθ_k) = σ² 2L

¯ a

¯ a¯

a^H(θ_k)UUU¯a¯a¯a(θ_k) d¯¯

d¯

d^H(θ_k)UUUn_nnPPP_k^⊥⊥⊥UUU_nnn^Hddd(θ¯¯¯ _k) (82) where ¯a¯aa(θ¯ _k) andUUU are defined in equations (18) and (81), respectively. Also, d¯¯

dd(θ¯ k) = ^{∂¯a¯aa(θ)¯}_∂θ θ=bθk

. Proof. See Appendix F.

It is interesting and easy to see that whenp = 1, the above MSE expression coincides with the MSE expression of MUSIC derived in [19]. In other words, if p= 1, we have¯aa¯¯a(θk) =aaa(θk),ddd(θ¯¯¯ k) =ddd(θk), andPPP_k^⊥⊥⊥=III, hence

var⁽¹⁾_f (bθ_k) = σ² 2L

a

aa^H(θ_k)UUUaaa(θ_k) dd

d^H(θk)UUUn_nnUUU_nnn^Hddd(θk) =varMU(bθ_k;aaa) (83) where varMU(bθ_k;aaa)is read as follows: The variance ofθb_kobtained by MUSIC by utilising a steering vectoraaa(θ). We adopt this notation because the MSE expres- sion, var^(p)_f (θb_k), could also be expressed as

var^(p)_f (θb_k) = 1

1−γ_k

var_MU(θb_k; ¯a¯aa)¯ (84) where

0≤γk=R PP

Pk, UUU_nnn^Hddd(θ¯¯¯ k)

<1 (85)

where the bounds in equation (85) are due to the fact thatγkis aRayleigh quotient, which is always bounded between the minimum and maximum eigenvalues ofPPP_k. SincePPP_k is a projector matrix, then the eigenvalues are either0 or1. Note that