Cramér-Rao bound for a mixture of real- and integer-valued parameter vectors and its application to the linear regression model

an author's

https://oatao.univ-toulouse.fr/26816

https://doi.org/10.1016/j.sigpro.2020.107792

Medina, Daniel and Vilà Valls, Jordi and Chaumette, Eric and Vincent, François and Closas, Pau Cramér-Rao bound

for a mixture of real- and integer-valued parameter vectors and its application to the linear regression model. (2021)

Signal Processing, 179. 107792. ISSN 0165-1684

Cramér-Rao

bound

for

a

mixture

of

real-

and

integer-valued

parameter

vectors

and

its

application

to

the

linear

regression

model

Daniel Medina

a,∗

_{, Jordi Vilà-Valls}

b

_{, Eric Chaumette}

b

_{, François Vincent}

b

_{, Pau Closas}

c a Institute of Communications and Navigation, German Aerospace Center (DLR), Germany

b ISAE-SUPAERO/University of Toulouse, DEOS Dept., Toulouse, France

c Electrical and Computer Engineering Dept., Northeastern University, Boston, MA, USA

Keywords: Cramér-Rao bound McAulay-Seidman bound

Mixed real-integer parameter vector estimation

Linear regression GNSS

Ambiguity resolution

a

b

s

t

r

a

c

t

Performancelowerboundsareknowntobeafundamentaldesigntoolinparametricestimationtheory.A plethoraofdeterministicboundsexistintheliterature,rangingfromthegeneralBarankinboundtothe well-knownCramér-Raobound(CRB),thelatterprovidingtheoptimalmeansquareerrorperformance oflocallyunbiasedestimators.Inthiscontribution,weare interestedintheestimationofmixed real-andinteger-valuedparametervectors.Weproposeaclosed-formlowerboundexpressionleveragingon thegeneralCRBformulation,beingthelimitingformoftheMcAulay-Seidmanbound.Suchformulation isthekeypointtotakeintoaccountinteger-valuedparameters.Asaparticularcaseofthegeneralform, weprovideclosed-formexpressionsfortheGaussianobservationmodel.Onenoteworthypointisthe as-sessmentoftheasymptoticefficiencyofthemaximumlikelihoodestimatorforalinearregressionmodel withmixedparametervectorsand knownnoisecovariancematrix,thuscomplementingtheratherrich literatureonthattopic.Arepresentativecarrier-phasebasedprecisepositioningexampleisprovidedto supportthediscussionandshowtheusefulnessoftheproposedlowerbound.

1. Introduction

Integer parameter estimation appears inmany signal process-ing, biology and communications problems, to name a few. For instance, consider a multi-hypothesis testing problem where we want to identify the received signal over a (finite) set of possi-ble transmitted signals, then a solution is to maximize the log-likelihood function over the (integer) set of candidates. Another problem involvingestimation ofinteger quantities,jointly witha real-valued vector, isthat ofcarrier-phasebasedprecise position-ing in the context of Global Navigation Satellite Systems (GNSS) receivers.Inthegeodesyandnavigationcommunity,awellknown estimation approach is referred to asReal Time Kinematic (RTK) positioning[1].Carrier-phase measurementshaveanunknown in-teger part,referred to asthe ambiguity,to be estimatedin order toachievecm-levelaccuracyonthereal-valuedunknownposition ofthereceiver.TheframeworkthatunderpinspreciseGNSScarrier phase-basedambiguityresolutionisthetheoryofintegeraperture

∗_{Corresponding author.}

E-mail addresses: daniel.ariasmedina@dlr.de (D. Medina), jordi.vila-valls@isae.fr

(J. Vilà-Valls), eric.chaumette@isae.fr (E. Chaumette),closas@ece.neu.edu (P. Closas).

estimation [2,3], which also applies to other carrier phase-based interferometrictechniques,suchasVeryLongBaseline Interferom-etry (VLBI) [4], Interferometric Synthetic Aperture Radar (InSAR)

[5],orunderwateracousticcarrierphase-basedpositioning[6]. Regardlessoftheestimationproblemaddressed, when design-ing andassessing estimators it is of fundamental importance to knowtheminimumachievableperformance,thatis,toobtaintight performance lower bounds (LBs). In general, in estimation prob-lems we are interested in minimal performance bounds in the meansquared error(MSE) sense, which providethe best achiev-ableperformanceontheestimationofparametersofasignal cor-ruptedbynoise.TherearetwomaincategoriesofLBs,deterministic

andBayesian[7].While theformerconsiders thatthe parameters tobeestimatedaredeterministic andevaluate thelocallybest es-timatorperformance, thelatterconsiderrandom parameterswith a givena priori probability andevaluate theglobally best estima-torbehavior.Inthiscontributionweareinterestedindeterministic parameterestimation,thusonlythefirstclasswillbediscussed.

It is worth saying that such LBs have been proved to be ex-tremelyuseful,notonlyforcharacterizinganestimatorasymptotic performance, butalsoforsystem design[7–9].The mostpopular LB is the well-known Cramér-Rao Bound (CRB)derived for real-valuedparametervector,mainlydueto:i)itssimplicityof calcula-https://doi.org/10.1016/j.sigpro.2020.107792

tion,forinstanceusingtheSlepian-Bangs’formula[10];ii)itisthe lowest bound onthe MSE ofanyunbiased estimator(i.e.,it con-siders localunbiasedness at the vicinity ofany selected parame-ters’value);andiii)itisasymptoticallyattainedbymaximum like-lihoodestimators(MLEs)undercertainconditions(i.e.,high signal-to-noise ratio (SNR) [11] and/or large numberof snapshots[12]), that is, MLEs are asymptotically efficient. Inherent limitations of such CRBsare their inabilityto:predict thethresholdphenomena; provide tightboundsincertaincases[13];anddeal with integer-valuedparameterestimation,whichisthecontributionofthis ar-ticle.

SincetheseminalCRBworks,severaldeterministicboundshave beenproposedintheliterature[14–22]toprovidecomputable ap-proximationsoftheBarankinbound(BB)[23],whichisthetightest (greatest) LBforanyabsolutemoment ofordergreater than1 of unbiasedestimators.Infact,theBBconsidersuniformunbiasedness

(i.e., unbiasedness over an interval ofparameter values including the selected value),resulting in a much strongerrestriction than thelocalunbiasednessconditionoftheCRB,butnotadmittingan analyticsolutioningeneral.

In thiscontribution,inorder toobtain a CRB-likeclosed-form expression forthe estimationofmixedparametervectors, includ-ing both real-andinteger-valuedparameters, we leverageonthe McAulay-Seidman bound (MSB)[16].The MSBis the BB approxi-mationobtainedfromadiscretizationoftheBarankinuniform un-biasednessconstraint,usingasetofselectedvaluesofthe param-etervector, so-calledtestpoints.TheMSByields toageneral def-inition ofthe CRB, expressedas a limitingform ofthe MSB. We derive aclosed-formgeneralCRBexpressionformixedparameter vectorsandprovideitsparticularclosed-formfortheGaussian ob-servation model,which encompassesthe well known conditional andunconditionalobservationmodels[24].

On another note, one must keep in mind that inmany prob-lemsofpracticalinterest,includingthegeneral(nonlinear)caseof theproblemunderconsideration,noevidenceoftheachievability ofa givenLBby realizableestimatorsexists [7,8,13].Thus,froma practicalperspective,theLBconsideredmaybetoooptimisticand unable to represent the actual performance of anyestimator. To circumventtheunavailabilityofLBachievabilityresults,asolution relatesto thederivation ofanupperbound toprovidea comple-mentaryvision tothat oftheLB.Unfortunately, upperboundson theMSEofunbiasedestimatesdonotgenerallyexistifthe obser-vationspaceisunbounded.Nonetheless,upperboundsonthe sta-tistical performance (notnecessarily theMSE) mayexistfor spe-cific estimators (not necessarily unbiased) in specific estimation problems [1,25–27]. Inparticular,forthe mixedintegerlinear re-gression model, a rich literature on the statistical performances of various estimators is alreadyavailable (see [1] and references therein), andan upperbound onthe probability that theMLE of thereal-valuedparametervectorliesinacertainregionexists[25]. Remarkably,aLBontheambiguitysuccessrate(probabilityof cor-rect estimationoftheinteger-valuedparametervector)doesexist aswell andappearsto be thecornerstonetoprove thatthe MLE of the real-valued parametervector is efficientin the asymptotic regime (high SNR) when the noise covariance matrix is known, thus completingtheliterature onthattopic(see Section3for de-tails). Asausecase,we considerthelinearregressionproblemin the contextofGNSSprecisepositioning tosupportthediscussion andshowtheusefulnessoftheproposedperformanceLB.

The article is organized as follows: Section 2 provides back-groundondeterministicLBsandtheir derivationasanorm mini-mizationproblem,mainlyfocusedontheCRBasthelimitingform oftheMSB. Section 3detailsthederivation ofthenewbound, in the general case and for the Gaussian observation model. It es-tablishesthe asymptoticefficiencyoftheMLE fora linear regres-sionmodelwithmixedparametervectorsandknownnoise

covari-ance matrix, and sketches possible generalizations and outlooks. Theseresultsare thenparticularized foralinearregression prob-lem,servingasmotivatingexampleanddiscussedinSection4.The paperconcludeswithadiscussionoftheresultsinSection5. 2. BackgroundonMcAulay-SeidmanandCramér-Raobounds forareal-valuedparametervector

2.1. TheMcAulay-Seidmanbound

Lety1_{bearandomreal-valuedobservationsvectorand}

_

_{⊂ R}M

the observation space. Denote by p(y;

θ

)p(y|

θ

) the pdf of the observationsconditionalonanunknowndeterministicreal-valued parametervector

θ

∈



⊂ RK_.LetL

2

(

)

be thereal vectorspace ofsquare integrablefunctionsover



.Ifweconsideranestimator

 g



θ

0



₍

_y

₎

_∈LN

2

(

)

ofg(

θ

0),where

θ

0 isaselectedvalueofthe pa-rameter

θ

andg

(

θ

)

₌

(

g1

(

θ

)

,...,gN

(

θ

))

 isareal-valued function

vector,thentheMSEmatrixwrites,

MSE_θ0



_ g



θ

0



_=E y;θ0





_ g



θ

0



(

_y

)

_{− g}



θ

0



₍

_·

₎





_{. (1)} By noticingthat (1)is a Gram matrix associated withthe scalar product



h

(

y

)

|

l

(

y

)



_θ0=Ey;θ0[h

(

y

)

l

(

y

)

],thesearch fora LBonthe

MSE (1) (w.r.t. the Löwner ordering for positive symmetric ma-trices [28]) can be performed with two equivalent fundamental results: the generalization of the Cauchy-Schwartz inequality to Gram matrices (generally referred to as the "covariance inequal-ity”[18]) and the minimization ofa norm under linear constraints (LCs)[17,19,20].We shallpreferthe“norm minimization” formas its userequiresexplicitlytheselection ofappropriate constraints, which then determine the value of the LB on the MSE matrix, hence providing a clear understanding of the hypotheses associ-ated with the different LBs on the MSE. To avoid the trivial so-lutiong

(

θ

0

)

(

_y

)

_=g

(

θ

0

₎

_{, some constraintsmustbe added.In that} perspective,Barankin[23]introducedtheuniformunbiasedness for-mulation,

Ey;θ



_

g



θ

0



(

_y

)



_=g



θ



_,

∀

θ

_∈



_{, (2a)}

leadingtotheBarankinbound(BB), min  g

(

θ0

)

_∈LN 2()



MSE_θ0



 g



θ

0



_{s.t. E} y;θ



 g



θ

0



₍

_y

₎



_=g



θ



_,

∀

θ

_∈



_, (2b) which does not admit an analytic solution in general. The McAulay-Seidman bound(MSB)isthecomputable BB approxima-tion obtained from a discretization of the uniform unbiasedness

constraint(2a).Let

{

θ}

L

{

θ}

[1,L]

=

{

θ

1_,...,

_θ

L

_}

_∈

_

L_beasubsetof Lselectedvaluesof

θ

(a.k.a.testpoints). Then,anyunbiased esti-matorg

(

θ

0

)

(

_y

)

_{verifying(2a)mustcomplywiththefollowing} sub-1 Italic indicates a scalar quantity, as in a ; lower case boldface indicates a col-

umn vector quantity, as in a ; upper case boldface indicates a matrix quantity, as in A . The n th row and m th column element of the matrix A will be denoted by An,m or [ A ] n,m . The n th coordinate of the column vector a will be denoted by a n or

[ a ] n . The matrix/vector transpose is indicated by a superscript ( · )  as in A  . | A | is

the determinant of the square matrix A . [ A B ] and A B



denote the matrix resulting from the horizontal and the vertical concatenation of A and B , respectively. I M is the

identity matrix of dimension M . 1 M is a M -dimensional vector with all components

equal to one. For two matrices A and B, A ≥ B means that A − B is positive semi- definite (Löwner partial ordering) . · denotes a norm. If θ= [ θ1 , θ2 , . . . , θK ]  ,

then: _∂θ∂ = _∂θ∂_{1 , ∂θ}∂_{2 , . . . , ∂θ}∂_K, _∂θ∂ = _∂ ∂θ1 , ∂θ∂2 , . . . , ∂θ∂K  and ∂h(θ0,y) ∂θ = ∂ h(θ,y) ∂θ _θ 0 . p ( y ; θ)  p ( y | θ) denotes the probability density function (pdf) of y parameterized by

θ. E y;θ[ g ( y ) ] denote the statistical expectation of the vector of functions g ( · ) with

setofLLCs, Ey;θl



_

g



θ

0



(

_y

)



_=g



θ

l



_{, 1≤ l≤ L, (3a)}

whichcanberecastas Ey;θ0

υ

θ0



y;



θ



L



 g



θ

0



(

_y

)

_{− g}



θ

0













 =

⎡

⎢

⎣



g



θ

1



_{− g}



θ

0



 . . .



g



θ

L



_{− g}



θ

0





⎤

⎥

⎦







V , (3b) where

υ

_θ0

(

y;

{

θ}

L

)

=

(

υ

_θ0

(

y;

θ

1

)

,...,

υ

_θ0

(

y;

θ

L

))

,

υ

_θ0

(

y;

θ

)

=

p

(

y;

θ

)

/p

(

y;

θ

0

₎

_{is the vector of likelihood ratios associated to} {

θ

}L_{.TheLLCs(3b)yields theapproximationof(2b)proposedby}

McAulayandSeidman[20], min  g

(

θ0

)

_∈LN 2()



MSE_θ0



_ g



θ

0



_s.t.



_{=V, (4a)}

anddefinestheMSB(Lemma1in[29])[16,20] Ey;θ0





_ g



θ

0



₍

_y

₎

_{− g}



θ

0



₍

_·

₎





_≥

_

g



θ

0



_R−1 υ_θ0



 g



θ

0



_,



g



θ

0



_

_

g



θ

0_,



_θ



L



=

g



θ

1



_{− g}



_θ

0



_{... g}



_θ

L



_{− g}



_θ

0



_, R_υθ0 Rυ_θ0



θ



L



=Ey;θ0



υ

θ0



y;



θ



L



υ

 θ0



y;



θ



L



, (4b) a generalization of the Hammersley-Chapman-Robbins bound (HaChRB)previouslyintroducedin[15,30]for2testpoints

(

L=2

)

.

2.2. CRBasalimitingformoftheMSB

TheCRBcanbedefinedforanyabsolutemoment(greaterthan 1) asthelimitingformoftheHaChRB [15,30],asshowedin[23]. Theextensiontothemultidimensionalreal-valuedparameterscase forthe MSE(i.e.,absolutemomentoforder2) wasintroduced in

[16],allowing to define the CRBasthe limitingformofthe MSB

(4b).Consideringthesubsetoftestpoints



θ



1+K =



θ

0_,

_θ

0_+i 1d

θ

1,...,

θ

0+iKd

θ

K



underd

θ

k=0, 1≤ k≤ K,

wherei_kisthekthcolumnoftheidentitymatrixIK,leadsto υθ0  y;θ1+K  =1 p(y;θ0+i1dθ1) p ₍y;θ0) ... p(y;θ0_+i KdθK) p ₍y;θ0)  , g  θ0_{=0 g}_θ0_+i 1dθ1  − gθ0 _{... g}_θ0_+i K dθK − gθ0_,

and,withd

θ

=

(

d

θ

1,...,d

θ

K

)

,yieldsto(seeAppendixA)



g



θ

0



_R−1 υθ0



 g



θ

0



₌

_

g



θ

0_,d

_θ



_F



_θ

0_,d

_θ



−1

_

 g



θ

0_,d

_θ



_{, (5a)} F



θ

0_,d

_θ



_=E y;θ0

⎡

⎢

⎢

⎢

⎣

⎛

⎜

⎜

⎜

⎝

p

(

y;θ0_+i 1dθ1

)

−p

(

y;θ0

)

dθ1p

(

y;θ0

)

. . . p

(

y;θ0_+i KdθK

)

−p

(

y;θ0

)

dθKp

(

y;θ0

)

⎞

⎟

⎟

⎟

⎠

(

.

)



⎤

⎥

⎥

⎥

⎦

(5b)



g



θ

0,d

θ



=



g

(

θ0_+i 1dθ1

)

−g

(

θ0

)

dθ1 ... g

(

θ0_+i KdθK

)

−g

(

θ0

)

dθK



, (5c) whichresultsinageneraldefinitionoftheCRB_g|θ(

θ

0_)as

CRBg | θ  θ0_{= lim} sup { d θ1=0, ... ,d θK=0 } →0 gθ0,dθFθ0,dθ−1g  θ0_,dθ_{. (6a)}

If

θ

0_∈

_

_{⊂ R}K_andg(

_θ

_)andp(y;

_θ

_)areC1_at

_θ

0_{,then(6a)yields} thewellknownFisherInformationMatrix(FIM)F(

θ

)andtheusual CRBexpression F



θ



=Ey;θ

"

∂

lnp



y;

θ



∂θ

(

.

)



#

, (6b) CRBg|θ



θ

0



₌

∂

g



θ

0



∂θ

 F



θ

0



−1

$

∂

g



θ

0



∂θ



%

 . (6c)

3. CRBforamixtureofreal-valuedandinteger-valued parameters

Leveragingthe MSBandCRBresultspresentedintheprevious

Section2,wederiveinthissectionaLBfordeterministic parame-tervectorestimation,wheresuchvectorcontainsbothreal-valued andinteger-valued parameters.A generalresultis provided, then particularizedforthecaseofGaussianobservations.

3.1. GeneralCRBformixedparametervectors

The main result derived in this article is summarized in the formof Theorem1. Acorollary follows,which simplifiesthe for-merinaparticularclassofmodels.

Theorem1(General CRBformixedparametervectors). Assumea setof observations y_∈



_{⊂ R}M _{and anunknown deterministic} real-valuedparametervector

θ

∈



⊂ RK_where

_θ

_=[

_ω

_,z],

_ω

_∈RK_ω,

z∈ZKz _{,Kω+Kz=K.Thosequantitiesarerelatedthroughastatistical}

modelof theformy|

θ

~ p(y|

θ

),whichis available.Then,theMSEof any unbiased estimator of a function g



θ

0



_∈L

2

(

)

for a selected

valueoftheparameter

θ

0 _{islowerboundedby}

CRB_g|θ



θ

0



₌

_

g



θ

0



_F



_θ

0



−1

_

 g



θ

0



_{, (7)} with



g



θ

0



₌



∂g

(

θ0

)

∂ω g



θ

1



_{− g}



_θ

0



_{... g}



_θ

2Kz



− g



θ

0



(8) F



θ

0



₌

F_ω|θ



θ

0



_H



_θ

0



H



θ

0



 _MS z|θ



θ

0





, (9)

wherethetestpoints



θ



2Kz aredefinedas

θ

j₌

_θ

0₊

₍

₋₁

₎

j−1_i Kω +



j+1 2 , 1≤ j≤ 2Kz, (10) thatis,

θ

1_,

_θ

2_,...,

_θ

2Kz−1,

θ

2Kz



=

θ

0_+i Kω +1,

θ

0− iKω +1,...,

θ

0+iK,

θ

0− iK



.

ThedifferenttermsinF



θ

0



_aregivenby

F_ω|θ



θ

0



_=E y;θ0

"

∂

lnp



y;

θ

0



∂ω

(

.

)



#

, (11a) H



θ

0



_=E y;θ0

"

∂

lnp



y;

θ

0



∂ω

t2Kz

#

(11b) =

h



θ

0_,

θ

1



_h



θ

0_,

θ

2



_{... h}



θ

0_,

θ

2Kz



, _(11c) MSz|θ



θ

0



_=E y;θ0

t2Kzt2Kz



− 12Kz12Kz, (11d) 3

wheret2K_z isdefinedas t2Kz

υ

θ0



y;



θ



2Kz



=

$

p



y;

θ

1



p



y;

θ

0



, p



y;

θ

2



p



y;

θ

0



,..., p



y;

θ

2Kz



p



y;

θ

0



%

 . (11e) Proof. First, notice that in the real-valued parameter case, that is, if

θ

0

k∈R, and both g(

θ

) and p(y;

θ

) are C1 at

θ

k0, then,

the constraints associated to the following two test points,



θ

0_+i kd

θ

k,

θ

0+ik

(

−d

θ

k

)



=



θ

0_+i kd

θ

k,

θ

0− ikd

θ

k



, Ey;θ0

⎡

⎣

⎛

⎝

p

(

y;θ0_+i kdθk

)

−p

(

y;θ0

)

dθkp

(

y;θ0

)

p

(

y;θ0_−i kdθk

)

−p

(

y;θ0

)

(−dθk)p

(

y;θ0

)

⎞

⎠



_g



_θ

0



₍

_y

₎

_{− g}



θ

0





⎤

⎦

=

⎡

⎣



_g

₍

_θ0_+i kdθk

)

−g

(

θ0

)

dθk







_g

₍

_θ0_−i kdθk

)

−g

(

θ0

)

(−dθk)





⎤

⎦

(12)

aim at the same single constraint in the limiting case where

d

θ

k→0,d

θ

k=0, Ey;θ0

"

∂

lnp



y;

θ

0



∂θ

k



_ g



θ

0



(

_y

)

_{− g}



θ

0





#

=

∂

g



θ

0



∂θ

k  . (13)

However, this phenomenon isunlikely tohappen if

θ

0

k ∈Z inthe

limitingcasewhered

θ

k→0,d

θ

k=0,since(12)thenbecomes

Ey;θ0

⎡

⎣

⎛

⎝

p

(

y;θ0_+i k

)

−p

(

y;θ0

)

p

(

y;θ0

)

p

(

y;θ0_−i k

)

−p

(

y;θ0

)

p

(

y;θ0

)

⎞

⎠



_g



_θ

0



₍

_y

₎

_{− g}



θ

0





⎤

⎦

=

"

_

g



θ

0_+i k



− g



θ

0







g



θ

0_{− i} k



− g



θ

0





#

, (14) where



p



y;

θ

0_+i k



− p



y;

θ

0



_/p



_y;

_θ

0



_and



p



y;

θ

0_{− i} k



− p



y;

θ

0



_/p



_y;

_θ

0



_{are unlikely to be linearly} dependent (i.e., notice that F



θ

0_,d

_θ



_{in (5b) must be invertible} to compute the CRB_g|θ(

θ

0_{) in (6a)). Therefore, in most cases, the} combination of LCs (13) and (14) yields, from Lemma 1 in [29], the generaldefinition(7)ofCRB_g|θ(

θ

0_{) wherethedifferent terms} inF



θ

0



_aregivenby F_ω|θ



θ

0



_=E y;θ0

"

∂

lnp



y;

θ

0



∂ω

(

.

)



#

, (15a) H



θ

0



_=E y;θ0

"

∂

lnp



y;

θ

0



∂ω

(

t2Kz− 12Kz

)



#

, (15b) MSz|θ



θ

0



_=E y;θ0

(

t2Kz− 12Kz

)

(

t2Kz− 12Kz

)





_{, (15c)}

and where H(

θ

0_{) and MS}

z|θ(

θ

0) can also been expressed as (11b)and(11d). _

Corollary1. Ifg

(

θ

)

=

θ

,matrix



_θ(

θ

0_{)inTheorem1simplifiesto}



θ



θ

0



=

i1 ... iK_ω iK_ω +1 −iK_ω +1 ... iK −iK



= (Kz=3)

⎡

⎢

⎣

IKω 0 0 0 0 0 0 0 1 −1 0 0 0 0 0 0 0 1 −1 0 0 0 0 0 0 0 1 −1

⎤

⎥

⎦

. (16)

3.2. TheGaussianobservationmodel

Let us consider an M-dimensional Gaussian real vector y with mean my=m



θ



and covariance matrix Cy=C



θ



: y_∼

NM



m



θ



,C



θ



andp



y;

θ



=p



y; m



θ



,C



θ



suchthat

p



y; m



θ



,C



θ



= e− 1 2

(

y−m

(

θ

)

)

 C−1

(

_θ

)

(

_y−m

(

_θ

)

)

√ 2

π

M

&



C



θ



. (17) Ifwedefine Ci j₌



_C



_θ

i



−1_+C



_θ

j



−1_{− C}



_θ

0



−1



−1_{, (18a)} mi j _=C



_θ

i



−1_m



_θ

i



_+C



_θ

j



−1_m



_θ

j



_{− C}



_θ

0



−1_m



_θ

0



_{, (18b)}

δ

i j _=m



_θ

i



_C



_θ

i



−1_m



_θ

i



_+m



_θ

j



_C



_θ

j



−1_m



_θ

j



− m



θ

0



_C



_θ

0



−1_m



_θ

0



_{, (18c)} thenwecanobtainthedifferentcomponentsrequiredtocompute the CRB (7)(see Appendix B fordetailed derivation of MS_z|θ(

θ

0₎ andh(

θ

0_,

_θ

j_))as

MSz|θ



θ

0



i, j=

'



Ci j



_C



θ

0





C



θ

i



_C



θ

j



e 1 2 

(

mi j

)

_Ci j_mi j_−δi j − 1, (18d)

h



θ

0,

θ

j



k=

⎛

⎜

⎜

⎜

⎜

⎝

1 2tr

(

∂C

(

θ0

)

−1 ∂ωk



C



θ

0



_{− C}



_θ

j



₋∂C

(

θ0

)

−1 ∂ωk ×



m



θ

j



_{− m}



_θ

0



_m



_θ

j



_{− m}



_θ

0







+∂m_∂ω

(

θ_k0

)

C



θ

0



−1



_m



θ

j



_{− m}



θ

0



⎞

⎟

⎟

⎟

⎟

⎠

(18e)

F_ω|θ



θ

0



k,l =

∂

m



θ

0



∂ω

k  C−1



θ

0

∂

m



θ

0



∂ω

l +1 2tr

$

C−1



θ

0

∂

C



θ

0



∂ω

k C−1



θ

0

∂

C



θ

0



∂ω

l

%

, (18f) where(18f)istheSlepian-Bangsformula[31,p.47].

Inthefollowing,forsakeoflegibility,

θ

denoteseitherthe vec-tor of unknown parameters or a selected vector value



θ

0



₌ [



ω

0



_,



_z0



_].

3.3. AsymptoticallyefficientestimatorsfortheGaussianlinear conditionalobservationmodel(g

(

θ

)

₌

θ

)

Acaseof particularinterest is theGaussian linear conditional signalmodel,alsoknownasthemixed-integermodel[1,Ch.23],

y=B

ω

+Az+n, n∼ NM

(

0,Cn

)

,

θ

_=[

_ω

_,z_],

_ω

_∈RKω _{, z∈Z}Kz, ₍₁₉₎

wherethenoise covariancematrixCnis knownandthe

parame-tervectorofinterestisg



θ



=

θ

.Forinstance,inGNSSRTKprecise positioning,theM-vectorycontainsthepseudorangeand carrier-phaseobservables,theKz-vectorztheintegerambiguities,andthe

real-valuedK_ω-vector

ω

theremainingunknown parameters,such as, forexample, position coordinates,atmospheric delay parame-ters (troposphere, ionosphere) and clock parameters. The theory that underpins theresolution of(19) in themaximum likelihood

senseisthetheoryofintegerinference[2,3][1,Ch.23].Thesearch for the MLE of a selected value

θ

for the mixed-integer model

(19)canbecastasaminimizationproblemovermixedinteger-real parameters, )

θ

=arg min ω∈RKω _,z∈ZKz

**

y− D

θ

**

2_C n, D=

B A



. (20)

A closed-form solution to (20) is not known,due to the integer natureofz.Instead,athree-stepdecompositionoftheproblemis typicallyconsidered[32],andtheresultingminimizationproblems aresequentiallyresolvedas[33]

min ω∈RK_{ω ,z∈Z}Kz

**

y− D

θ

**

2_C n=

**

**

y− D

(

¯

ω

¯z

+**

**

2 Cn (21a) +min z∈ZKz

¯z− z

2 Cω¯ (21b) + min ω∈RKω

)

ω

(

z

)

−

ω

2 C)ω( z) , (21c)

where)

ω

(

z

)

=

ω

¯− Cω¯,¯zC−1¯z

(

¯z− z

)

.Thefirstterm(21a)corresponds to the MLE solution where z is treated as a real valued vector (instead of aninteger valuedvector).The output ofthisestimate

¯

θ

_=[

_ω

_¯_,¯z_{]isreferredtoasfloatsolutionanditsassociated} co-variancematrixis C_θ¯=

Cω¯ Cω¯,¯z C¯z,ω¯ C¯z



=



D_C−1 n D



−1 ,

which, by exploiting the four-blocks matrix inversion expression

[28],leadsto C)ω(z)=Cω¯− Cω¯,¯zC−1¯z C¯z,ω¯ =



B_C−1 n B



−1 .

The second term (21b) in the decomposition corresponds to the integer-least-square(ILS),forwhichanintegersolution)zisfound. Finally,thethirdterm(21c)isthe fixedsolution,consistingon en-hancingtheestimates

ω

¯ upontheestimatedintegervector)z )

ω

=)

_ω

₍

)z

)

=

ω

¯ − C_ω¯,¯zC−1¯z

(

¯z−)z

)

. (22)

The improvement in )

ω

accuracy is dueto constrainingthe float solution ¯z to a more restrictive class of estimators. Three differ-ent classes of estimators have beendeveloped for mixedinteger models [34], andforeach one the optimalestimators have been identified:i)theclassofinteger(I)estimators[35];ii)theclassof integer-aperture (IA) estimators[36];andiii)the classof integer-equivariant (IE)estimators[37].Thefirstclassisthemost restric-tive class. Thisis due to the fact that the outcomes ofany esti-mator within this class are required to be integers. Well-known examples of estimators fromthis class are integerrounding ()zR),

integerbootstrapping()zB)andtheoptimalsolutionso-called

inte-ger least-squares ()zLS) which is the MLE. The most relaxedclass

istheclassofIE-estimators.Theseestimatorsarereal-valued,and theyonlyobeytheintegerremove-restoreprinciple.Animportant estimator in thisclass is the best IE-estimator ()zBIE) since it has

the smallestvariance,even smallerthan thevariance ofthe float solution.TheclassofIA-estimatorsisasubsetoftheIE-estimators butitencompassestheclassofI-estimators.TheIA-estimatorsare ofahybridnature inthesense thattheir outcomesareeither in-tegers or real. It is also worth noting that distributional results are readily available [25,38].Interestingly enough, integer round-ing,integerbootstrapping,andintegerleast-squaresestimators()zR,

)zB,)zLS)areuniformlyunbiased[39]underGaussianadditivenoise

(19),leadingtoanuniformlyunbiasedestimator

ω

)(22),sincethen E[)

ω

]=E[

ω

¯]=

ω

.Thus theproposed CRB_θ|θ(

θ

) (7)isarelevantLB fortheGaussianlinearconditionalsignalmodel(19)and

C_)ω=C_)ω()z)≥ CRBω|θ



θ



,)z∈

{

)zR,)zB,)zLS

}

. Firstly, as [1, (23.54)] P

(

)zLS=z

)

≥ P

(

)zB=z

)

≥ P

(

)zR=z

)

, and [1, (23.23)] lim tr(Cn)→0 P

(

)zR=z

)

=1,then lim tr(Cn)→0 C)zLS= lim tr(Cn)→0 C)zB= lim tr(Cn)→0 C)zR=0.

Thus, for any )z∈

{

)zR,)zB,)zLS

}

, since [25, (29)] C)ω()z)=C)ω(z)+ Cω¯,¯zC−1¯z C)zC−1¯z C¯z,ω¯, lim tr(Cn)→0 C_ω)()z)=



BC−1n B



−1 .

Secondly,sinceitiswell knownthat addingunknownparameters leadstoanequalorhigherCRB,then(25a)

C)ω()z)≥ CRBω|θ



θ



≥ F−1ω|θ



θ



=



B_C−1 n B



−1 .

Therefore,forany)z_∈

{

)zR,)zB,)zLS

}

,

lim tr(Cn)→0 Cω)()z)=_tr(lim_C n)→0 CRB_ω|θ



θ



=



B_C−1 n B



−1 ,

which proves that )zR,)zB and)zLS are asymptotically efficient

es-timators. Last, since)zBIE is also uniformly unbiased with a MSE

lessthan oratthemostequaltotheMSEof)zLS[37,(24)],itisan

asymptoticallyefficientestimatoraswell.

3.4. Generalizationsandoutlooks

TheproposedCRBformixedparametervectors(7)hasbeen de-rived in thecontext of “standard” deterministicestimation prob-lemsforwhichaclosed-formexpressionofp(y;

θ

)isavailable.In the context of “non standard” deterministic estimation problems (see[40]andreferencestherein),p(y;

θ

)resultsfromthe marginal-ization of an hybrid p.d.f. dependingon both random



θ

r∈RPr



and deterministic (

θ

) parameters, i.e., p



y;

θ



=,_RP r p



y,

θ

r

|

θ



d

θ

r,

which ismathematically intractable andprevents fromusing the proposed standardCRB (7). Fortunately,anyLB deriving fromthe MSB, as for instance (7), can be transformed into two variants, namely the so-called “modified” LB [40, Section III] and “non-standard” LB [40,Section IV]fitted tonon-standard deterministic estimation. Thus, two CRB variants for mixed parameter vectors canbe readilyintroduced inthecontext of“nonstandard” deter-ministic estimation. Since the proposed CRB (7) can also be re-gardedasaCRBdealingwithrestrictingthesetofpossiblevalues of some of the unknown parameters, namely the integer-valued parameters,itisworthnoticingthat(7)canalsotakeintoaccount continuous restriction on the set of possible values of the real-valued parameters. When the continuous restriction is described by a set of P equality constraints, f

(

ω

)

=0∈RP_{, 1≤ P≤ K}

ω− 1, wherethematrix

∂

f

(

ω

)

/

∂

ω

_∈RP×Kω hasfullrowrank(P),which is equivalentto requiringthat the constraintsare not redundant, it leads to the so-calledconstrained CRB [41–44]. In the caseof mixedparametervectors,it amountstoreplacetheLCs(13)with

[29] Ey;θ

"

$

U

(

ω

)



∂

lnp



y;

θ



∂ω

%

_

 g



θ



(

y

)

− g



θ





#

=

$

∂

g



θ



∂ω

 U

(

ω

)

%

 ,

where U

(

ω

)

∈RK_ω×(K_ω−P) _{is a basis of ker}



∂f(ω) ∂ω

, which leads to update the definition of F_ω|θ(

θ

) (11a), H(

θ

) (11b) and



g(

θ

)

(8) as follows: F_ω|θ(

θ

) → U(

ω

)F_ω|θ(

θ

)U(

ω

), H(

θ

) → U(

ω

)H(

θ

), and



g



θ



=



∂_∂ωg

(

θ

)

U

(

ω

)

g



θ

1



_{− g}



θ

0



_{... g}



θ

2Kz



_{− g}



_θ

0



_. 5

4. ExampleofGaussianlinearregressionproblem:GNSSRTK precisepositioning

Inthissection,weexemplifytheaforementionedresultswitha particularexampleofGaussianlinearconditionalsignalmodel,aka thelinearregressionproblemwithmixedrealandinteger param-eters,thatistheGNSSRTKprecisepositioningproblem.

4.1. SignalmodelforGNSSRTKprecisepositioning

RTK is a GNSS-based relative positioning method, where the position of a target is determined with respect to a base sta-tion of known coordinates [45]. RTK exploits the use of code and carrier-phase pseudorange observations. Carrier-phase obser-vationsarecharacterizedbyanoise(typically)twoordersof mag-nitude lower than code pseudorange measurement, but they are ambiguous by an unknown numberof integer ambiguities.Thus, the achievement of high precision positioning requires the esti-mation ofthe dynamicalparameters ofthe target alongwiththe unknown integerambiguitieswithinaprocessreferred toas Inte-gerAmbiguityResolution(IAR)[33].Weassume thatM+1 satel-lites are tracked simultaneously by thebase andtarget GNSS re-ceivers. First, single-differencingthe observations,i.e., subtracting the observations atthe target from the base stations, eliminates the atmosphericandsatellite-related delays.Then, the process of double-differencing,i.e., subtracting thesingle-difference observa-tions withrespect toa reference satellite, removesthe effects of receiversclockoffsets.TheresultingGNSSRTKmodelcanbe mod-eledasy_{∼ N}M



m

(

θ

)

,Cn



with m

(

θ

)

=D

ω

z



, D=[BA], B=

B B



, A=

A 0



, Cn=

C_φ 0 0 C_ρ



, (23)

where y=

φ

_,

_ρ





_{is theobservation vector, composed by the} double-differencecarrier-phaseandcodeobservations,denotedas

φ

,

ρ

∈RM _{respectively, whose corresponding covariance matrices}

are C_φ andC_ρ.

ω

∈R3 _{isthetarget receivertobasestation} base-linevector;andz∈ZM _{isthevectorofunknowninteger}

ambigui-ties. ThematrixBistheso-calledgeometrymatrixwhichis com-posed ofthe unit line-of-sight vectorspointingfromthereceiver to eachsatellite. Aisthediagonal matrixwiththe wavelengthof the carrier-phase measurements [32,46]. The covariance matrices C_φandC_ρaredefinedasC_φ=2

σ

2

φTW−1TandCρ=2

σ

ρ2TW−1T,

where

σ

_φ and

σ

_ρ are thezenith-referenced undifferenced phase and code standard deviations [46], T₌[IM − 1M] is the

double-differencing matrix, W=diag

(

w1,...,wM+1

)

is a diagonal values andwi isthesatellite elevation-dependentweight.Asitis

formu-lated, the RTK problem can be seen to fit the linear regression problem discussed earlier in Section 4. In practice, the solution forthe mixedrealandinteger model(20)isgenerallysolved via themixedreal-integerregression((21a)–(21c)).Thus, theRTK po-sitioning modelconstitutesa practicalexampleofGaussian linear regressionwithmixedreal-andinteger-valuedparameters.

4.2. CRBforGNSSRTKprecisepositioning

Asaforementioned,intheRTKproblem,theGaussianlinear ob-servationmodelreads

y=

φ

ρ



=D

θ

+n, n∼ NM

(

0,Cn

)

,B=[b1...bKω ], A=[a1...aKz]. (24)

From theresultspresentedinSection 3.1,werecall thattheCRB, referredtoasCRBReal/Integerinthefollowing,isgivenby

CRB_θ|θ



θ



=



θ



θ



F



θ



−1



θ



θ



, F



θ



=

F_ω|θ



θ



H



θ



H



θ



 MSz|θ



θ





,

where



_θ(

θ

)isgivenby(16),andwehavetocomputeF



θ



using the generalGaussian modelequations givenin Section 3.2. Since Cndoesnotdependon

θ

,(18a)–(18c))become

Ci j_=C n, mi j=C−1n



m



θ

i



_+m



θ

j



_{− m}



θ



_,

δ

i j_=m



_θ

i



_C−1 n m



θ

i



_+m



_θ

j



_C−1 n m



θ

j



− m



θ



C−1_n m



θ



, [MS]_{i, j}=e

(

m

(

θ

)

−m

(

θi

)

_−m

(

_θj

)

)

_C−1 nm

(

θ

)

+m

(

θi

)

 C−1nm

(

θj

)

.

TheseequationsallowcomputationoftheelementsintheCRB for-mula.Particularly,computingF_ω|θ(

θ

)for1≤ k,k≤ Kω becomes

F_ω|θ



θ



k,k=

∂

m



θ



∂ω

k  C−1_n

∂

m



θ



∂ω

k =

bk bk



 C−1_n

bk bk



, suchthat F_ω|θ



θ



=B_C−1 n B. (25a) Let

θ

j₌

_θ

₊

₍

₋₁

₎

j−1_i K_ω+-j+1 2 . and

θ

i₌

_θ

₊

₍

₋₁

₎

i−1_i K_ω+/i +1 2 0. Then,

MSz|θ



θ



_{i, j}=e

(

m

(

θ

)

−m

(

θi

)

−m

(

θj

)

)

C−1n m

(

θ

)

+m

(

θi

)

 C−1 nm

(

θj

)

− 1 =e

(

θ−θi−θj

)

DC−1nDθ+

(

θi

)

 D_C−1 nD

(

θj

)

− 1, _(25b)

andfor1≤ k≤ Kω,wehavethat

h



θ

,

θ

j



k=

∂

m



θ



∂ω

k  C−1_n



m



θ

j



_{− m}



_θ



=

bk bk



 C−1n D



θ

j₋

_θ



_, whichleadsto H



θ



=B_C−1 n D

iKω +1 −iKω +1 ... iK −iK



. (25c)

It is worth remembering that relaxing the condition on the integer-valued part ofthe parameters’ vector, and assuming that bothparameters arereal-valued,

ω

_∈RK_{ω, z∈R}Kz _,then the stan-dardCRBisgivenbytheinverseofthefollowingFIM,

F_θ|θ



θ



=D_C−1

n D, (26)

andisreferredtoasCRBRealinthefollowing. 4.3. Illustrativeexample

To illustrate the validity of the proposed LB, a realistic GNSS RTK experiment was simulated.Particularly, the receiver-satellite geometryconsideredisillustratedinFig.1(M+1=13satellites), underawiderangeofprecision levelsfortheundifferencedcode observations –preserving the noise ofcarrier-phase (

φ

) measure-mentstwoordersofmagnitudelowerthancode(

ρ

)observable–. Toevaluate theLSperformance,the root(total)MSE (RMSE), ob-tained from104 _{Monte Carlo runs, forboth 3D positionand the} 12(M)integerambiguitieswasconsideredasameasureof perfor-mance.

Fig.2(left)showsthe3D position(

ω

) RMSEoverthestandard deviation ofcode observations

σ

_ρ, aswell asthe square-root of

Fig. 1. Skyplot of the simulated GNSS receiver-satellite geometry.

the corresponding (total)CRBs,witha zoomofthelow noise re-giongiveninFig.2(right).First,noticethatthestandardLS (equiv-alenttotheMLE)RMSE,asexpected,coincideswiththeCRB_Realfor therangeoftested

σ

_ρ values,whichgivestheultimateachievable performance withboth code andphase observablesif nointeger constraint isimposed forambiguitiesz. Secondly,the ILS perfor-mance, considering thatthe floatpositionestimate isalways cor-rectedbytheoutputambiguitiesoftheIAR,clearlydependsonthe noise level. Three regions of performance can identified: i) large noiseregime:theILScoincideswithbothstandardLSandCRBReal,

which is clear from the ILS success rate shown in Fig. 3, where wecanseethatfor

σ

_ρ >5[m]acorrectintegersolutionisnever found,then,onaverage,isasifnointegerconstraintwasimposed;

ii) low noise regime:the IAR obtains the correct ambiguity solu-tionwithhighprobability,thentheILScoincideswiththeso-called

CorrectILS(whichonlyconsidersthesuccessfuloutputsoftheIAR)

and theCRBReal|Integer, whichshowsthat theILS is asymptotically

efficient; and iii) threshold region: below the so-called threshold point (inthiscase,

σ

_{ρ >}0.1[m]),theILSRMSEdeparts fromthe

CRBReal|Integerandrises towards theCRBReal,witheven asmall

re-gionwheretheRMSEoverpassestheperformanceofastandardLS

Fig. 3. Success rate for the integer parameter estimation of the RTK positioning problem.

(inthiscase,for1<

σ

ρ < 10[m]).Thisregiondescribesthe be-haviour ofthe ILS, which abandons its asymptotic efficiencyand ambiguouserrorsoccurduetothe(partially)wrongestimationof theintegerambiguities.Thethresholdpointvarieswiththe satel-litegeometry,numberofobservations(i.e.,numberoffrequencies tracked) andobservationnoises. Therefore, theprecise prediction forthe transitionpoint remains an open challenge. Finally,if we considered onlythesuccessfullyfixed ambiguities,theCorrect ILS

wouldcoincide withtheCRBReal|Integer.However, the correct

solu-tiontotheILSproblemcannotbe guaranteedoutsidethe asymp-toticregion.

Regarding theCRB_Real|Integer andCRBReal comparison,it isclear

that considering the integer nature ofa part of the vector to be estimatedhasastrongimpactontheachievableperformance,and therefore, highlights the interest of the estimating the so-called integer ambiguities. As a byproduct, this highlights the impor-tanceof theLB proposed inthiscontribution.Obviously, restrict-ing the set ofpossible values (integer instead ofreal) leads to a

LB such that CRBReal|Integer ≤ CRBReal. For this bound there exists

alsoa noisethresholdregion fromwherethereal/integer

parame-Fig. 2. Positioning RMSE and square-root of CRBs as a function of the standard deviation of observation noise σρ.