An Efficient Numerical Method for the Symmetric Positive Definite Second-Order Cone Linear Complementarity Problem

https://doi.org/10.1007/s10915-019-00907-4

An Efficient Numerical Method for the Symmetric Positive Definite Second-Order Cone Linear Complementarity Problem

Xiang Wang¹·Xing Li²·Lei-Hong Zhang³·Ren-Cang Li⁴

Received: 10 April 2018 / Revised: 6 November 2018 / Accepted: 7 January 2019 / Published online: 1 February 2019

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Abstract

An efficient numerical method for solving a symmetric positive definite second-order cone linear complementarity problem (SOCLCP) is proposed. The method is shown to be more efficient than recently developed iterative methods for small-to-medium sized and dense SOCLCP. Therefore it can serve as an excellent core computational engine in solutions of large scale symmetric positive definite SOCLCP solved by subspace projection methods, solutions of general SOCLCP and the quadratic programming over a Cartesian product of multiple second-order cones, in which small-to-medium sized SOCLCPs have to be solved repeatedly, efficiently, and robustly.

Keywords Second-order cone·Linear complementarity problem·SOCLCP·Globally uniquely solvable property·GUS

Mathematics Subject Classification 90C33·65K05·65F99·65F15·65F30·65P99

B

[email protected] Xing Li

[email protected] Lei-Hong Zhang [email protected]

1 Department of Mathematics, Nanchang University, 999 Xuefu Road, Nanchang 330031, China 2 School of Mathematics, Shanghai University of Finance and Economics, 777 Guoding Road,

Shanghai 200433, China

3 School of Mathematical Sciences, Soochow University, Suzhou 215006, Jiangsu, China 4 Department of Mathematics, University of Texas at Arlington, P.O. Box 19408, Arlington,

TX 76019-0408, USA

1 Introduction

Given a symmetric positive definite matrix M ∈ R^n×n, we are interested in solving the followingsecond-order cone linear complementarity problem(SOCLCP):

xxx∈Kⁿ, qqq+Mxxx∈Kⁿ, xxx^T(qqq+Mxxx)=0, (1.1) whereqqq∈Rⁿ, andKⁿis thesecond-order cone(SOC):

Kⁿ :=

x1,xxx^T₂T

∈R×Rⁿ⁻¹ : xxx22≤x1

.

One source of (1.1) is the KKT system of the following conic optimization problem

xxxmin∈Kⁿ

1

2xxx^TMxxx+qqq^Txxx, whereMis symmetric positive definite.

Since M is assumed symmetric positive definite, SOCLCP (1.1) admits the globally uniquely solvable(GUS) property, i.e., it has a unique solution for any givenqqq ∈Rⁿ. The GUS property for SOCLCP has been studied in, e.g., [13–15,32]. In particular, Yang and Yuan [32]

provided an algebraic characterization of the GUS property. Based on the characterization, several efficient iterative methods [36–38] have been developed. Other numerical methods, related theory and applications for SOCLCP, can be found in, e.g., [3,7,8,11,16,20,26,33,34].

Besides the GUS property, the symmetric positive definiteness ofMallows us to use the efficient state-of-the-art eigendecomposition subroutines in LAPACK [2] to decomposeM or the matrix pencil M−λJ_n to develop more efficient methods for (1.1), especially for modestn, than what we have in the literature, where

Jn:=diag(1,−1, . . . ,−1)∈R^n×n. (1.2) Our main goal of this paper is to develop an efficient numerical method for solving SOCLCP (1.1) with a symmetric positive definiteMalong this line. The method starts by computing the eigendecomposition ofM−λJn to turn SOCLCP (1.1) into a zero-finding problem for a rational function and then a very fast zero-finder based on a simple rational approximation is devised for the transformed problem. The proposed method is fast and robust for small-to- medium sized SOCLCP and, though iterative in nature, can be regarded as a direct method, just as nowadays the QR algorithm for the eigenvalue problem is widely treated as a direct method due to the fact that it is such a robust iterative algorithm.

Conceivably, there are several scenarios the new method can be called upon to play critical roles: (1) when a symmetric positive definite SOCLCP (1.1) itself is of modest size, (2) during intermediate steps of solving a large scale SOCLCP by projections, numerous SOCLCP of modest sizes have to be efficiently solved [38, Theorem 6.7], (3) any general SOCLCP

xxx∈K×m, qqq+Mxxx∈K×m, xxx^T(qqq+Mxxx)=0, (1.3) over a Cartesian product of multiple second-order conesK_×m:=Kⁿ¹×Kⁿ²× · · · ×K^nmby the matrix splitting method [37, Algorithm 1] in which a sequence of small size SOCLCPs have to be solved during each iterative step, and (4) the quadratic programming overK×m

(QP-SOC) (i.e., minimization of a quadratic¹₂xxx^TMxxx+xxx^Tqqqsubject toAxxx=bbb, xxx∈K×m) by the Augmented Lagrangian Method (ALM) [18,27] in which a particular SOCLCP (1.3) needs to be solved in each iteration.

The rest of this article is organized as follows. In Sect.2, we first present basic properties of SOCLCP, and in Sect.3, we apply the eigendecomposition of the symmetric positive

definite pencilM−λJ_n to transform SOCLCP into a zero-finding problem. Basing on the new reformulation, we next describe the solution for the critical case in Sect.4. Theoretical results and detailed algorithmic procedure for the generic case are detailed in Sect.5. To demonstrate potential usages of our proposed algorithm, in Sect.6, we provide two applica- tions of our algorithm, namely the general SOCLCP (1.3), and QP-SOC. Numerical results of our new algorithm as well as its performance in these two applications are reported in Sect.7. In particular, we conduct our numerical testing by comparing the efficiency of the proposed algorithm withCVX(a modeling system for constructing and solving various con- vex programming) and two other recent algorithms: the bisection-Newton method (BN) [36]

and the linear complementarity problem via the Arnold process (LCPvA) [38] for (1.1). Final remarks are drawn in Sect.8.

Notation: Throughout this paper, all vectors are column vectors and are typeset in bold lower case letters. ForA∈R^m×n(the set of allm×nreal matrices),A^Tdenotes its transpose, andR(A)andN(A)represent the range and kernel ofA, respectively. ThusR(A)^⊥=N(A^T), whereR(A)^⊥denotes the orthogonal complement ofR(A). As usual, the identity matrix in R^n×nwill be denoted byIn ≡ [eee1,eee2, . . . ,eeen], whereeeeiis itsicolumn. We shall also adopt MATLAB-like convention to access the entries of vectors and matrices. For example,xxx_(i)is theith element ofxxxandA_(i,j)is the(i,j)th entry of A, where(i : j)stands for the set of integers fromito jinclusive, andA_(k:,i:j)is the submatrix ofAthat consists of intersections from rowkto rowand columnito column j. Notation · 2is either the matrix spectral norm or the Euclidean vector norm, depending on its arguments:

xxx2:=

i

|xxx₍i)|², A2:=max

xxx=0

Axxx2

xxx2 .

2 Basic Properties of SOCLCP

To simplify our presentation, we will denote (1.1) by SOCLCP(M,Kⁿ,qqq) whose set of solutions will be denoted by SOL(M,Kⁿ,qqq). In this section, we review several basic results for SOCLCP(M,Kⁿ,qqq). Denote by∂(Kⁿ)and int(Kⁿ)the boundary and the interior ofKⁿ, respectively.

Lemma 2.1 ([32])The following statements hold:

(i) For nonzero vectors xxx,yyy ∈ Kⁿ, xxx^Tyyy = 0if and only if xxx ∈ ∂(Kⁿ), yyy ∈ ∂(Kⁿ)and yyy=s Jnxxx for some s>0.

(ii) Let xxx∈Kⁿ. Then xxx^Tyyy>0for all nonzero vectors yyy∈Kⁿ if and only if xxx∈int(Kⁿ).

The next theorem characterizes the three mutually exclusive cases for SOCLCP(M,Kⁿ,qqq).

We point out (C2) can be equivalently restated as −M⁻¹qqq ∈ Kⁿ (which implies that xxx= −M⁻¹qqqis the solution). In (C3),M−s_∗J_nmay or may not be singular.

Theorem 2.1 ([36,37]) There are three mutually exclusive cases for the solution xxx ∈ SOL(M,Kⁿ,qqq), namely:

(C1) qqq∈Kⁿ(which implies that xxx=0is the solution);

(C2) SOL(M,Kⁿ,qqq)⊇ {xxx∈Kⁿ : Mxxx+qqq=0} =∅;

(C3) there exists s_∗>0such that Mxxx+qqq=s_∗Jnxxx∈∂(Kⁿ).

Recalling that M is assumed symmetric positive definite, we then have the following eigendecomposition.

Lemma 2.2 There is a nonsingular matrix V ∈R^n×nsuch that

V^TM V =Ω≡diag(ω1, ω2, . . . , ωn), V^TJnV = Jn, (2.1) where0< ω1and0< ω2≤ · · · ≤ωn.

Proof Since M is assumed symmetric positive definite, it has a Cholesky decomposition M = R^TR. Now notice thatR^−TJnR⁻¹ ∈ R^n×n is symmetric and let its eigenvalues be {μi}ⁿ_i=1. BecauseR^−TJ_nR⁻¹ ∈R^n×n has the same inertia asJ_n, these eigenvalues can be ordered in such a way that

μn ≤ · · · ≤μ2<0< μ1. R^−TJ_nR⁻¹has an eigendecomposition

R^−TJ_nR⁻¹=U DU^T, D=diag(μ1, μ2, . . . , μn), whereUis an orthogonal matrix. Setωi =1/|μi|for 1≤i≤n. We have

R^−TJ_nR⁻¹=U DU^T=U J_nΩ⁻¹U^T=UΩ^−1/2J_nΩ^−1/2U^T.

Finally setV =R⁻¹UΩ^1/2to conclude the proof.

The equations in (2.1) give an eigendecomposition of the matrix pencil M −λJ_n. In particular, it has one positive eigenvalueω1andn−1 negative eigenvalues−ωi:

−ωn≤ · · · ≤ −ω2<0< ω1,

and the columns ofV are the corresponding eigenvectors. In fact, it can be verified that M V J_n =J_nVΩ.

We also have

M Jn(JnV)=M V =V^−TV^TM V =V^−TΩ= JnV JnΩ=(JnV)JnΩ,

where we have usedV^−T = J_nV J_nby the second equation in (2.1). This gives an eigen- decomposition of M J_n. In particular, M J_n has the same eigenvalues as M −λJ_n with corresponding eigenvectors given by the columns ofJnV.

The next theorem provides an algebraic-geometric characterization of the solution set SOL(M,Kⁿ,qqq). The reader is referred to [32,36,37] for proofs and more.

Theorem 2.2 If xxx∈SOL(M,Kⁿ,qqq)is not in the cases(C1)and(C2), i.e., qqq ∈/(−MKⁿ)∪ Kⁿ, then there exists a unique s_∗>0such that Mxxx+qqq=s_∗Jnxxx and

s_∗

⎧⎪

⎨

⎪⎩

=ω1, if(−qqq)^Tvvv=0 or equivalently,qqq∈R(M−ω1J_n),

< ω1, if(−qqq)^Tvvv >0,

> ω1, if(−qqq)^Tvvv <0,

wherevvvis the first column of V , the eigenvector of M−λJ_nassociated withω1: Mvvv=ω1J_nvvv, andR(M−ω1J_n)=vvv^⊥:= {uuu∈Rⁿ :uuu⊥vvv}.

3 Transform SOCLCP into a Zero-Finding Problem Our new method relies on the following simple observation.

Theorem 3.1 ([38])Suppose s∈Ris not an eigenvalue of M−λJn. Let xxx(s)≡

x1(s) xxx₂(s)

:= −(M−s Jn)⁻¹qqq,

where Jnis given in(1.2). Then xxx(s)∈∂(Kⁿ)if and only if x1(s) >0and h(s)=0, where h(s):=xxx(s)^TJnxxx(s)=qqq^T(M−s Jn)^−TJn(M−s Jn)⁻¹qqq.

Using the decompositions in (2.1), we have

xxx(s)= −V(Ω−s J_n)⁻¹V^Tqqq, and

h(s)=qqq^TV(Ω−s J_n)⁻¹J_n(Ω−s J_n)⁻¹V^Tqqq

= ξ₁² (ω1−s)² −

n i=2

ξ_i² (ωi+s)²

= ξ₁² (s−ω1)² −

n i=2

ξ_i²

(s+ωi)², (3.1)

whereξifor 1≤i≤nare the entries ofV^Tqqq, i.e.,

V^Tqqq= [ξ1, . . . , ξn]^T. (3.2) According to Theorem3.1, we know that findings_∗ for the case (C3) in Theorem2.1is equivalent to finding the positive zeros_∗ofh(s).

Lemma 3.1 Supposeξ1=0and not allξifor2≤i≤n are0.

1. If h(0)≥0, i.e., qqq^TM^−TJnM⁻¹qqq≥0, and if(−M⁻¹qqq)₍1)≥0, then xxx= −M⁻¹qqq is in SOL(M,Kⁿ,qqq).

2. If h(0) <0, then h(s)has one and only one zero in(0, ω1); if h(0) >0, then h(s) >0 for all s∈ [0, ω1).

3. If qqq^TJnqqq ≥0, then h(s) >0for all s∈(ω1,∞); if qqq^TJnqqq <0, then h(s)has one and only one zero in(ω1,∞).

Proof For item 1, by assumption,xxx = −M⁻¹qqq∈Kⁿand at the same timeMxxx+qqq=0.

For−ω2<s< ω1, we have h(s)= −2 ξ₁²

(s−ω1)³ +2 n i=2

ξ_i²

(s+ωi)³ >0. Now item 2 is a direct consequence of this.

Consider nows> ω1and letg(s)=(s−ω1)²h(s). We have g(s)=ξ₁²−

n i=2

ξ_i²(s−ω1)²

(s−ω1+ωi+ω1)² =ξ₁²− n i=2

ξ_i²

1+^ω_s−ω^i+ω₁¹₂

is monotonically decreasing andg(ω1)=ξ₁²>0. The functionsh(s)andg(s)has the same sign in(ω1,∞). It can be verified that fors∈(ω1,∞)

g(s) > ξ₁²−

i=2

ξ₂²= lim

s→∞g(s).

It suffices to show thatqqq^TJ_nqqq=ξ₁²−

i=2ξ₂². To this end, we have ξ₁²−

i=2

ξ₂² =(V^Tqqq)^TJ_n(V^Tqqq)=qqq^TV J_nV^Tqqq.

It remains to showV JnV^T=Jn. SinceV^TJnV = Jn, we haveV^T=JnV⁻¹Jnand thus V J_nV^T=V J_n(JnV⁻¹J_n)=J_n,

as expected.

Lemma 3.2 If qqq^TM^−TJ_nM⁻¹qqq≥0and qqq^TJ_nqqq≥0, then either qqq∈Kⁿor−M⁻¹qqq∈Kⁿ. Proof Assume to the contrary that bothqqq ∈/Kⁿ and−M⁻¹qqq∈/Kⁿ, i.e., bothqqq₍₁₎<0 and xxx₍1) >0, wherexxx = M⁻¹qqq. This, together with the conditions thatqqq^TM^−TJnM⁻¹qqq ≥0 andqqq^TJnqqq≥0, imply

xxx₍1)≥ ⁿ

i=2

|x₍i)|², −qqq₍1)≥ ⁿ

i=2

|q₍i)|².

Thus q

qq^Txxx=qqq₍₁₎xxx₍₁₎+ n i=2

q q

q_(i)xxx_(i)≤qqq₍₁₎xxx₍₁₎+ ⁿ

i=2

|x_(i)|^{2 n}

i=2

|q_(i)|²≤0.

On the other hand, since M is symmetric positive definite, so is M⁻¹ and thusqqq^Txxx =

qqq^TM⁻¹qqq>0, a contradiction.

4 Special Case1=0

In this and the next section, we shall devise numerical schemes to efficiently solveh(s)=0 withh(s)given by (3.1).

The caseξ1 =0 corresponds to the case whens_∗ =ω1in [38]. Nowh(s) < 0 for all s> 0, and straightforwardly solvingh(s)=0 for somes_∗ >0 is not going to work. But whens_∗=ω1,Mxxx+qqq=s_∗Jnxxxgives, upon using (2.1),

(Ω−ω1Jn)V⁻¹xxx= −V^Tqqq. (4.1) Although the equation (4.1) is consistent, it only determines uniquely the lastn−1 entries ofV⁻¹xxx withβ := (V⁻¹xxx)(1) completely free. In fact, we will have V⁻¹xxx = −(Ω− ω1Jn)^†V^Tqqq+βeee1, giving

xxx=V[−(Ω−ω1J_n)^†V^Tqqq+βeee1], (4.2a)

Table 1 The sign ofh(s)on(0,∞)in terms of location ofqqq[38]

Cases Where isqqq? Solutionx_∗orh(s)

1 qqq∈Kⁿ 0=xxx_∗∈SOL(M,Kⁿ,qqq)

2 qqq∈ −MKⁿ −M⁻¹qqq=xxx_∗∈SOL(M,Kⁿ,qqq)

3 qqq∈(−Kⁿ)\(−MKⁿ),

qqq∈/R(M−ω1Jn) h(s)=

⎧⎪

⎨

⎪⎩

−fors∈(0,s_∗), 0 fors=s_∗, +fors∈(s_∗, ω1), +fors∈(ω1,∞).

4 qqq∈MKⁿ\Kⁿ, q

q

q∈/R(M−ω1Jn) h(s)=

⎧⎪

⎨

⎪⎩

+fors∈(0, ω1), +fors∈(ω1,s_∗), 0 fors=s_∗,

−fors∈(s_∗,∞).

5 qqq∈/(−MKⁿ)∪MKⁿ∪Kⁿ∪(−Kⁿ), qq

q∈/R(M−ω1Jn) h(s)=

⎧⎪

⎪⎪

⎪⎪

⎨

⎪⎪

⎪⎪

⎪⎩

−fors∈(0,s_∗;1), 0 fors=s_∗;1, +fors∈(s_∗;1, ω1), +fors∈(ω1,s_∗;2), 0 fors=s_∗;2,

−fors∈(s_∗;2,∞).

where(Ω −ω1J_n)^† is the Moore-Penrose inverse ofΩ−ω1J_n [28, p.102]. We have to determineβto makexxx∈∂Kⁿ. To this end, we notice, using (2.1) and (3.2),

xxx^TJnxxx= − n

i=2

ξ_i²

(ωi−ω1)² +β². To makexxx∈∂Kⁿ, we need

β= ± ⁿ

i=2

ξ_i²

(ωi−ω1)² (4.2b)

such that the first component ofxxxis positive.

5 General Case1 =0

This corresponds to the case whens_∗=ω1in [38]. According to Theorem2.2(c),s_∗=ω1

if and only ifvvv^TJnqqq=0or, equivalently,qqq∈/R(M−ω1Jn).

The following theorem due to [38] gives a complete picture ofh(s)on(0,∞)as detailed in Table1for the current case, where, and in what follows, “h(s)= −” meansh(s) <0 and likewise “h(s) = +” meanh(s) > 0. In particular, it shows thath(s)has at least one but at most two positive zeros in(0,∞). For cases 3 and 4 in the table,qqq ∈/R(M−ω1J_n)is redundantly added for clarity since it is in fact implied byqqq ∈(−Kⁿ)\(−MKⁿ)for case 3 and byqqq∈MKⁿ\Kⁿfor case 4.

Theorem 5.1 ([38])If qqq ∈/R(M−ω1J_n)(and thus s_∗=ω1), then

(1) the sign of h(s)for s∈(0,+∞)can be characterized by cases3,4, and5in Table1.

In cases3and4, h(s)has one positive root s_∗, and in case5, it has two positive roots s_∗;1and s_∗;2one of which is s_∗, and

-

6 _...

...

...

...

...

...

...

...

...

...

...

...

...

..

ω1

s∗

0 h(s)

Case 3

-

6 _...

...

...

...

...

...

...

...

...

...

...

...

...

..

Case 4

h(s)

ω1

0

s∗

-

6 _...

...

...

...

...

...

...

...

...

...

...

...

...

..

Case 5

s∗;2

s∗;1

ω1

0 h(s)

Fig. 1 h(s)corresponding to cases 3, 4, and 5 in Table1[38]

(2) lim

s→ω1h(s)= +∞, which combining with Table1, implies thatω1is the unique pole of h(s)in[0,+∞).

Except for the trivial cases: cases 1 and 2, Theorem5.1reveals a complete picture ofh(s), which is illustrated in Figure1. We point out that the three patterns ofh(s)in Figure1are one-to-one corresponding to the three locations ofqqq. In actual computations, we will first check if it is in case 1 or case 2. If neither case occurs, then it must fall into one of the cases 3, 4, and 5. Now the location ofqqqand, thereby, the sign pattern ofh(s)or, equivalently, the relationship betweenω1ands_∗, are then determined.

With the above discussions, we can outline the framework of our algorithm in Algo- rithm5.1. For Algorithm5.1to be workable and efficient, the remaining task is to solve h(s) = 0 for its solutions_∗ > 0 for the three cases as described in Table1and Figure1, whereh(s)is, as given by (3.1),

h(s)= ξ₁² (s−ω1)² −

n i=2

ξ_i²

(s+ωi)². (3.1)

We have two practical ways for this task. One is the Newton method, the standard method to solve any nonlinear equation such ash(s) = 0. The other one is specifically tailored towardsh(s) = 0 by taking the special form ofh(s)into full account. The design of this latter method draws inspiration from the past work on solving secular equations arising from

Algorithm 5.1Linear Complementarity Problem

Input:M(symmetric positive definite),qqq;

Output: an approximation toxxx∈SOL(M,Kⁿ,qqq). 1:ifqqq∈Kⁿthenxxx=0andreturn;

2:if−M⁻¹qqq∈Kⁿthenxxx= −M⁻¹qqqandreturn;

3: compute the decompositions (2.1);

4: compute the vectorwww=V^Tqqqas in (3.2);

5:ifξ1=0then 6: computexxxby (4.2);

7:else

8: ifh(0) <0then

9: find the zeros_∗ofh(s)in(0, ω1)and check ifxxx= −V(Ω−s_∗Jn)⁻¹www∈∂(Kⁿ); if it is,return;

10: end if

11: h(s)must have one and only one zeros_∗in(ω1,∞); find the zeros_∗and computexxx = −V(Ω− s_∗Jn)⁻¹wwwandreturn.

12:end if

the divide-and-conquer method for the symmetric eigenvalue problem [6,9,21]. As expected, the second method performs much better later in our numerical experiments. But before we explain both methods, we will discuss an effective initial guess strategy, similar to what was done in [21].

5.1 Initial Guess Strategy

We have two cases to worry about: eithers_∗∈(0, ω1)ors_∗> ω1. For convenience, write f(s)=

n i=2

ξ_i²

(s+ωi)², h(s)= ξ₁²

(s−ω1)² − f(s).

Cases_∗∈(0, ω1). We first evaluate f(ω1/2)and thenh(ω1/2), as a by-product, which will become useful later. Then we solve

ξ₁²

(s−ω1)² − f(ω1/2)=0 for its smaller root

s₀=ω1− |ξ1|

f(ω1/2)₋1/2

=ω1− |ξ1| _n

i=2

ξ_i² (ωi+ω1/2)²

_−1/2

(5.1) and use the root as an initial guess. We claim thats0 ∈ (0, ω1). To see this, we note that evidentlys0 < ω1. Becauses_∗∈(0, ω1), we have

h(0)= ξ₁² ω₁² −

n i=2

ξ_i²

ω²_i <0 ⇒ ξ₁²

ω²_{1 −1/2}

>

_n

i=2

ξ_i² ω_i²

_−1/2 .

Now

s0> ω1− |ξ1| _n

i=2

ξ_i² (ωi)²

₋1/2

= |ξ1|

⎡

⎣ ξ₁²

ω²_{1 −}1/2

− _n

i=2

ξ_i² ω²_i

₋1/2⎤

⎦>0,

as expected. In fact, more can be said as in the next theorem. We note thats_∗∈(0, ω1/2)if h(ω1/2) >0, ands_∗∈(ω1/2, ω1)ifh(ω1/2) <0. In this regard, Theorem5.2provides a theoretical justification as to whys₀in (5.1) is a good initial guess.

Theorem 5.2 Suppose s_∗ ∈ (0, ω1) and let s₀ be given by (5.1). If h(ω1/2) > 0, then s₀∈(0, ω1/2); if h(ω1/2) <0, then s₀∈(ω1/2, ω1).

Proof We already know thats₀∈(0, ω1). Write f₀= f(ω1/2). By (5.1), we have s0−ω1

2 = ω1

2 − |ξ1|f₀^−1/2= |ξ1|

⎡

⎣ ξ₁²

(ω1/2)² ₋1/2

− f₀^−1/2

⎤

⎦. (5.2)

Ifh(ω1/2) >0, i.e., _(ω^ξ12

1/2)² − f₀ >0, thens₀−ω1/2<0 by (5.2). On the other hand, if h(ω1/2) <0, i.e.,_(ω^ξ12

1/2)² − f0<0, thens0−ω1/2>0 by (5.2).

A word of caution is warranted here in using the formula (5.1) in the case wheres_∗is very close toω1. First, there is a danger, i.e., possible catastrophic cancellation, in computing s0 formulated as in (5.1). Second, in the extreme case,s_∗andω1 may be the same in the working floating point environment. Therefore, for numerical stability, instead of computing s_∗, we should compute the difference betweenω1ands_∗, sayω1−s_∗, and, accordingly, we compute

ω1−s0= |ξ1| _n

i=2

ξ_i² (ωi+ω1/2)²

_−1/2

as an initial guess toω1−s_∗. This idea of computing the difference shares the same reasoning as in the practice of solving secular equations [21] we mentioned moments ago. As a guideline, we propose to computes_∗whenh(ω1/2) >0 for which cases_∗∈(0, ω1/2)orω1−s_∗when h(ω1/2) <0 for which cases_∗∈(ω1/2, ω1)and thusω1−s_∗∈(0, ω1/2).

Cases_∗> ω1. We solve

ξ₁²

(s−ω1)² − f(ω1)=0 for its larger root

s0=ω1+ |ξ1|

f(ω1)_−1/2

=ω1+ |ξ1| _n

i=2

ξ_i² (ω1+ωi)²

_−1/2

and use the root as an initial guess. Evidentlys₀> ω1. 5.2 Newton’s Iteration

As the position of zeros_∗and poles ofh(s)are now clear, we can use the generic Newton iteration

s_k+1=s_k− h(sk)

h(s_k) (5.3)

to finds_∗. Unfortunately, we don’t knowa prioriifs_k+1∈(0, ω1)in the case ofs_∗∈(0, ω1) or ifs_k+1 > ω1in the case ofs_∗> ω1. This could cause problems numerically, but it can be easily overcame by a bisection safeguard strategy, using lower and upper bounds ofs_∗ that are iteratively refined during the iteration process. The basic idea is as follows. Initially

during the computations of the initial guess in Sect.5.1, we determine an interval(α, β)that containss_∗as tightly as we can get at the moment. Consider nowτ =s_k ∈(α, β). We first update the interval according to

cases_∗∈(0, ω1): β←τ ifh(τ) >0;α←τifh(τ) <0; cases_∗> ω1: β←τ ifh(τ) <0;α←τ ifh(τ) >0.

Afters_k+1 in (5.3) is computed, we take it as the next approximation ifs_k+1 ∈ (α, β)or (α+β)/2 otherwise. Repeat the process until|h(sk)| ≤h, a prescribed tolerance, which is set to be 10⁻¹²in our numerical testing.

5.3 Method by Simple Rational Approximation

A more natural way for solvingh(s)=0 is through approximateh(s)by rational approxi- mations sinceh(s)is rational. To simplify notation, we writeτ =s_k ≈s_∗andτ1 =s_k+1. The Newton iteration is the result of approximatingh(s)by its tangent line ats =τ. But given the special form, we propose to approximateh(s)by

g(s)= a

(s−ω1)² − b

(s+ω2)² (5.4)

satisfyingg(τ)=h(τ)andg(τ)=h(τ). One particular argument for using such a rational function is thatg(s)has poles at−ω2andω1, arguably the most important ones ofh(s)for the part ofh(s)we are interested in. Note alsog(s)andh(s)match at the infinities.

We have to computeaandb. For that, we resort tog(τ)=h(τ)andg(τ)=h(τ):

a

(τ −ω1)² − b

(τ+ω2)² =h(τ), −2a

(τ −ω1)³ − −2b

(τ+ω2)³ =h(τ), yielding

a= −(τ−ω1)³(τ+ω2) ω1+ω2

h(τ)

τ+ω2 +h(τ) 2

, b= −(τ−ω1)(τ+ω2)³

ω1+ω2

h(τ)

τ−ω1 +h(τ) 2

. (5.5)

Onceaandbare computed, we solveg(s)=0 for its root in(0, ω1)or to the right ofω1, depending whethers_∗∈(0, ω1)ors_∗> ω1, as the next approximation tos_∗.

The question is whetherg(s)=0 has the desired root. The answer depends on the signs ofaandb. Ideally, we would expect botha>0 andb>0. Plugging inh(τ)andh(τ), we find

a= (ω1−τ)³ ω1+ω2

(ω1+ω2)ξ₁² (ω1−τ)³ −

n i=2

(ωi−ω2)ξ_i² (τ+ω2)³

, b= (τ+ω2)³

ω1+ω2

n i=2

(ω1+ωi)ξ_i²

(τ+ω2)³ >0. (5.6)

Sob > 0 forτ ∈ (0,∞) anda > 0 forτ ∈ (ω1,∞), but it is not clear ifa > 0 for τ ∈ (0, ω1)as well because within the big bracket in the right-hand side of (5.6) it is the difference between two positive numbers whenτ ∈(0, ω1)(unlessn=2). In the case when h(τ) >0, we do havea>0 by (5.5), because

h(τ)= − ξ₁² (τ −ω1)³ +

n i=2

2ξ_i²

(τ+ωi)³ >0 forτ ∈(0, ω1).

Obviously, possiblya <0 is disappointing theoretically, but numerically it may not matter much and conceivably happens extremely rarely. When it does happen, we can employ the same safeguard strategy as explained for the Newton iteration. In what follows, we present a formula for the next approximationτ1 ≈s_∗, assuminga>0, and then comment on what we will do ifa<0, among other undesirable situations.

Assumea>0 (noteb>0 always). In case ofs_∗∈(0, ω1), we seek the smaller rootτ1

ofg(s)=0. Thus we have

ω1−τ1

τ1+ω2 = a

b. (5.7)

It is usually a good idea in an iterative algorithm to compute the difference between the current approximation and the next one because often the difference is smaller in magnitude than the approximation and consequently adding the difference to the approximation yields the next one that is more accurate. Taking this into consideration, we letδ =τ1−τ and rewrite (5.7) as

δ+(τ−ω1)

δ+(τ+ω2) = − a b, yielding

τ1=τ+δ with δ= −(τ−ω1)−(τ+ω2)√ a/b 1+√

a/b . (5.8)

In case ofs_∗> ω1, we seek the bigger rootτ1ofg(s)=0. Thus we have τ1−ω1

τ1+ω2 = a

b ⇒ δ+(τ−ω1) δ+(τ +ω2) = a

b, yielding

τ1=τ+δ with δ= −(τ−ω1)+(τ+ω2)√ a/b 1−√

a/b . (5.9)

Now that we have the formulas forτ1 in (5.8) and (5.9), what we don’t knowa prioriif ω1−τ1∈(0, ω1)in the case of (5.8), or ifτ1−ω1>0 in the case of (5.9). This again could cause problems numerically, but it can be easily overcame by the same safeguard strategy as explained for the Newton iteration.

Part of the reason why we pickg(s)as in (5.4) is because theng(s)=0 is easy to solve, while it keeps two most important poles ofh(s)with respect to its zeros in(0, ω1)and/or (ω1,∞). There are other possible forms equally effective for approximating h(s) for the purpose of computing the zeros, e.g.,

ξ₁²

(s−ω1)² − b

(s+ω2)² +c, a

(s−ω1)² − b

(s+ω2)² +c.

The last one can even be made to matchh(s)up to the second derivative at a point. The only downside is that we have to use another iterative method, e.g., the Newton method, to compute the zeros of these approximating rational functions.

6 Applications of Algorithm5.1

We now provide two applications where our Algorithm5.1for (1.1) can be extended or incorporated into a larger framework to solve more general problems. The first application of SOCLCP (1.1) is just an extension of the single second-order coneKⁿ to a Cartesian product of multiple second-order cones, and our second application is to solve the quadratic programming over the second-order cones (QP-SOC).

6.1 SOCLCP over Multiple Second-Order Cones

Within the matrix-splitting framework [17,24,38], we can apply our Algorithm5.1to solve the general SOCLCP (1.3) overK×m:=Kⁿ¹×Kⁿ²× · · · ×K^nmwhere_m

i=1n_i =n. The matrix-splitting method for solving (1.3) starts by splittingMintoM= B+Cso thatB−C is positive definite¹ (but not necessarily symmetric) in the sense thatxxx^T(B−C)xxx >0 for any 0=xxx∈Rⁿ. Such a splitting(B,C)ofMis saidregular[24]. The convergence of the matrix-splitting method by a regular splitting is guaranteed [37, section 3]. To demonstrate an application of Algorithm5.1for solving (1.3) by the matrix-splitting method, we let

M=

⎡

⎢⎢

⎢⎣

n1 n2 ··· nm

n1 M11 M12 . . . M1m n2 M₂₁ M₂₂ . . . M_2m ... ... ... ... ...

nm Mm1 Mm2 . . . Mmm

⎤

⎥⎥

⎥⎦, B=

⎡

⎢⎢

⎢⎣ M11

M₂₁ M₂₂ ... ... ...

Mm1 Mm2. . . Mmm

⎤

⎥⎥

⎥⎦, (6.1)

and thenC=M−B. They are partitioned according toK×m. It can be verified that (B−C)+(B−C)^T=2 diag(M₁₁,M₂₂, . . . ,M_mm),

which is positive definite and thus the splitting (6.1) is regular. We call the matrix-splitting method [37, Algorithm 1] with this splitting theblock SOR method(BSOR).

Partitionxxx= [xxx^T₁, . . . ,xxx^T_m]^Twithxxx_i∈Rⁿⁱ. It can be seen that

xxx∈K_×m, yyy∈K_×mandxxx^Tyyy=0⇐⇒xxxi ∈Kⁿⁱ, yyyi∈Kⁿⁱ andxxx^T_iyyyi =0

fori=1,2, . . . ,m. Letxxx⁽⁾be theth iterative approximation by the matrix-splitting method [37, Algorithm 1]. The next iterative approximationxxx⁽⁺¹⁾ ∈K_×mis obtained via solving sequentially the following subproblems:

xxxi∈Kⁿⁱ, Miixxxi+t⁽⁾_i ∈Kⁿⁱ, xxx^T_i(Miixxxi+ttt⁽⁾_i )=0 (6.2) fori=1,2, . . . ,m, where

ttt⁽⁾_i :=

# qq

q⁽⁾₁ , ifi=1,

_i−1

j=1M_{i j}xxx⁽⁺¹⁾_j +qqq⁽⁾_i , ifi>1,

andqqq⁽⁾ :=qqq+Cxxx⁽⁾ = [(qqq⁽⁾₁ )^T, . . . , (qqq⁽⁾m)^T]^T. Using Algorithm5.1for the subproblem (6.2), in Sect.7, we will present numerical experiments of BSOR for (1.3).

1Equivalently,(B−C)+(B−C)^Tis symmetric positive definite.