We consider the solution of the large-scale nonsymmetric algebraic Riccati equation XCX− XD−AX+B= 0, withM ≡[D,−C;−B, A]∈R(n1+n2)×(n1+n2) being a nonsingular M-matrix

EQUATIONS BY DOUBLING

TIEXIANG LI^∗, ERIC KING-WAH CHU^†, YUEH-CHENG KUO^‡, AND WEN-WEI LIN^§ Abstract. We consider the solution of the large-scale nonsymmetric algebraic Riccati equation XCX− XD−AX+B= 0, withM ≡[D,−C;−B, A]∈R^{(n1+n2)×(n1+n2)} being a nonsingular M-matrix. In addition, A and D are sparse-like, with the productsA⁻¹u, A^−>u, D⁻¹v and D^−>v computable inO(n) complexity (withn= max{n1, n2}), for some vectorsuandv, andB, Care low-ranked. The structure-preserving doubling algorithm by Guo, Lin and Xu (2006) is adapted, with the appropriate applications of the Sherman-Morrison- Woodbury formula and the sparse-plus-low-rank representations of various iterates. The resulting large-scale doubling algorithm has anO(n) computational complexity and memory requirement per iteration and converges essentially quadratically. A detailed error analysis, on the effects of truncation of iterates with an explicit forward error bound for the approximate solution from the SDA, and some numerical results will be presented.

Keywords. doubling algorithm, M-matrix, nonsymmetric algebraic Riccati equation, nu- merically low-ranked solution

AMS subject classifications. 15A24, 65F50

1. Introduction. Consider the nonsymmetric algebraic Riccati equation (NARE)

R(X)≡XCX−XD−AX+B= 0, (1.1)

whereA,B,CandDare realn1×n1,n1×n2,n2×n1andn2×n2matrices, respectively. From the solvability conditions in [12, 13], we assume the matrix

M =

D −C

−B A

∈R^{(n1+n2)×(n1+n2)} (1.2) is a nonsingular M-matrix, i.e.,M has nonpositive off-diagonal entries and all elements ofM⁻¹ are nonnegative. In this paper, we are interested in developing an efficient algorithm for solving the minimal nonnegative solution X of NAREs in (1.1).

The structure-preserving doubling algorithm (SDA) in [17] is first proposed for solving the NARE (1.1) with quadratical convergence. Then in [5], more general convergence results were given, especially for the critical case. Later in [4], Bini, Meini, and Poloni developed a doubling algorithm called SDA ss, which has shown efficient improvements over SDA in some of numerical tests, but it can happen that sometimes SDA ss runs slower than SDA. Recently in [31], the alternating-directional doubling algorithm (ADDA) has been developed by Wang, Wang and Li to improve the convergence of the SDA dramatically. In practice, ADDA is always faster than SDA and SDA ss, however, it may encounter overflow in Fk andEk beforeHk andGk converge with a desired accuracy, and the scaling technique in [31] is not suitable for the large scale case which we will study.

We state the SDA for solving (1.1) as follows. Choose suitable parameterγ such that γ≥γ0≡max

1≤i≤nmax1

aii, max

1≤i≤n2

dii

, (1.3)

∗Department of Mathematics, Southeast University, Nanjing 211189, People’s Republic of China;

txli@seu.edu.cn(Corresponding Author)

†School of Mathematical Sciences, Building 28, Monash University 3800, Australia;eric.chu@monash.edu

‡Department of Applied Mathematics, National University of Kaohsiung, Kaohsiung 811, Taiwan;

yckuo@nuk.edu.tw

§Department of Applied Mathematics, National Chiao Tung University, Hsinchu 300, Taiwan;

wwlin@math.nctu.edu.tw

1

where aii anddii are the diagonal entries ofAandD, respectively. Compute F0=In₁−2γW_γ⁻¹, E0=In₂−2γV_γ⁻¹,

H₀= 2γW_γ⁻¹BD⁻¹_γ , G₀= 2γD⁻¹_γ CW_γ⁻¹ (1.4) with Aγ ≡A+γIn₁, Dγ ≡D+γIn₂, Wγ ≡Aγ−BD⁻¹_γ C, Vγ ≡Dγ−CA⁻¹_γ B. The SDA [17]

has the form (for k≥0)

F_k+1=F_k(I_n1−H_kG_k)⁻¹F_k, E_k+1=E_k(I_n2−G_kH_k)⁻¹E_k,

H_k+1=H_k+F_k(I_n1−H_kG_k)⁻¹H_kE_k, G_k+1=G_k+E_k(I_n2−G_kH_k)⁻¹G_kF_k, (1.5) where Fk ∈R^n1×n1, Ek∈R^n2×n2, Hk∈R^n1×n2 andGk∈R^n2×n1.

For A = [aij], B = [bij] ∈ R^m×n, we write A ≥B (A > B) if aij ≥bij (aij > bij) for all i, j. A matrixAis called positive (nonnegative) if aij>0 (aij >0). We denote|A|= [|aij|] and kAk:=kAk2the 2-norm of A.

Let

D(Y)≡Y BY −Y A−DY +C= 0 (1.6)

be the dual equation of NARE (1.1). The following convergence theory for (1.5) is originally given in [17] and improved in [5].

Theorem 1.1. Let M in (1.2) be a nonsingular M-matrix. Then the NARE (1.1) and its dual equation (1.6)have minimal nonnegative solutionsX ≥0 andY ≥0, respectively. Moreover S =A−BY andR=D−CX are nonsingular M-matrices. Let S_γ = (S+γI_n1)⁻¹(S−γI_n1) and R_γ = (R+γI_n2)⁻¹(R−γI_n2). Then the sequences{F_k}, {E_k},{H_k} and{G_k} generated by the SDA(1.5) are well-defined, and for allk≥0, we have

(a) E₀, F₀≤0 andF_k= (I_n1−H_kY)S_γ^2k ≥0, E_k= (I_n2−G_kX)R²_γ^k≥0;

(b) I−G_kH_k andI−H_kG_k are nonsingular M-matrices;

(c) 0≤Hk ≤Hk+1≤X and0≤X−Hk= (In₁−HkY)S²_γ^kXR²_γ^k≤S_γ^2kXR²_γ^k; (d) 0≤Gk≤Gk+1≤Y and0≤Y −Gk= (In₂−GkX)R_γ^2kY S_γ^2k≤R²_γ^kY S_γ^2k; (e) S_γ, R_γ ≤0, the spectral radii ρ(S_γ), ρ(R_γ)<1, andS_γ^2k, R_γ^2k →0 ask→ ∞;

(f ) I_n1−XY andI_n2−Y X are nonsingular M-matrices.

Motivated by the low-ranked cases from the applications in transport theory [19, 22, 23], in this paper we further assume thatn₁andn₂are large,AandDare sparse-like (with the products A⁻¹u,A^−>u,D⁻¹vandD^−>vcomputable inO(n) complexity, wheren= max{n₁, n₂}, for some vectors u and v), and B and C are of low-ranked (m and l, respectively, with m, l n1, n2).

In [19, 22, 23], A and D are low-ranked updates of nonsingular diagonal matrices, which are nonsingular but not sparse. We shall adapt the SDA [17] to solve the NARE (1.1), resulting in a large-scale doubling algorithm (SDA ls ε) with anO(n) computational complexity and memory requirement per iteration. Note that the orthodox SDA in [17] has a computational complexity ofO(n³).

More generally, algebraic Riccati equations arise in many important applications, including the total least squares problems with or without symmetric constraints [9], the spectral fac- torizations of rational matrix functions [10], the linear and nonlinear optimal controls [2], the contractive rational matrix functions [20], the structured complex stability radius [18], transport theory [19, 22, 23], the Wiener-Hopf factorization of Markov chains [32], and the optimal solutions of linear differential systems [21]. Symmetric algebraic Riccati equations have been the topic of extensive research, and the theory, applications and numerical solutions of these equations are the subject of [5]–[8] as well as the monographs [21, 29]. The minimal positive solution to the NARE (1.1), for medium size problems without the sparseness and low-ranked assumptions, has

been studied recently by several authors, employing functional iterations, Newton’s method and the structure-preserving algorithm; see [1, 3, 4], [12]–[17], [22, 23, 26, 27, 30, 31] and the refer- ences therein. Evidently, the applications associated with and the numerical solution to NAREs have attracted much attention in the past decade but this paper is the first on general large-scale NAREs.

Main Contribuations. Apart from being the first paper on the numerical solution to gen- eral large-scale NAREs, we shall formalize the discussion on the numerical rank of the solutionX, showing constructively when X is numerically low-ranked. We adapt the well-known structure- preserving doubling method efficiently for large-scale NAREs. Then we show how the exponential growth in the rank of the approximate solution is controlled by compression and truncation. A first order error estimate will show that the difference between the approximate solution by the SDA with small truncation and the exact solution has the same order as the truncation without affecting the convergence of the SDA.

2. Large-Scale Doubling Algorithm. Borrowing from [24], we shall apply the Sherman- Morrison-Woodbury formula (SMWF) in order to avoid the inversion of large or unstructured matrices, and use sparse-plus-low-ranked matrices to represent iterates when appropriate. Also, some matrix operators are computed recursively, to preserve the corresponding sparsity or low- ranked structures, instead of forming them explicitly. If necessary, we compress and truncate fast growing components in the iterates, to trade off the negligible amount of accuracy for better efficiency. Together with the careful organization of convergence control in the algorithm, we obtain an O(n) computational complexity and memory requirement per iteration.

2.1. Large-Scale SDA. We assumeB ∈R^n1×n2andC∈R^n2×n1in (1.1) have, respectively, the full low-ranked decompositions

B =B₁B^>₂, C=C₁C₂^>, (2.1) where B₁ ∈ R^n1×m, B₂ ∈ R^n2×m, C₁ ∈ R^{n2×`</sup>, C<sub>2</sub> ∈ R<sup>n1×`} with m, l n ≡ max{n₁, n₂}.

We first state a basic large-scale SDA, and then propose a practical large-scale SDA later in Section 2.2.

For the initial matrices in (1.4), we have F₀ = I_n1 −2γW_γ⁻¹, E₀ = I_n2 −2γV_γ⁻¹, H₀ = Q₁₀Σ₀Q^>₂₀,and G₀=P₁₀Γ₀P₂₀^>, where

Q10≡2γW_γ⁻¹B1, Q20≡D^−>_γ B2, Σ0≡Im;

P₁₀≡2γD⁻¹_γ C₁, P₂₀≡W_γ^−>C₂, Γ₀≡I_l. (2.2) Note that efficient linear solvers for the large-scaleAandD, and thus forAγandDγ, are available.

Applying the SMWF,W_γ⁻¹wand V_γ⁻¹vcan be computed economically by W_γ⁻¹w=n

In₁+A⁻¹_γ B1

Im−(B₂^>D_γ⁻¹C1)(C₂^>A⁻¹_γ B1)⁻¹

(B^>₂D⁻¹_γ C1)C₂^>o

A⁻¹_γ w, (2.3) V_γ⁻¹v=n

In₂+D_γ⁻¹C1

Il−(C₂^>A⁻¹_γ B1)(B₂^>D_γ⁻¹C1)⁻¹

(C₂^>A⁻¹_γ B1)B^>₂o

D_γ⁻¹v. (2.4) Fork= 1,2,· · ·, we shall organize the SDA so that the iterates have the recursive forms

Hk =Q1kΣkQ^>_2k, Gk=P1kΓkP_2k^>, (2.5) F_k =F_k−1² +F_1kF_2k^>, E_k=E_k−1² +E_1kE_2k^>, (2.6) whereF_ik∈R^n1×lk−1,E_ik∈R^n2×mk−1(i= 1,2), and the kernels Σ_k∈R^mk×mkand Γ_k∈R^lk×lk.

We should compute products likeEku,E_k^>u,Fkv,F_k^>v, for some vectorsuandv, by applying (2.6) recursively. Without actually forming Ek and Fk, we avoid any possible deterioration of their sparse-like properties or other structures as kgrows, and preserve theO(n) computational complexity of the algorithm. As a trade-off, we need to store all the Qik, Σk, Pik, Γk, Fik and Eik for all previousk, as we shall see below.

Applying the SMWF again, we obtain

(In₂−GkHk)⁻¹=In₂+GkQ1kΣk Im_k−Q^>_2kGkQ1kΣk

⁻¹ Q^>_2k

=I_n2+P_1k I_lk−Γ_kP_2k^>H_kP_1k⁻¹

Γ_kP_2k^>H_k, (2.7a) (I_n1−H_kG_k)⁻¹=I_n1+Q_1k I_mk−Σ_kQ^>_2kG_kQ_1k⁻¹

Σ_kQ^>_2kG_k

=In₁+HkP1kΓk Il_k−P_2k^>HkP1kΓk⁻¹

P_2k^>. (2.7b) Denote the direct sum of square matrices by⊕. From (1.5) and (2.7), we can choose the matrices in (2.5) and (2.6) recursively as

Q1,k+1= [Q1k, FkQ1k], Q2,k+1= [Q2k, E_k^>Q2k], (2.8) P1,k+1= [P1k, EkP1k], P2,k+1= [P2k, F_k^>P2k]; (2.9) Σk+1= Σk⊕

Σk+ (Im_k−ΣkQ^>_2kGkQ1k)⁻¹ΣkQ^>_2kGkQ1kΣk

= Σk⊕

Σk+ ΣkQ^>_2kP1kΓk(Il_k−P_2k^>HkP1kΓk)⁻¹P_2k^>Q1kΣk

≡Σ_k⊕Σ˘_k, (2.10)

Γk+1= Γk⊕

Γk+ ΓkP_2k^>Q1kΣk(Im_k−Q^>_2kGkQ1kΣk)⁻¹Q^>_2kP1kΓk

= Γk⊕

Γk+ (Il_k−ΓkP_2k^>HkP1k)⁻¹ΓkP_2k^>HkP1kΓk

≡Γ_k⊕Γ˘_k; (2.11)

F_1,k+1=F_kH_kP_1kΓ_k(I_lk−P_2k^>H_kP_1kΓ_k)⁻¹

=F_kQ_1k(I_mk−Σ_kQ^>_2kG_kQ_1k)⁻¹Σ_kQ^>_2kP_1kΓ_k, (2.12)

F2,k+1=F_k^>P2k; (2.13)

and

E1,k+1=EkGkQ1kΣk(Im_k−Q^>_2kGkQ1kΣk)⁻¹

=EkP1k(Il_k−ΓkP_2k^>HkP1k)⁻¹ΓkP_2k^>Q1kΣk, (2.14)

E_2,k+1=E_k^>Q_2k. (2.15)

Ultimately from (2.6), (2.8) and (2.9), we see that the SDA is dominated by the computation of products like E_ku, E_k^>u, F_kv, F_k^>v, for arbitrary vectors uand v. By applying (2.6) recur- sively, these products can be computed using (2.3) and (2.4) in O(n) complexity and memory requirement, because of our assumptions onA, B, C andD.

The SDA for large-scale NAREs is a competition between the convergence of the doubling iteration and the exponential growth in the dimensions of Qik and Pik. From (2.5), (2.8) and (2.9), we have

rank(Hk)≤rank(Qik)≤2^km, rank(Gk)≤rank(Pik)≤2^kl. (2.16)

Note that 2^km and 2^kl are the numbers of columns in Qik and Pik, respectively. The success of the SDA depends on the trade-off between the accuracy of the approximate solution and its CPU-time and memory requirements, controlled by the compression and truncation of Qik and Pik in Section 2.2. With the truncation and compression process, rank(Qik) and rank(Pik) will be much reduced even with high accuracy for the approximate solutions Hk andGk.

From the convergence results in Theorem 1.1, as well as (2.8), (2.9) and (2.16), the fact that X and Y are numerically low-ranked can be considered constructively. We next define what we mean by being numerically low-ranked of X (and similarly forY):

Definition 2.1. Let X ∈C^n×n.

(i) For a given numerical rank tolerance τ >0, the numerical rank of X with respect toτ, denoted byrank_τX, is defined as the lowest rank ofXb ∈C^n×n such that kXb−Xk6τ.

(ii) X is said to be numerically low-ranked with respect to the numerical rank toleranceτ >0 if rank_τ(X)n.

We first give a useful lemma which is simple but has not appeared in the literature.

Lemma 2.1. For anyA, B∈R^n×r, if0≤A≤B, thenkAk ≤ kBk.

Proof. Since 0 ≤ A ≤B, we have 0 ≤ A^> ≤ B^> and then 0 ≤A^>A ≤ B^>B. From the Perron-Fronbenius Theorem, we have kAk=p

ρ(A^>A)≤p

ρ(B^>B) =kBk.

2.2. Truncation and Compression of Qik and Pik. The truncation and compression process described in this section is necessary when the convergence of the SDA is slow in compar- ison with the exponential growth in the dimensions of the iteratesGk andHk. In this situation, the numerical rank of X will be high and we obviously cannot achieve high accuracy in the ap- proximation Hk of X by any method. We then have to compromise its accuracy for the sake of less memory and CPU-time consumption. We should then either choose larger tolerances for the truncation and compression process (ε_k below), to control the growth in the iterates and adjust them until the accuracy of the approximate solution is acceptable, or simply abandon the truncation and compression process and accept whatever approximate solution obtained within reasonable computing constraints.

We now propose a large-scale SDA with truncation error ε(SDA ls ε). For a given sequence of tolerances ε = {εk}^k_k=0^¯ we first compute truncated initial matrices for the SDA lsε. From (2.2) we compute the QR decompositions

Q10=Qe10R1q, Q20=Qe20R2q, P10=Pe10R1p, P20=Pe20R2p,

where Qe10, Qe20, Pe10 and Pe20 are orthogonal andR1q, R2q, R1p and R2p are upper triangular.

Then we compute the SVD decompositions

R_1qΣ₀R^>_2q= [U₁₀^τ, U₁₀^ε](Σ^τ₀⊕Σ^ε₀)[U₂₀^τ, U₂₀^ε]^>, kΣ^ε₀k< ε₀; R_1pΓ₀R^>_2p= [V₁₀^τ, V₁₀^ε](Γ^τ₀⊕Γ^ε₀)[V₂₀^τ, V₂₀^ε]^>, kΓ^ε₀k< ε₀;

where [U_i0^τ, U_i0^ε] and [V_i0^τ, V_i0^ε] (i = 1,2) are orthogonal, Σ^τ₀ ⊕Σ^ε₀ and Γ^τ₀ ⊕Γ^ε₀ are nonnegative diagonal with Σ^τ₀ ∈R^m0×m0 and Γ^τ₀∈R^l0×l0. By setting

Q^τ₁₀=Qe₁₀U₁₀^τ, Q^τ₂₀=Qe₂₀U₂₀^τ, P₁₀^τ =Pe₁₀V₁₀^τ, P₂₀^τ =Pe₂₀V₂₀^τ, (2.17) we have the (truncated) initial matrices

F₀^τ=F0, E₀^τ =E0, H₀^τ =Q^τ₁₀Σ^τ₀Q^τ₂₀^>, G^τ₀ =P₁₀^τΓ^τ₀P₂₀^τ> (2.18) for the SDA lsε. Let

∆H₀≡H₀−H₀^τ=Qe₁₀U₁₀^εΣ^ε₀U₂₀^ε>Qe^>₂₀, ∆G₀≡G₀−G^τ₀ =Pe₁₀V₁₀^εΓ^ε₀V₂₀^ε>Pe₂₀^>. (2.19)

The truncation errors of the initial matrices can be estimated by

k∆H0k=kΣ^ε₀k< ε0, k∆G0k=kΓ^ε₀k< ε0. (2.20) We repeat this process and suppose it holds at the kstep that

H_k^τ =Q^τ_1kΣ^τ_kQ^τ_2k^>, G^τ_k=P_1k^τΓ^τ_kP_2k^τ>;

E_k^τ =E_k−1^τ2 +E_1k^τ E_2k^τ>, F_k^τ =F_k−1^τ2 +F_1k^τF_2k^τ>; (2.21) where Q^τ_ik andP_ik^τ are orthogonal with widths beingmk andlk (i= 1,2), respectively.

To estimate the (k+ 1)th truncation error, from (2.10)–(2.15) as well as (2.21), we compute Σ˘^τ_k = Σ^τ_k+ Σ^τ_kQ^τ_2k^>P_1k^τΓ^τ_k(I_lk−P_2k^{τ >}H_k^τP_1k^τ Γ^τ_k)⁻¹P_2k^τ>Q^τ_1kΣ^τ_k,

Γ˘^τ_k = Γ^τ_k+ (Il_k−Γ^τ_kP_2k^τ>H_k^τP_1k^τ)⁻¹Γ^τ_kP_2k^τ>H_k^τP_1k^τ Γ^τ_k; (2.22) and

F_1,k+1^τ =F_k^τH_k^τP_1k^τΓ^τ_k Il_k−P_2k^τ>H_k^τP_1k^τΓ^τ_k⁻¹

, F_2,k+1^τ =F_k^τ>P_2k^τ; E_1,k+1^τ =E_k^τP_1k^τ

Il_k−Γ^τ_kP_2k^τ>H_k^τP_1k^τ−1

Γ^τ_kP_2k^τ>Q^τ_1kΣ^τ_k, E_2,k+1^τ =E_k^τ>Q^τ_2k. (2.23) From the QR decompositions

[Q^τ_1k, F_k^τQ^τ_1k] = [Q^τ_1k,Qe_1k]

I S1q

0 R1q

k

, [Q^τ_2k, E_k^τ>Q^τ_2k] = [Q^τ_2k,Qe_2k]

I S2q

0 R2q

k

; [P_1k^τ , E_k^τP_1k^τ ] = [P_1k^τ,Pe1k]

I S1p

0 R1p

k

, [P_2k^τ, F_k^τ>P_2k^τ] = [P_2k^τ,Pe2k]

I S2p

0 R2p

k

, (2.24) we set

Σbk+1≡

I S1q

0 R1q

k

Σ^τ_k 0 0 Σ˘^τ_k

I S2q

0 R2q

>

k

, bΓk+1≡

I S1p

0 R1p

k

Γ^τ_k 0 0 Γ˘^τ_k

I S2p

0 R2p

>

k

. (2.25) We next compute the SVDs

Σbk+1 =

U_1,k+1^τ U_1,k+1^ε

Σ^τ_k+1 0 0 Σ^ε_k+1

U_2,k+1^τ U_2,k+1^{ε >}

,

bΓk+1 =

V_1,k+1^τ V_1,k+1^ε

Γ^τ_k+1 0 0 Γ^ε_k+1

V_2,k+1^τ V_2,k+1^{ε >}

(2.26) withkΣ^ε_k+1k< εk+1 andkΓ^ε_k+1k< εk+1. To truncate, we compute

Q^τ_1,k+1≡[Q^τ_1k,Qe1k]U_1,k+1^τ ∈R^n1×mk+1, Q^τ_2,k+1≡[Q^τ_2k,Qe2k]U_2,k+1^τ ∈R^n2×mk+1; P_1,k+1^τ ≡[P_1k^τ ,Pe1k]V_1,k+1^τ ∈R^n2×lk+1, P_2,k+1^τ ≡[P_2k^τ,Pe2k]V_2,k+1^τ ∈R^n1×lk+1. (2.27) Let

Hbk+1=H_k^τ+F_k^τ(I−H_k^τG^τ_k)⁻¹H_k^τE_k^τ, Gbk+1=G^τ_k+E_k^τ(I−G^τ_kH_k^τ)⁻¹G^τ_kF_k^τ. (2.28) We define the local truncation errors ofH_k+1^τ andG^τ_k+1 as

δH_k+1 ≡Hb_k+1−H_k+1^τ , δG_k+1≡Gb_k+1−G^τ_k+1. (2.29)

From (2.26), we see that kδHk+1k=

[Q^τ_1k,Qe1k]U_1,k+1^ε Σ^ε_k+1U_2,k+1^ε> [Q^τ_2k,Qe2k]^>

=kΣ^ε_k+1k< εk+1, (2.30) kδGk+1k=

[P_1k^τ,Pe1k]V_1,k+1^ε Γ^ε_k+1V_2,k+1^ε> [P_2k^τ,Pe2k]^>

=kΓ^ε_k+1k< εk+1. (2.31) Moreover, we define the global truncation errors of H_k+1^τ andG^τ_k+1 by

∆Hk+1 ≡Hk+1−H_k+1^τ , ∆Gk+1≡Gk+1−G^τ_k+1, (2.32) which will be estimated in Section 3.

The SDA ls εfor solving large-scale NAREs realizes the iterations in (1.5) with initial matrices in (2.18), and the help of (2.3), (2.4), (2.21)–(2.27).

Algorithm 1 (SDA ls ε)

Input: A∈R^n1×n1,B∈R^n1×n2,C∈R^n2×n1,D∈R^n2×n2 withB=B₁B₂^> and C=C₁C₂^> being full low-ranked as in (2.1); suitable shift γas in (1.3);

truncation tolerancesε={εk}^k_k=0^¯ and convergence toleranceεc >0.

Output: H_¯k^τ=Q^τ_1¯kΣ^τ_¯kQ^τ_2¯k^> andG^τ_¯k=P_1¯^τ_kΓ^τ_k¯P_2¯^τ_k^> withQ^τ_i¯k∈R^ni×mk¯, P_j^τ

i,k¯∈R^ni×l¯k orthogonal (i= 1,2;j1= 2, j2= 1), approximating the solutionsX andY to the large-scale NARE (1.1) and its dual equation (1.6), respectively.

Initial matrices:

Set k= 0;

Compute Qi0, Pi0(i= 1,2) in (2.2);

Compute Q^τ_i0, P_i0^τ (i= 1,2) in (2.17) with truncation toleranceε0; Do until convergence:

Compute Σ˘^τ_k, ˘Γ^τ_k as in (2.22) andF_i,k+1^τ , E_i,k+1^τ (i= 1,2) as in (2.23);

OrthogonalizeF_k^τQ^τ_1k, E_k^τ>Q^τ_2k,E_k^τP_1k^τ andF_k^τ>P_2k^τ as in (2.24);

Compute Σb_k+1 andbΓ_k+1 as in (2.25), and their SVDs as in (2.26);

Compute Q^τ_i,k+1,P_i,k+1^τ (i= 1,2) using the toleranceε_k+1 as in (2.27);

Compute k←k+ 1,dk= max{kdH_k^τk,kdG^τ_kk}andrk =kR(H_k^τ)k;

(An economic way for computingkdH_k^τk, kdG^τ_kk andkR(H_k^τ)kcan be found in (4.1), (4.2) and (4.3) in Section 4.1.)

If dk< εc,Setk¯=k;Stop;End If;

End Do

2.3. SDA and Krylov Subspaces. There is an interesting relationship between the SDA and Krylov subspaces. Define the Krylov subspaces

K_k(A, V)≡

span{V} (k= 0),

span{V, AV, A²V,· · · , A^2k−1V} (k >0).

From (1.4), (2.2)–(2.4) and (2.8), we see that

Q10= 2γW_γ⁻¹B1⊆ K0(A⁻¹_γ , A⁻¹_γ B1), Q20=D^−>_γ B2⊆ K0(D^−>_γ , D_γ^−>B2);

Q11= [Q10, F0Q10]⊆ K1(A⁻¹_γ , A⁻¹_γ B1), Q21= [Q20, E₀^>Q20]⊆ K1(D^−>_γ , D^−>_γ B2).

(We have abused notations, with V ⊆ K_k(A, B) meaning span{V} ⊆ K_k(A, B).) Similarly, it is easy to show that

Q_1k ⊆ K_k(A⁻¹_γ , A⁻¹_γ B₁), Q_2k⊆ K_k(D^−>_γ , D_γ^−>B₂);

P_1k ⊆ Kk(D_γ⁻¹, D_γ⁻¹C₁), P_2k ⊆ Kk(A^−>_γ , A^−>_γ C₂).

In other words, the general SDA is closely related to approximating the solutionsX andY using Krylov subspaces, with additional components diminishing quadratically. However, for problems of moderate size n,QikandPikbecome full-ranked after a few iterations.

The link between the SDA and the Krylov subspaces defined above is important in explaining the fast convergence of the SDA. We used to believe the convergence of the SDA came from the following inequalities:

kHk−H_k−1k ≤ kF_k−1kk(In₁−H_k−1G_k−1)⁻¹H_k−1kkE_k−1k, kGk−Gk−1k ≤ kEk−1kk(In2−Gk−1Hk−1)⁻¹Gk−1kkFk−1k,

and the fact thatkE_k−1k,kF_k−1k →0 quadratically, ask→ ∞. This is consistent with numerical results from examples associated withM in (1.2) which is barely a nonsingular M-matrix, where the correspondingEk, Fk →0 slowly but the overall convergence forHk andGk are much faster.

3. Truncation Error Estimates. In this section, we shall estimate the global truncation errors defined in (2.32). For simplicity, we derive only the first order error bounds.

From Theorem 1.1, we have 0≤Hk ≤X, 0≤Gk ≤Y, and 0≤(I−G_kH_k)⁻¹=I+G_kH_k+ (G_kH_k)²+· · ·

≤I+Y X+ (Y X)²+· · ·= (I−Y X)⁻¹ and 0≤Fk = (In₁−HkY)S_γ^2k≤S_γ^2k. By Lemma 2.1, we have

kHkk ≤ kXk, kGkk ≤ kYk, (3.1) and

k(I−GkHk)⁻¹k ≤ k(I−Y X)⁻¹k ≡β1, kFkk ≤ kS_γ^2kk →0. (3.2) Similarly, from Theorem 1.1 and Lemma 2.1 we also have

k(I−HkGk)⁻¹k ≤ k(I−XY)⁻¹k ≡β2, kEkk ≤ kR²_γ^kk →0. (3.3) Denote

ρk= max{kR²_γ^kk,kS_γ^2kk}, α= max{kXk,kYk}, β= max{β1, β2}. (3.4) In the following we abuse the notation ”=” and ” ≡ ”, ignoring the higher order terms.

Suppose that ρ(H_k^τG^τ_k)<1,k∆Hkk andk∆Gkk are sufficiently small. From (2.32) we have the first order approximation of (I−H_k^τG^τ_k)⁻¹:

(I−H_k^τG^τ_k)⁻¹= [I−(H_k−∆H_k)(G_k−∆G_k)]⁻¹

= [I−HkGk+ ∆HkGk+Hk∆Gk−∆Hk∆Gk]⁻¹

= (I−HkGk)⁻¹−(I−HkGk)⁻¹(∆HkGk+Hk∆Gk)(I−HkGk)⁻¹. From (2.19) and (2.20), we have H₀^τ =H₀−∆H₀, G^τ₀ =G₀−∆G₀ with k∆H₀k,k∆G₀k < ε₀. SinceE₀^τ =E₀andF₀^τ =F₀, (2.28) implies

Hb₁=H₀^τ+F₀^τ(I−H₀^τG^τ₀)⁻¹H₀^τE^τ₀

=H0−∆H0

+F₀

(I−H₀G₀)⁻¹−(I−H₀G₀)⁻¹(∆H₀G₀+H₀∆G₀)(I−H₀G₀)⁻¹

(H₀−∆H₀)E₀

=H₀+F₀(I−H₀G₀)⁻¹H₀E₀−∆H₀−F₀(I−H₀G₀)⁻¹∆H₀E₀

−F0(I−H0G0)⁻¹(∆H0G0+H0∆G0)(I−H0G0)⁻¹H0E0

≡H₁−δHb ₁, (3.5)

where bδH1 is the first order truncation error given by

δHb 1= ∆H0+F0(I−H0G0)⁻¹∆H0E0+F0(I−H0G0)⁻¹(∆H0G0+H0∆G0)(I−H0G0)⁻¹H0E0. From (2.29), (2.32) and (3.5), it follows that ∆H1 = H1−H₁^τ = bδH1+δH1. By (2.30) and (3.1)–(3.4), we have

k∆H1k ≤ kδH1k+kbδH1k

≤ε₁+k∆H₀k+kF₀kkE₀kk(I−H₀G₀)⁻¹kk∆H₀k

+kF0kkE0kkH0kk(I−H0G0)⁻¹k²(k∆H0kkG0k+kH0kk∆G0k)

≤ε1+ (1 +ρ²₀β+ρ²₀α²β²)k∆H0k+ρ²₀α²β²k∆G0k.

Similarly, we have

k∆G1k ≤ε1+ (1 +ρ²₀β+ρ²₀α²β²)k∆G0k+ρ²₀α²β²k∆H0k.

From (2.21), we have

F₁^τ=F₀^τ(I−H₀^τG^τ₀)⁻¹F₀^τ =F0(I−H₀^τG^τ₀)⁻¹F0

=F₀(I−H₀G₀)⁻¹F₀−F₀(I−H₀G₀)⁻¹(∆H₀G₀+H₀∆G₀)(I−H₀G₀)⁻¹F₀

≡F1−∆F1,

where ∆F1 =F0(I−H0G0)⁻¹(∆H0G0+H0∆G0)(I−H0G0)⁻¹F0 is the first order truncation error and satisfies

k∆F1k ≤ kF0k²k(I−H0G0)⁻¹k²(k∆H0kkG0k+kH0kk∆G0k)

≤ρ²₀αβ²(k∆H0k+k∆G0k).

Similarly, we also have

k∆E1k ≤ρ²₀αβ²(k∆H0k+k∆G0k).

Performing the (k+ 1)th step in the SDA lsεalgorithm, we obtain Hbk+1=H_k^τ+F_k^τ(I−H_k^τG^τ_k)⁻¹H_k^τE_k^τ

=Hk−∆Hk+ (Fk−∆Fk)(I−HkGk)⁻¹(Hk−∆Hk)(Ek−∆Ek)

−(Fk−∆F_k)(I−H_kG_k)⁻¹(∆H_kG_k+H_k∆G_k)(I−H_kG_k)⁻¹(H_k−∆H_k)(E_k−∆E_k)

=Hk+Fk(I−HkGk)⁻¹HkEk−∆Hk−Fk(I−HkGk)⁻¹(Hk∆Ek+ ∆HkEk)

−∆F_k(I−H_kG_k)⁻¹H_kE_k−F_k(I−H_kG_k)⁻¹(∆H_kG_k+H_k∆G_k)(I−H_kG_k)⁻¹H_kE_k

≡Hk+1−bδHk+1, where

δHb k+1= ∆Hk+Fk(I−HkGk)⁻¹(Hk∆Ek+ ∆HkEk) + ∆Fk(I−HkGk)⁻¹HkEk

+Fk(I−HkGk)⁻¹(∆HkGk+Hk∆Gk)(I−HkGk)⁻¹HkEk

is the first order truncation error. Then (2.30)–(2.32) and (3.1)–(3.4) imply k∆Hk+1k ≤ kδHk+1k+kbδHk+1k

≤ kδHk+1k+k∆Hkk+k(I−HkGk)⁻¹kkFkk(kHkkk∆Ekk+kEkkk∆Hkk) +k(I−HkGk)⁻¹kkHkkkEkkk∆Fkk

+kFkkkEkkkHkkk(I−H_kG_k)⁻¹k²(k∆HkkkGkk+kHkkk∆Gkk)

≤εk+1+ (1 +ρ²_kβ+ρ²_kα²β²)k∆Hkk

+ρ²_kα²β²k∆G_kk+ρ_kαβ(k∆E_kk+k∆F_kk). (3.6)