HAL Id: hal-02928697
https://hal.archives-ouvertes.fr/hal-02928697
Submitted on 2 Sep 2020
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
On a non-archimedean broyden method
Xavier Dahan, Tristan Vaccon
To cite this version:
Xavier Dahan, Tristan Vaccon. On a non-archimedean broyden method. ISSAC ’20: International Symposium on Symbolic and Algebraic Computation, Jul 2020, Kalamata Greece, France. pp.114-121,
�10.1145/3373207.3404045�. �hal-02928697�
Xavier Dahan
Tohoku University, IEHE Sendai, Japan 980-8576
xdahan@gmail.com
Tristan Vaccon
Université de Limoges; CNRS, XLIM UMR 7252 Limoges, France 87060
tristan.vaccon@unilim.fr
ABSTRACT
Newton’s method is an ubiquitous tool to solve equations, both in the archimedean and non-archimedean settings — for which it does not really differ. Broyden was the instigator of what is called “quasi- Newton methods”. These methods use an iteration step where one does not need to compute a complete Jacobian matrix nor its inverse.
We provide an adaptation of Broyden’s method in a general non- archimedean setting, compatible with the lack of inner product, and study its Q and R convergence. We prove that our adapted method converges at least Q-linearly and R-superlinearly with R-order 2
1 2m
in dimension m. Numerical data are provided.
KEYWORDS
System of equations, Broyden’s method, Quasi-Newton, p-adic ap- proximation, Power series, Symbolic-numeric, p-adic algorithm
ACM Reference format:Xavier Dahan and Tristan Vaccon. 2020. On A Non-Archimedean Broyden Method. InProceedings of The 45th International Symposium on Symbolic and Algebraic Computation, Kalamata, Greece, July 2020 (ISSAC’20),9 pages.
DOI:
1 INTRODUCTION
In the numerical world. Quasi-Newton methods refer to a class of variants of Newton’s method for solving square nonlinear systems, with the twist that the inverse of the Jacobian matrix is “approxi- mated” by another matrix. When compared to Newton’s method, they benefit from a cheaper update at each iteration (See e.g. [10, p.49-50, 53]), but suffer from a smaller rate of convergence. They were mainly introduced by Broyden in [6], which has sparked numerous improvements, generalizations, and variants (see the surveys [10, 19]). It is now a fundamental numerical tool (that finds its way in entry level numerical analysis textbooks [8, § 10.3]). To some extent, this success stems from: the specificities of machine precision arithmetic as commonly used in the numerical commu- nity, the fact that Newton’s method is usually not quadratically convergent from step one, and that the arithmetic cost of an itera- tion is independent of the quality of the approximation reached. In another direction, variants of Broyden’s method have known dra- matic success for unconstrained optimization — the target system is the gradient of the objective function, the zeros are then critical points— where it takes advantage of the special structure of the
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner /author(s).
ISSAC’20, Kalamata, Greece
© 2020 Copyright held by the owner/author(s).. . . $15.00 DOI:
Hessian (see Sec. 7 of [10]). Another appealing feature of Broyden’s method is the possibility to design derivative-free methods gener- alizing to the multivariate case the classical secant method (which can be thought of as Broyden’s in dimension one). This feature is a main motivation for this work.
Non-archimedean. It is a natural wish to transpose such a fun- damental numerical method to the non-archimedean framework, offering new tools to perform exact computations, typically for sys- tems with p -adic or power series coefficients. For this adaptation, several non-trivial difficulties have to be overcome: e.g. no inner products, a more difficult proof of convergence, or a management of arithmetic at finite precision far more subtle. This article presents satisfactory solutions for all these difficulties, which we believe can be expanded to a broader variety of quasi-Newton methods.
Bach proved in [1] that in dimension one, the secant method can be on an equal footing with Newton’s method in terms of com- plexity. We investigate how this comparison is less engaging in superior dimension (see Section 6). To our opinion, this is due to the remarkable behavior of Newton’s method in the non-archimedean setting. No inversion of the Jacobian is required at each iteration (simply a matrix multiplication, this is now classical see [5, 16, 17]).
The evaluation of the Jacobian is also efficient for polynomial func- tions (in dimension m , it involves only O(m) evaluations, instead of m
2over
R, see [2]). It displays also true quadratic behavior from step one which, when combined with the natural use of finite pre- cision arithmetic (against machine precision over
R), offers a ratio cost/precision gained that is hard to match.
And indeed, our results show that for large dimension m and polynomials as input, there is little hope for Broyden to outperform Newton, although it depends on the order of superlinear conver- gence of Broyden’s method. In this respect more investigation is necessary, but for now the interest lies more in the theoretical ad- vances and in the situations mentioned in “ Motivations ” thereafter.
Relaxed arithmetic. Since the cost of one iteration of Broyden’s method involves m
2instead of m
ωfor Newton, we should mention the relaxed framework (a.k.a online [11]) which show essentially the same decrease of complexity, while maintaining quadratic conver- gence. It has been implemented efficiently for power series [23], and for p -adic numbers [3]. In case of a smaller m and a larger precision of approximation required, FFT trading [24] has to be mentioned.
These techniques are however unlikely to be suited to the Broyden iteration, since it is a priori not described by a fixed-point equation, a necessity for the relaxed machinery.
Motivations. As explains Remark 6.4, it seems unlikely in the non-archimedean world that with polynomials or rational fractions, a quasi-Newton method meets the standard of Newton’s method.
The practical motivations concern:
1/ Derivative-free method: instead of starting with the Jacobian at
precision one, use a divided-difference matrix. A typical application
ISSAC’20, July 2020, Kalamata, Greece Xavier Dahan and Tristan Vaccon
is when the function is given by a “black-box” and there is no direct access to the Jacobian.
2/ When computing the Jacobian does not allow shortcuts like in the case of rational fractions [2], evaluating it may require up to Lm
2operations, where L is the complexity of evaluation of the input function. Regarding the complexity of Remark 6.4, Broyden’s method then becomes beneficial when L
&m
2− m
ω−1.
3/ While Newton’s method over general Banach spaces of infinite dimension can be made effective when the differential is effectively representable (integral equations [15, § 5][14] are a typical exam- ple), it is in general difficult or impossible to compute it. On the other hand, Broyden’s method or its variants have the ability to work with approximations of the differential, including of finite rank , by considering a projection (as shown in [14, 15] and the ref- erences therein; the dimension of the projection is increased at each iteration). In the non-archimedean context, ODEs with parameters, for example initial conditions, constitute a natural application.
Organization of the paper. Definitions and notations are intro- duced in Section 2. Section 3 explains how Broyden’s method can be adapted to an ultrametric setting. In Section 4, we study the Q and R-order of convergence of Broyden’s method (see Definition 2.1), presenting our main results. It is followed by Section 5, where are introduced developments and conjectures on Q-superlinearity.
Finally, in Section 6, we explain how our Broyden’s method can be implemented with dynamical handling of the precision, and we conclude with some numerical data in Section 7.
2 BROYDEN’S METHOD AND NOTATIONS 2.1 General notations
Throughout the paper, K refers to a complete, discrete valuation field, val : K
Z∪ {
+∞} to its valuation, O
Kits ring of integers and π a uniformizer.
1For k ∈
N, we write O ( π
k) for π
kO
K.
Let m ∈
Z≥1. We are interested in computing an approximation of a non-singular zero x
?of f : K
m→ K
mthrough an itera- tive sequence of approximations, (x
n)
n∈N∈ (K
m)
N. Note that all our vectors are column-vectors. For any x ∈ K
mwhere it is well- defined, we denote by f
0(x ) ∈ M
m(K) the Jacobian matrix of f at x . We will use the following notations (borrowed from [13]):
f
n =f (x
n), y
n=f
n+1− f
n, s
n=x
n+1− x
n(1) We denote by ( e
1, . . . , e
m) the canonical basis of K
m. In K
m, O ( π
k) means O ( π
k) e
1+· · ·
+O ( π
k) e
m.
Newton’s iteration produces a sequence (x
n)
n∈Ngiven by:
x
n+1=x
n− f
0( x
n)
−1· f ( x
n) . (N) For quasi-Newton methods, the iteration is given by:
x
n+1=x
n− B
−1n· f (x
n), (⇒ s
n =−B
−1n· f
n) (QN) with B
npresumably not far from f
0( x
n) . More precisely, it is a generalization of the design of the secant method over K where one approximates f
0( x
n) by
f(xn)−f(xn−1)xn−xn−1
. In quasi-Newton, it is thus required that:
B
n· (x
n− x
n−1)
=f (x
n) − f (x
n−1) (⇒ B
n· s
n−1=y
n−1) (2) By this condition alone, B
nis obviously underdetermined. To miti- gate this issue, B
nis taken as a one-dimensional modification of
1Discrete valuation is only needed in Section 6. For the rest complete and ultrametric
is enough.
B
n−1satisfying (2). Concretely, a sequence ( u
n)
n∈N∈ ( K
m)
Nis introduced such that:
B
n =B
n−1+( y
n−1− B
n−1s
n−1) · u
n−1t. (3)
1
=u
n−1t· s
n−1. (4)
In Broyden’s method over
R, u
n−1is defined by:
u
n−1=s
n−1s
n−1t· s
n−1. (5) The computation of the inverse of B
ncan then be done using the Sherman-Morrison formula (see [22]):
B
−1n =B
−1n−1+(s
n−1− B
−n1−1y
n−1) · s
n−1tB
−n−11s
n−1tB
n−1−1y
n−1. (6)
This formula gives rise to the so-called “good Broyden’s method”.
Using [22] provides the following alternative formulae:
B
n=B
n−1+f
n· u
n−1t. (7) B
n−1=B
−1n−1− B
−n−11f
n· u
n−1tB
n−−11u
n−1tB
n−1−1y
n−1. (8)
2.2 Convergence
We recall some notions on convergence of sequences commonly used in the analysis of the behavior of Broyden’s method.
Definition 2.1 ([20] Chapter 9). A sequence ( x
k)
k∈N
∈ ( K
m)
Nhas Q-order of convergence µ ∈
R>1to a limit x
?∈ K
m, if:
∃r
∈
R+,
∀klarge enough, k x
k+1− x
?k kx
k− x
?k
µ≤ r.
If we can take µ
=1 and r < 1 in the previous inequality, we say that ( x
k)
k∈N
has Q-linear convergence. For µ
=2 , we say it has Q-quadratic convergence. The sequence is said to have Q-superlinear convergence if
lim
k→+∞
kx
k+1− x
?k k x
k− x
?k
=0 . It is said to have R-order of convergence
2µ ∈
R≥1if
lim sup kx
k− x
?k
1/µk< 1 .
Remark 2.2 . For both Q and R, we write has convergence µ to mean has convergence at least µ.
Broyden’s method satisfies the following convergence results:
Theorem 2.3. Over
Rm, under usual regularity assumptions, Broy- den’s method defined by Eq. (5) converges locally
3Q-superlinearly [7], exactly in 2 m steps for linear systems, and with R-order at least 2
1
2m
> 1 [13].
Unfortunately, for general K, Eq. (5) is not a good fit. Indeed, the quadratic form x 7→ x
tx can be isotropic over K
m, i.e. there can be an s
n,0 such that s
nt· s
n=0. This is the case, for example if s
n =(X , X ) in
F2JX
K2
. Consequently, (5) has to be modified. Trying to seek for another quadratic form that would not be isotropic is pointless, since for example there is none over
Qmpfor m ≥ 5 [21].
2R-convergence is a weaker notion, aimed at sequences not monotonically decreasing.
3By locally, we mean that for anyx0andB0in small enough balls aroundx?and f0(x?), the following convergence property is satisfied.
Remark 2.4 . In the sequel, all the B
i’s will be invertible matrices.
Consequently, s
n+1 =0 if and only if f ( x
n)
=0 . We therefore adopt the convention that if for some x
n, we have f ( x
n)
=0 , then the sequences ( x
v)
v≥nand ( B
v)
v≥nwill be constant, and this case does not require any further development.
3 NON-ARCHIMEDEAN ADAPTATION 3.1 Norms
We use the following natural (non-normalized) norm on K defined from its valuation: for any x ∈ K, k x k
=2
−val(x), except for K
= Qp, where we take the more natural p
−val(x)over
Qp. Our norm
4on K can naturally be extended to K
m: for any x
=(x
1, . . . , x
m) ∈ K
m, k x k
=max
i| x
i| . We denote by val( x ) the minimal valuation among the val (x
i) ’s. It defines the norm of x .
Lemma 3.1. Let
g·
gbe the norm on M
m(K) induced by k · k . Let us abuse notations by denoting with k · k the max-norm on the coefficients of the matrices of M
m( K ) . Then
g·
g=k · k .
Proof. Let A ∈ M
n( K ) . If x ∈ K
mis such that k x k ≤ 1 , then by ultrametricity, it is clear that kAx k ≤ kAk, hence
gA
g≤ kAk . If i ∈
Nis such that k A k is obtained with a coefficient on the column of index i , then kAe
ik
=kAk , whence the equality.
Consequently, the max -norm on the coefficients of a matrix is a matrix norm. For rank-one matrices, the computation of the norm can be made easy using the following corollary of Lemma 3.1.
Corollary 3.2. Let a, b ∈ K
mbe two vectors. Then
k a
t· b k
=k a k · k b k . (9)
3.2 Constraints and optimality
For the sequence ( x
n)
n∈N
to be well defined, the sequence ( u
n)
n∈N
must satisfy Eqs (3)-(4) and also:
s
ntB
n−1y
n,0 , (10) to ensure Eq. (6) makes sense. Many different u
n’s can satisfy those conditions. Over
R,Broyden’s choice of u
ndefined by (5) can be characterized by minimizing the Frobenius norm of B
n+1− B
n. We can proceed similarly over K.
Lemma 3.3. If B
n+1satisfies (2) , then:
k B
n+1− B
nk ≥ k y
n− B
ns
nk
k s
nk . (11)
Proof. It is clear as in this case, ( B
n+1− B
n) s
n=y
n− B
ns
n. This inequality can become an equality with a suitable choice of u
nas shown in the following lemma.
Lemma 3.4. Let l be such that val ( s
n,l)
=val ( s
n) . Then u
n =s
n,l−1e
lsatisfies (4) and reaches the bound in (11) .
Nevertheless, this is not enough to have B
ninvertible in general, as we can see from the Sherman-Morrison formula (8):
4OverR, it is of course denoted byk · k∞, but when based on a non-archimedean absolute value, this notation is not used since it is implicitly unambiguous: other norms such as thek · kpare mostly useless.
Lemma 3.5. B
ndefined by Eq. (3) is invertible if and only if u
n−1tB
−1n−1y
n−1,0 . (12) The next lemma shows how to choose l , up to the condition (B
−1n−1y
n−1)
l ,0, which actually never occurs after Corollary 4.3.
Lemma 3.6. Let l be the smallest index such that val ( s
n,l)
=val (s
n). If
B
n−1−1y
n−1
l ,
0 , then
u
n =s
−1n,le
l(13) satisfies Eq. (4) , reaches the bound in Eq. (11) and satisfies Eq. (12) .
4 LOCAL CONVERGENCE 4.1 Local Linear convergence
Let E and F be two finite-dimensional normed vector spaces over K We denote by L ( E, F ) the space of K -linear mappings from E to F . Definition 4.1. Let U be an open subset of E. A function f : U → F is strictly differentiable at x ∈ U if there exists an f
0( x ) ∈ L(E, F) satisfying the following property: for all ε > 0 , there exists a neighborhood U
x,ε⊂ U of x, on which for any y, z ∈ U
x,ϵ:
k f ( z ) − f ( y ) − f
0( x ) · ( z − y )k
F≤ ε · k z − y k
E. (14) Note that both z and y can vary. This property is natural in the ultrametric context (see 3.1.3 of [9]), as the counterpart of Fréchet differentiability over
Rdoes not provide meaningful local infor- mation. Polynomials and converging power series satisfy strict differentiability everywhere they are defined.
We can then adapt Theorem 3.2 of [7] in our ultrametric setting.
Theorem 4.2. Let f : K
m→ K
mand x
?∈ U be such that f is strictly differentiable at x
?, f
0(x
?) is invertible and f (x
?)
=0 . Then any quasi-Newton method whose choice of u
nyields for all n, k u
nk
=k s
nk
−1(which includes Broyden’s choice of Eq. (13) ), is locally Q-linearly converging to x
?with ratio r for any r ∈ ( 0 , 1 ) .
Proof. Let r ∈ ( 0 , 1 ) . Let the constants γ , δ , and λ be satisfying:
γ ≥ kf
0(x
?)
−1k, 0 < δ ≤ r
γ ( 1
+r)( 3 − r) , 0 < λ ≤ δ( 1 −r ). (15) Let η > 0 be given by the strict differentiability at x
?and such that on the ball B(x
?, η),
k f ( z ) − f ( y ) − f
0( x
?) · ( z − y )k ≤ λ · k z − y k .
We restrict further η so as to have: η ≤ δ (1 − r ) . Let us assume that kB
0− f
0(x
?)k ≤ δ , kx
0− x
?k < η.
We have from the condition on δ that δ γ ( 1
+r)( 3 − r ) ≤ r. Since 3 − r > 2 , then 2 δ γ (1
+r ) ≤ r . Consequently,
1
1 − 2 δγ ≤ 1
+r,
the denominator being non zero because δ < ( 2 γ )
−1.
Since k f
0( x
?)
−1k ≤ γ and k B
0− f
0( x
?)k < 2 δ , the Banach Perturbation Lemma ([20] page 45) in the Banach algebra M
m(K) implies that B
0is invertible and:
k B
−10k ≤ γ
1 − 2 γδ ≤ ( 1
+r ) γ .
We can now estimate what happens to x
1=x
0− B
0−1f ( x
0) .
ISSAC’20, July 2020, Kalamata, Greece Xavier Dahan and Tristan Vaccon
k x
1− x
?k
=k x
0− x
?− B
−10f ( x
0)k , (16)
=
k − B
−10f (x
0) − f (x
?) − f
0(x
?) · (x
0− x
?)
− B
0−1f
0( x
?)( x
0− x
?) − B
0( x
0− x
?) k ,
=
k − B
−10f ( x
0) − f ( x
?) − f
0( x
?) · ( x
0− x
?)
− B
0−1( f
0( x
?) − B
0)( x
0− x
?) k ,
≤ k B
0−1k λ k x
0− x
?k
+2 δ k x
0− x
?k ,
≤ kB
0−1k(λ
+2 δ)kx
0− x
?k,
≤ γ (1
+r )( δ (1 − r )
+2 δ )k x
0− x
?k ,
≤ γ (1
+r ) δ (3 − r )k x
0− x
?k by Eq. (15) (middle)
≤ r k x
0− x
?k . (17)
Consequently, k x
1− x
?k ≤ r k x
0− x
?k and k x
1− x
?k ≤ rη < η, i.e. x
1∈ B (x
?, η).
Eq. (3) defines B
1by B
1 =B
0− ( y
1− B
0s
1) · u
1tfor some u
1verifying k u
1k
=k s
1k
−1(see Eqs. (4), Corollary 3.2). Then:
k B
1− B
0k
=k f ( x
1) − f ( x
0) − B
0( x
1− x
0)k · k x
1− x
0k
−1. Therefore,
kB
1− f
0(x
?)k ≤ max
kB
0− f
0(x
?)k , (18) k f (x
1) − f (x
0) − B
0(x
1− x
0)k kx
1− x
0k
−1,
≤ max
kB
0− f
0(x
?)k ,
k B
0− f
0( x
?) ( x
1− x
0)k k x
1− x
0k
−1,
k f ( x
1) − f ( x
0) − f
0( x
?)( x
1− x
0)k k x
1− x
0k
−1,
≤ max (δ, λ) ≤ δ.
We can then carry on and prove by induction that for all k , (i) k x
k− x
?k ≤ r
kk x
0− x
?k , and (ii) B
k∈ B ( f
0( x
?) , δ ) . (19)
Heredity for Inequality (19)-(i) comes from: a same use of the Banach Perturbation Lemma on B
kso that B
kis invertible; that k B
−1kk ≤ ( 1
+r ) γ and by repeating the computations (16) to (17):
k x
k+1− x
?k ≤ k B
kk
−1( λ
+2 δ )k x
k− x
?k ,
≤ ( 1
+r ) γδ ( 3 − r )k x
k− x
?k ,
≤ r kx
k− x
?k.
We can deal with (19)-(ii) using a similar computation as (18):
kB
k+1− f
0(x
?)k ≤ max kB
k− f
0(x
?)k, (20) k f ( x
k+1) − f ( x
k) − B
k( x
k+1− x
k)k k x
k+1− x
kk
−1≤ max k B
k− f
0( x
?)k ,
k f (x
k+1) − f (x
k) − f
0(x
?)(x
k+1− x
k)k kx
k+1− x
kk
−1,
≤ max( δ , λ ) ≤ δ .
Corollary 4.3. Locally, one can take definition (13) to define all the u
n’s and all the B
n’s will still be invertible.
Proof. With the assumptions of the proof of Theorem 4.2, for u
ndefined by (13), ku
n−1k
=ks
n−1k
−1and (4) are satisfied, and by the Banach Perturbation Lemma, B
ndefined by (3) is invertible.
Remark 4.4 . The fact that Broyden’s method has locally Q-linear
convergence with ratio r for any r is not enough to prove that ithas Q-superlinear convergence. Indeed, as x
kis going closer to x
?, there is no reason for B
kto get closer to f
0( x
?) . Consequently, we cannot expect from the previous result that x
kand B
kenter loci of smaller ratio of convergence as k goes to infinity. In fact, in general, B
kdoes not converge to f
0( x
?) .
Finally, the next lemma, consequence of the previous theorem, will be useful in the next subsection to obtain the R-superlinear convergence.
Lemma 4.5. Using the same notations as in the proof of Theorem 4.2, ifr ≤
γkf0(x?) k2
−1
, and k B
0− f
0( x
?)k < δ and k x
0− x
?k < η, then for all n ∈
N,
k f
n+1k ≤ kf
nk .
Proof. Let n ∈
N. We have k s
nk ≤ r k s
n−1k. Indeed, from k x
n+1− x
nk ≤ max (k x
n+1− x
?k , k x
?− x
nk) , and k x
n+1− x
nk <
k x
n− x
?k, we see that k s
nk
=k x
?− x
nk ≤ r k x
?− x
n−1k
=r k s
n−1k.
Then using (QN) and the Q-linear convergence with ratio r, we get that k f
n+1k ≤ r k B
n+1k k B
n−1k k f
nk . Using (20), the definition of δ , γ in (15), and the fact that 0 < r < 1 , we get that kB
n+1k kB
n−1k ≤ 2 γ k f
0( x
?)k , which concludes the proof.
4.2 Local R-superlinear convergence
We first remark that the 2 n -step convergence in the linear case proved by Gay in [13] is still valid. Indeed, it is only a matter of linear algebra.
Theorem 4.6 (Theorem 2.2 in [13]). If f is defined by f ( x )
=Ax − b for some A ∈ GL
m( K ) , then any quasi-Newton method con- verges in at most 2 m steps (i.e. f ( x
2m)
=0 ).
With this and under a stronger differentiability assumption on f , we can obtain R-superlinearity, similarly to Theorem 3.1 of [13].
The proof also follows the main steps thereof.
Theorem 4.7. Let us assume that on a neighborhood U of x
?, there is a c
0∈
R>0such that f satisfies
5∀
x, y ∈ U , k f (x ) − f (y) − f
0(x
?) · (x − y)k ≤ c
0kx −y k
2. (21) Then there are η, δ and
Γin
R>0such that if x
0∈ B ( x
?, η ) and B
0∈ B(f
0(x
?), δ), then for any w ∈
Z≥0,
kx
w+2m− x
?k ≤
Γkx
w− x
?k
2.
Proof. Step 1: Preliminaries. Condition (21) is stronger than strict differentiability as stated in Theorem 4.2. From its proof and Lemma 4.5, let r ∈ ( 0 , 1 ) and γ ≥ k f
0( x
?)
−1k , as well as η and δ such that: r ≤
γkf0(x?) k2
−1
, and if x
0∈ B(x
?, η) and B
0∈ B ( f
0( x
?) , δ ) , the sequences ( x
n)
n∈Nand ( B
n)
n∈Ndefined by Broyden’s method (using (13)) are well defined and moreover the four following inequalities are satisfied: for any k ∈
N,k B
k− f
0( x
?)k ≤ δ , k x
k+1− x
?k ≤ r k x
k− x
?k , kB
−k1k ≤ ( 1
+r )γ , k f (x
k+1)k ≤ k f (x
k)k.
Let x
0∈ B ( x
?, η ) , B
0∈ B ( f
0( x
?) , δ ) , and ( x
n)
n∈Nand ( B
n)
n∈Nbe defined by Broyden’s method. Let w ∈
Nand h
=kx
w−x
?k . We
5This condition is satisfied by polynomials or converging power series.
must show that there is a
Γ,independent of w such that k x
w+2m− x
?k ≤
Γh2.
Step 2: reference to a linear map. Let the linear affine map f ˆ ( x )
=f
0( x
?) x − x
?, and x ˆ
0=x
wand ˆ B
0=B
w. Broyden’s method (us- ing first (13)) applied to those data produces the sequences ( x ˆ
n)
n∈Nand ( B ˆ
n)
n∈N
, which are constant for n ≥ 2 m, as a result of Theorem 4.2. We define similarly ˆ s
n =x ˆ
n+1− x ˆ
n. We have again for all k ∈
Nthe four inequalities:
k B ˆ
k− f
0( x
?)k ≤ δ , k x ˆ
k+1− x
?k ≤ r k x ˆ
k− x
?k , k B ˆ
−k1k ≤ ( 1
+r)γ k f ˆ (x
k+1)k ≤ k f ˆ (x
k)k.
The key to the proof is that ˆ x
2m =x
?and ˆ x
kand x
w+kare not too much far apart.
Step 3: Statement of the induction. More concretely, we prove by induction on j that there exist γ
1,jand γ
2,j, independent of w , such that for 0 ≤ j ≤ 2 m, we have the two inequalities:
kB
w+j− B ˆ
jk · kf
w+jk ≤ γ
1,jh
2, ( E
1,j) k x
w+j− x ˆ
jk ≤ γ
2,jh
2. ( E
2,j) Step 4: Base case. Since B
w =B ˆ
0and x
w =x ˆ
0, ( E
1,0) and ( E
2,0) are clear, with γ
1,0=γ
2,0=0 . Now, let us assume that (E
1,k) and ( E
2,k) are true for a given k such that 0 ≤ k < 2 m.
Step 5: We first prove (E
2,k+1). One part of the inequality (22) is obtained thanks to: B
w−1+k− B ˆ
−k1=B
−w1+k( B ˆ
k− B
w+k) B ˆ
−k1.
k s
w+k− s ˆ
kk
=k B
−1w+kf
w+k− B ˆ
−1kf ˆ ( x ˆ
k)k
≤ max
kB
w−1+kk · k B ˆ
−1kk · kB
w+k− B ˆ
kk · k f
w+kk, (22) k B ˆ
−1kk · kf
w+k− f ˆ ( x ˆ
k)k
≤ k B ˆ
−1kk max
kB
−1w+kk · kB
w+k− B ˆ
kk · kf
w+kk , k f
w+k− f ˆ (x
w+k)k , k f ˆ (x
w+k) − f ˆ ( x ˆ
k)k
(23) The first term on the r.h.s. of (23) is upper-bounded by (1
+r)
2γ
2γ
1,kh
2using (E
1,k) and kB
−w1+kk ≤ ( 1
+r )γ .
For the second term of (23), using (21):
k f
w+k− f (x
?) − f
0(x
?) · (x
w+k− x
?)k ≤ c
0kx
w+k− x
?k
2and k x
w+k− x
?k ≤ k x
w− x
?k
=h, it is upper-bounded by c
0h
2. Finally, the last term is equal to f
0(x
?)(x
w+k− x ˆ
k) whose norm is upper-bounded by k f
0( x
?)k γ
2,kh
2thanks to ( E
2,k) . This is enough to define γ
3,ksuch that ks
w+k− s ˆ
kk ≤ γ
3,kh
2 (‡). Consequently, with γ
2,k+1 =max ( γ
3,k, γ
2,k) , we do have k x
w+k+1− x ˆ
k+1k ≤ γ
2,k+1h
2, and (E
2,k+1) is satisfied.
Step 6.0: We now prove ( E
1,k+1) . We first deal with some pre- liminary cases. If s
w+k =0 , (that is x
w+k+1 =x
w+k) then the property (2) s
w+k =− B
−1w+kf
w+kimplies that f
w+k =0, and the property B
w+k+1s
w+k =y
w+kimplies that f
w+k =f
w+k+1=0.
Thus ( E
1,k+1) is satisfied with γ
1,k+1=0 . If ˆ s
k=0 , then similarly f ˆ ( x ˆ
w+k)
=f ˆ ( x ˆ
w+k+1)
=0. Therefore, as we have seen before,
k f
w+k+1k
=k f
w+k+1− f ˆ ( x
w+k+1)
+f ˆ ( x
w+k+1) − f ˆ ( x ˆ
k+1)k ,
≤ max c
0, k f
0(x
?)kγ
2,k+1h
2.
Then, using that k B
w+k+1− B ˆ
k+1k ≤ max (k B
w+k+1− f
0( x
?)k , k B ˆ
k+1− f
0( x
?)k) ≤ δ, ( E
1,k+1) is satisfied with:
γ
1,k+1=δh
2max c
0, k f
0( x
?)k γ
2,k+1.
Step 6.1 : We can now assume that both s
kand ˆ s
kare non zero.
To prove that there is a γ
1,k+1(independent of w ) such that ( E
1,k+1) holds, then in view of the fact that k f
w+k+1k ≤ k f
w+kk (Lemma 4.5) of ( E
1,k) and of the definition (Eq. (3)) of B
k+1and ˆ B
k+1, it is enough to prove that there is some γ
4,k+1(independent of w ) such that:
k ( y
w+k− B
w+ks
w+k) u
w+kt−
y ˆ
k− B ˆ
ks ˆ
ku ˆ
ktk · k f
w+k+1k ≤ γ
4,k+1h
2. (24) Using that k f
w+k+1k ≤ k f
w+kk (by Lemma 4.5), we obtain:
k f
w+k+1k · k ( y
w+k− B
w+ks
w+k) u
w+kt−
y ˆ
k− B ˆ
ks ˆ
ku ˆ
ktk
≤k f
w+kk max k y
w+k− f
0( x
?) s
w+kk · k u
w+ktk , k(f
0(x
?) − B
w+k)s
w+ku
w+kt− (f
0(x
?) − B ˆ
k) s ˆ
ku ˆ
ktk
≤k f
w+kk max ky
w+k− f
0(x
?)s
w+kk · ku
w+ktk , (25) k( f
0( x
?) − B ˆ
k)( s
w+ku
w+kt− s ˆ
ku ˆ
kt)k , (26)
k( B
w+k− B ˆ
k) s
w+ku
w+ktk
. (27)
Step 6.2: From f
w+k=−B
w+ks
w+k, we have k f
w+kk ≤ ks
w+kk · max (k B
w+k− f
0( x
?)k , k f
0( x
?)k) ≤ k s
w+kk · max ( δ, k f
0( x
?)k)
(•). Otoh by (21), k y
w+k− f
0( x
?) s
w+kk ≤ c
0k s
w+kk
2. It follows that the first term (25) can be upper-bounded in the following way:
(25) ≤ c
0ks
w+kk
3ku
w+ktk max (δ, k f
0(x
?)k) ≤ c
0h
2max (δ, k f
0(x
?)k), the rightmost inequality being obtained from k u
w+ktk
=k s
w+kk
−1and k s
w+kk ≤ max(k x
w+k+1− x
?k , k x
w+k− x
?k)
=k x
w+k− x
?k ≤ k x
w− x
?k
=h .
Step 6.3: The third one (27) can be upper-bounded using ( E
1,k):
(27) ≤ k f
w+kk k( B
w+k− B ˆ
k) s
w+ku
w+ktk ≤ γ
1,kh
2. Step 6.4: For the second one (26), observe that:
s
w+ku
w+kt− s ˆ
ku ˆ
kt=( s
w+k− s ˆ
k) u
w+kt− s ˆ
k( u
w+kt− u ˆ
kt) . (28) The first term is easy to manage using the previous inequality
(•)on k f
w+kk , the inequality
(‡)on ks
w+k− s ˆ
kk and ks
w+kk ku
w+ktk
=1:
k f
w+kk · k( s
w+k− s ˆ
k) u
w+ktk ≤ max( δ, k f
0( x
?)k) γ
3,kh
2. (29) The second one of Eq. (28) is a little bit trickier. Define as in (13), u
w+k =s
w−1+k,le
land ˆ u
k =s ˆ
−1k,ˆ l
e
ˆl
for some given l and l. ˆ If l
=l, ˆ we have: (the last inequality below follows from
(‡)).
k u
w+k− u ˆ
kk
=| s
w+k,l−1− s ˆ
k,l−1|
=| s
w+k,l− s ˆ
k,l|
| s
w+k,l| · |ˆ s
k,l|
=| s
w+k,l− s ˆ
k,l| k s
w+kk · k s ˆ
kk
≤ k s
w+k− s ˆ
kk
k s
w+kk · k s ˆ
kk ≤ γ
3,kh
2k s
w+kk · k s ˆ
kk . From this and from k f
w+kk
=kB
w+kk · ks
w+kk we get:
k f
w+kk · ku
w+k− u ˆ
kk · k s ˆ
kk ≤ γ
3,kmax δ, k f
0(x
?)k h
2. (30) If l
,l, ˆ then either k s
w+k− s ˆ
kk
=k s
w+kk , if k s ˆ
kk ≤ k s
w+kk , or ks
w+k− s ˆ
kk
=k s ˆ
kk, if ks
w+kk ≤ k s ˆ
kk. In the first case, we have
k u
w+k− u ˆ
kk
=k s ˆ
kk
−1,
ISSAC’20, July 2020, Kalamata, Greece Xavier Dahan and Tristan Vaccon
and then, the second term of (28) multiplied by k f
w+kk verifies:
k f
w+kk · k u
w+k− u ˆ
kk · kˆ s
kk ≤ max δ, k f
0( x
?)k k s
w+kk
≤ max δ, k f
0( x
?)k
γ
3,kh
2. (31) The second case follows with the same computation. Eqs (31) (30) (29) prove together the bound on the expression (26) in (28). In turn with the bounds on the terms (25) and (27), prove (24). This concludes the proof of ( E
1,k+1) , and finally the induction.
Step 7: Consequently, kx
w+2m− x ˆ
2mk ≤ γ
2,2mh
2. Thanks to Theorem 4.2, ˆ x
2m =x
?, and thus, we have proved that for any w, k x
w+2m− x
?k ≤ γ
2,2mk x
w− x
?k
2. Theorem 4.7 has for immediate consequence:
Theorem 4.8. Broyden’s method has locally R-order of conver- gence 2
1 2m
.
Proof. Let us take x
0and B
0as in the proof of the previous theorem, and same constants and notations. For any w, k x
w+2m− x
?k ≤
Γk x
w− x
?k
2.
Consequently, for 0 ≤ k < 2 m, l ∈
N, and µ
=2
1/2m, k x
2lm+k− x
?k
µ−2l m−k≤ k x
k− x
?k
2lµ−2l m−kΓ(2l−1)µ−2l m−k≤ k x
k− x
?k
2l2−l−k
2mΓ(2l−1)2−l−2km
≤ k x
k− x
?k
2−k
2mΓ(1−2−l)2−2km
. For simplicity, we can assume that
Γ≥ 1 . Thus,
kx
2lm+k− x
?k
µ−2l m−k≤ kx
k− x
?k
2−k 2mΓ2−2km
.
≤ k x
0− x
?k
2−k 2mΓ2−2km
. Therefore, for k x
0− x
?k small enough, we get that for all k such that 0 ≤ k < 2 m , k x
0− x
?k
2−k
2mΓ2−2km
< 1 , and hence, lim sup
s
k x
s− x
?k
µs< 1 . From 9.2.7 of [20], we then obtain that Broyden’s method do have locally R-order of convergence 2
1
2m
.
5 QUESTIONS ON Q-SUPERLINEARITY
A Q-order of µ implies an R-order of µ. The converse is not true.
Over
R, one of the most important result concerning Broyden’s method is that it is Q-superlinear. The extension of this result to the non-archimedean case remains an open question.
5.1 Dimension 1 : secant method
In dimension one, Broyden’s method reduces to the secant method.
It is known since [1] that the p -adic secant method applied on polynomials has order
Φ,the golden ratio. Its generalization to a general non-archimedean context is straightforward.
Proposition 5.1. Let us assume that m
=1 and on a neighborhood U of x
?, there is a c
0∈
R>0such that f satisfies (21) on U . Then the secant method has locally Q-order of convergence
Φ.Proof. Let us assume that we are in the same context as in the proof of Theorem 4.7, with some Q-linear convergence of ratio r < 1 . Let us define ε
k =x
k− x
?for k ∈
N. For all k ∈
N,
| ε
k+1| < | ε
k| . Then by ultrametricity, | x
k+1− x
k|
=| ε
k|. Also,
we further assume that c
0| ε
0| < | f
0( x
?)| so that for all k ∈
N,
| f
0( x
?) × ( x
k+1− x
k)| > c
0|( x
k+1− x
k)|
2, which also implies by ultrametricity and (21) that for all k ∈
N,|f (x
k+1) − f (x
k)|
=|f
0(x
?) × (x
k+1− x
k)|.
Similarly, | f ( x
k)|
=| f
0( x
?)|| ε
k| .
Now, let n ∈
Z>0. Broyden’s iteration is given by:
x
n+1=x
n− x
n− x
n−1f (x
n) − f (x
n−1) . It rewrites as:
| ε
n+1|
=| ε
n− ε
nf (x
n) − ε
n−1f (x
n)
f ( x
n) − f ( x
n−1) |
=| ε
n−1f (x
n) − ε
nf (x
n−1) f ( x
n) − f ( x
n−1) |
≤ c
0max |ε
n−1||ε
n|
2, |ε
n−1|
2|ε
n|
| f ( x
n) − f ( x
n−1)| ≤ c
0| f
0( x
?)| | ε
n|| ε
n−1| . Let us write C
= |f0(xc0?) |and v
n=Cε
n. Then, v
n+1≤ v
nv
n−1for any n > 0 and consequently,
v
n+1v
Φn≤ v
n1−Φv
n−1≤ v
nv
Φn−1!1−Φ
, as
Φ2=Φ+1 . If we define (Y
n)
n∈Z≥1by Y
1= v1vΦ0
and Y
n+1=Y
n1−Φ, then
vn+1vnΦ
≤ Y
n. Since | 1 −
Φ| < 1 , then Y
nconverges to 1 . Therefore, it is bounded by some D ∈
R+, and
vn+1vnΦ
≤ D for all n ∈
Z≥1. This
concludes the proof.
5.2 General case
Over
R, Broyden’s method is known to converge Q-superlinearly.
The key point is that for any E ∈ M
m(
R) and s ∈
Rm\ { 0 } , k E
I − s · s
t(s
t· s)
k
2F =
k E k
2F
− k Es k
2ksk
22
, (32)
equation ( 5 . 5 ) of [10]. The minus sign is a blessing as it allows the appearance of a telescopic sum which plays a key role in proving that
kxn+1−x?kkxn−x?k
converges to zero. Unfortunately, there does not seem to be a non-archimedean analogue to this equality. Thanks to Theorem 4.7, we nevertheless believe in the following conjecture.
Conjecture 5.2. In the same setting as Theorem 4.7, Broyden’s method has locally Q-superlinear convergence.
6 FINITE PRECISION 6.1 Design and notations
One remarkable feature of Newton’s method in an ultrametric context is the way it can handle precision. For example, if π is a uniformizer, if we assume that k f
0( x
?)
−1k
=1 , x
nknown at precision O(π
2n) is enough to obtain x
n+1at precision O (π
2n+1).
To that intent, it thus suffices to double the precision at each new iteration. Hence the working precision of Newton’s method can be taken to grow at the same rate as the rate of convergence.
The handling of precision is more subtle in Broyden. This is
however crucial to design efficient implementations. Note that in
the real numerical setting, most works using Broyden’s methods
are employing fixed finite precision arithmetic, and do not address
precision. Additionally, the lack of a knowledge of a precise expo- nent of convergence requires special care, and the presence of a division also complicates the matter. We explain hereafter how to cope with those issues.
For simplicity, we will make the following hypotheses through- out this section, which correspond to the standard ones in the Newton-Hensel method. They are that the starting x
0and B
0are in a basin of convergence at least linear. This allows us to replace any encountered x
nby its lift ˜ x
nto a higher precision (and same for B
n). Indeed, x ˜
nwill still be in the basin of convergence and then follows the same convergence property. These liftings allow to mitigate the fact that some divisions are reducing the amount of precision so that only arbitrary added digits are destroyed by the divisions.
6Assumption 6.1. We assume that x
0and x
?are in O
K, and that k f
0(x
?)k
=k f
0(x
?)
−1k
=kB
0k
=kB
−01k
=1 . We also as- sume that some ρ
1≤ 1 and ρ
2≤ 1 are given such that B ( x
?, ρ
1) × B(f
0(x
?), ρ
2), is a basin of convergence at least linear and for any x ∈ B ( x
?, ρ
1) , and ρ ≤ ρ
1, f ( x
+B ( 0 , ρ ))
=f ( x )
+f
0( x
?) · B ( 0 , ρ ) (see the Precision Lemma 3.16 of [9])
The assumption on B
0and f
0( x
?) states that they are unimodu- lar, which is the best one can assume regarding to conditioning and precision. Indeed if M ∈ GL
m( K ) is unimodular ( k M k
=k M
−1k
=1), then for any x ∈ K
m, k Mx k
=k x k . Over
Qp, M ∈ M
m(
Zp) is unimodular if and only if its reduction in M
m(
Z/ pZ ) is invert- ible (and idem for
QJT
Kand
Q). The last assumption is there to provide the precision on the evaluations f (x
k) ’s. It is satisfied if f ∈ O
K[ X
1, . . . , X
m] .
Precision and complexity settings.
Let M (N ) be a superadditive upper-bound on the arithmetic complexity over the residue field of O
Kfor the computation of the product of two elements in O
Kat precision O ( π
N), and L be the size of a straight-line program that computes the system f . One can take M (N ) ∈ O ˜ (N ) .
Working over K with zealous arithmetic, the ultrametric coun- terpart of interval arithmetic [9, § 2.1], the interval of integers [[ a, b [[ indicates the coefficients of an element x ∈ K represented in the computer as x
=Íb−1i=a
x
iπ
i, with x
i∈ O
K/hπ i . In this way val ( x )
=a , its absolute precision is abs ( x )
=b , and its relative preci- sion is rel (x )
=b − a . We recall the usual precision formulae, and assume in the algorithm below that it is how the software manages zealous arithmetic (as in Magma, SageMath, Pari). See loc. cit. for more details.
[[ a, b [[×[[ c, d [[
=[[ a
+c, min ( a
+d, b
+c )[[
[[ a, b [[/[[ c, d [[
=[[ a − c, min ( a
+d − 2 c, b − c )[[ (P) The cost of multiplying two elements of relative precision a and b is within M ( max (a, b)) , and to divide one by the other is in 4 M ( max ( a, b ))
+max ( a, b ) [25, Thm 9.4].
To perform changes in the precision, we use the same notation as Magma’s function for doing so. If x has interval [[ a, b [[ , the (destructive) procedure “ChangePrec(~ x, c )” either truncates x to absolute precision c if c ≤ b , or lifts with zero coefficients 0 π
b+· · ·
+0 π
c−1to fit the interval [[ a, c [[ , if c > b . The non-destructive counterpart is denoted “ChangePrec( x , c )” without ~.
6This an example of an adaptive method, which can also be used in Newton’s method
when divisions occur.
6.2 Effective Broyden’s method
We start from an initial approximation x
0at precision one, for ex- ample given by a modular method. The inverse of the Jacobian at precision one provides B
−10. It yields a cost of O ( m
ω), but the com- plexity analysis of Remark 6.4 shows that it is negligible. Obtaining these data is not always obvious [12], but is the standard hypothesis in the context of modular methods. We write v
k=val ( f
k) ,
In an ideal situation.Assume an oracle provides the valuations
v
0, v
1,v
2, . . . ,v
n, . . . (computed by a Broyden method at arbitrarily
large precision). From this ideal situation, we derive the simple and costless modifications required in reality. This analysis allows us to know how efficient can a Broyden method be, which is noteworthy for comparing it to Newton’s. The implementation of Iteration n ( n
=0 included) follows the lines hereunder. The rightmost inter- val indicates the output interval precision of the object computed (following (P)), while the middle indicates a complexity estimate.
Input: (1) B
n−1has interval [[ 0 , v
n[[ and is unimodular.
(2)
xnhas interval[[0, vn+vn+1[[(non-zero entries in[[0, vn−1+vn[[).(3) f
nhas interval [[ v
n, v
n+vn+1[[.
Output: (i) B
n+−11with interval [[ 0 , v
n+1[[ , (val ( det ( B
n−1))
=0).
(ii)
xn+1in the interval[[0,vn+1+vn+2[[(non-zero entries in[[0,vn+vn+1[[).(iii) f
n+1in the interval [[0 , v
n+1+vn+2[[.
(1) ChangePrec(~ B
n−1, v
n+1) ; [[0 , v
n+1[[
(2) s
n← −B
n−1· f
n; m
2M (v
n+1) [[ v
n, v
n+vn+1[[
(3) x
n+1← x
n+s
n; [[0 , v
n+vn+1[[
(4) ChangePrec(~ x
n+1, v
n+1+vn+2) ; [[ 0 , v
n+1+vn+2[[
(5) f
n+1← f ( x
n+1) ;
.
L · M (v
n+1+v
n+2) [[ v
n+1, v
n+1+vn+2[[
(6) f
n+1← ChangePrec( f
n+1, v
n+vn+1) ; [[v
n+1, v
n+1+vn[[
(7) h
n← B
−1n· f
n+1; m
2M (v
n+1) [[ v
n+1, v
n+vn+1[[
(8) u
n← Eq.(13) ; (negligible) [[− v
n, v
n+1− v
n[[
(9) r
n← u
Tn· ChangePrec( B
n−1, v
n) ;
m2M(vn+1) [[−vn, 0[[(10) ChangePrec(~ f
n+1, 2 v
n) ; [[v
n+1, 2 v
n[[
(11) den ← 1
+r
n· f
n+1; mM (v
n+1) [[ 0 , v
n[[
(12) Num ← h
n· r
n; m
2M ( v
n) [[ v
n+1− v
n, v
n+1[[
(13) N
n← Num/den ; 4 m
2M ( v
n) [[ v
n+1− v
n, v
n+1[[
(14) B
−n+11← B
−n1− N
n; [[ 0 , v
n+1[[
(15) return B
n+−11, x
n+1, f
n+1We emphasize again that thanks to the careful changes of pre- cision undertaken, the precisions are automatically managed by the software, would it have zealous arithmetic implemented. It is then immediate to check that the output verifies the specifications.
Moreover from the positive valuation of N
nit is clear that B
n+1is unimodular. Thus Iteration n
+1 can be initiated with these outputs.
Complexity of the ideal situation.
