Tips and Tricks
Chapter 3. Linear Systems of Equations
4.2 Polynomial Interpolation
4.2.6 Change of Basis, Relation with LU and QR
In the last sections, we discussed different representations of the interpolation polynomial. The first one used aspolynomial basisthemonomials
m(x) = [1, x, x2, . . . , xn].
The Lagrange interpolation formula uses theLagrange polynomial basis l(x) = [l0(x), l1(x), . . . , ln(x)].
Polynomial Interpolation 133
For the Newton interpolation, we use the basis of theNewton polynomials π(x) = [π0(x), π1(x), . . . , πn(x)] with πi(x) =
i−1(
j=0
(x−xj), and for the approximation in the least squares sense, we introduced the basis oforthogonal polynomials, which also interpolates ifk=n:
p(x) = [p0(x), p1(x), . . . , pn(x)]. We introduce the vector of coefficients of the monomials
a= [a0, a1, . . . , an], the vector of the function values
f = [f0, f1, . . . , fn],
the vector of divided differences used for Newton’s interpolation d= [d0, d1, . . . , dn],
and the vector of the coefficients of the orthogonal polynomials b= [b0, b1, . . . , bn].
With this notation, a scalar product describes the four representations of the interpolation polynomial:
Pn(x) =am(x) =fl(x) =dπ(x) =bp(x). (4.23) There must exist matrices which compute the transformations between the polynomial bases. The coefficientsaiused for the monomial basis are, as we discussed before, given by the solution of the linear system Va=f with the Vandermonde matrix V. Now using Equation (4.23) we have
fl=am= (V−1f)m=fV−Tm.
Since the last equation is valid for all possible choices of function valuesf, we can conclude that
l=V−Tm =⇒ Vl=m.
Thus, the transposed Vandermonde matrix maps the Lagrange polynomials to the monomials,
Note that Equation (4.24) is just the Lagrange interpolation representation of the monomials!
There must be a similar relationship between the Lagrange and Newton polynomials. Let us denote byUd=f with
the linear system (4.12) with which the divided differences can be computed from the function values. Using again Equation (4.23) we obtain
dπ=fl=dUl =⇒ Ul(x) =π(x).
Thus, the upper triangular matrix U maps the Lagrange polynomials to the Newton polynomials: Note again that this result is obvious since (4.25) is the Lagrange represen-tation of the Newton polynomials!
Next we would like to find a relation between the monomials and the Newton polynomials. We are looking for a lower triangular matrix Lwith Lπ(x) =m(x). Lhas to be lower triangular, since the Newton polynomials and the monomials have the same degree. To determine Lwe need to know more aboutdivided differences.
The divided differences may be interpreted as an elimination process. Let the function Hp(x0, . . . , xk) denote the sum of all homogeneous products of
For these homogeneous products the followingLemma of Miller [91] holds:
Lemma 4.2.
Hp(x0, . . . , xk) =Hp+1(x0, . . . , xk−1)−Hp+1(x1, . . . , xk)
x0−xk (4.26)
Polynomial Interpolation 135
Proof. We rewrite the right-hand side of (4.26) as Hp+1(x0, . . . , xk−1)−Hp+1(x1, . . . , xk)
x0−xk =
p+1
s=1(xs0−xsk)Hp+1−s(x1, . . . , xk−1)
x0−xk .
But
xs0−xsk x0−xk
=
s−1
q=0
xq0xq+1−sk =Hs−1(x0, xk).
Thus, the right-hand side of (4.26) is equal to p
s=0
Hs(x0, xk)Hp−s(x1, . . . , xk−1) =Hp(x0, . . . , xk).
Consider now the interpolation polynomial expanded in monomials:
Pn(x) =a0+a1x+· · ·anxn,
and the system of linear equations (the interpolation condition) with the Vandermonde matrix
n j=0
ajxji =f(xi), i= 0, . . . n. (4.27)
Theorem 4.5. The divided difference scheme eliminates the unknown coefficients of the interpolation polynomial in (4.27). More precisely,
f[xi, . . . , xi+k] =ak+ n j=k+1
ajHj−k(xi, . . . , xi+k). (4.28)
Proof. The proof is by induction on the columns. For the base case, we have
f[xi, xi+1] = n
j=0ajxji−n
j=0ajxji+1 xi−xi+1
=a1+ n j=2
ajHj−1(xi, xi+1).
Thus, the theorem is valid for the second column. Assume now it holds for the first k−1 columns. Then
f[xi, . . . , xi+k] = f[xi, . . . , xi+k−1]−f[xi+1, . . . , xi+k] xi−xi+k
= n
j=kaj(Hj−k+1(xi, . . . , xi+k−1)−Hj−k+1(xi+1, . . . , xi+k)) xi−xi+k
.
By multiplying the denominator into the sum and by applying Lemma 4.2, As a conclusion, if the interpolated valuesf(xi) are values of a polynomial of degreenthen
f[x0, . . . , xn] =an= Pn(n)(0)
n! = leading coefficient ofPn (4.29) andf[x0, . . . , xn+k] = 0 fork= 1,2, . . ..
Furthermore, (4.28) gives us the relation between the divided differences diand the coefficients ai. We haveLa=dwhere
From Equation (4.23) we obtain
am=dπ=aLπ =⇒ Lπ(x) =m(x).
Thus we obtained the result: the lower triangular matrixLmaps the Newton polynomials to the monomials.
From the relation Vl= mbetween the Lagrange polynomials and the monomials and fromUl=πrelating the Lagrange polynomials to the New-ton polynomials we can eliminate land thus also obtain the connection be-tween the Newton polynomials and the monomials
VU−1π=m =⇒ VU−1=L ⇐⇒ V=LU.
The last equation tells us that the transposed Vandermonde matrix has been factored into the product of a lower and an upper triangular matrix. Since the diagonal of L is one, the factorization is unique and nothing else than the factorization obtained by applying Gaussian elimination toV.
Theorem 4.6. The LU decomposition of the Vandermonde matrix
V =
Polynomial Interpolation 137
The coefficients of the interpolating polynomial in the basis of monomials are the solution ofVa=f. They may be computed using the LU decomposition V =UL by
1. solvingUd=f by forward substitution, giving the intermediate vector dof the divided differences and
2. solving La=dby back substitution.
The polynomial basis can be computed by solving Vl(x) = m(x) also usingV=LU:
1. forward substitution forLπ(x) =m(x), yielding as intermediate vector the Newton polynomials.
2. backward substitution forUl(x) =π(x), giving the Lagrange polynomi-als.
UsingMaple, we can confirm that the theorem indeed holds;
with(LinearAlgebra):
V:=VandermondeMatrix([x_0,x_1,x_2,x_3]);
VT:=Transpose(V);
(p,L,U):=LUDecomposition(VT);
U:=map(factor,U);
We obtain as predicted the decomposition
L=
U=
⎡
⎢⎢
⎢⎣
1 1 1 1
0 x1−x0 x2−x0 x3−x0
0 0 −(x2−x1) (−x2 +x0) (−x3+x1) (−x3 +x0) 0 0 0 −(−x3+x1) (−x3+x2) (−x3+x0)
⎤
⎥⎥
⎥⎦.
For the orthogonal polynomial basis, the interpolation condition is b0p0(xj) +b1p1(xj) +· · ·+bnpn(xj) =fj, j= 0, . . . , n.
The matrix of this linear system for the coefficients bi,
P =
⎛
⎜⎜
⎜⎝
p0(x0) p1(x0) · · · pn(x0) p0(x1) p1(x1) · · · pn(x1)
... ... ... ... p0(xn) p1(xn) · · · pn(xn)
⎞
⎟⎟
⎟⎠,
is orthogonal: PP =D2whereD= diag{p0,p1, . . . ,pn}. Thus, as we have seen before, the solution is easy to compute,
b=D−2Pf. Substituting this into (4.23), we get
Pn(x) =fl(x) =bp(x) =fP D−2p(x).
Since this is true for allf, we conclude
l(x) =P D−2p(x) ⇐⇒ Pl(x) =p(x).
There must exist a lower triangular matrixGwhich maps the orthogonal polynomials to the monomials
Gp(x) =m(x).
If we write this equation forx=x0, x1, . . . , xnthen we get forGthe matrix equation
GP=V ⇐⇒ V =P G=
P D−1 DGT .
Thus we have decomposed the Vandermonde matrix V into a product of an orthonormal matrix Q=P D−1 and an upper triangular matrix R=DGT, this is the QR decomposition, which we introduced on the one hand to solve in a very stable fashion linear systems, see Section3.5, and also for the solution of least squares problems, see Subsection 6.5.1.
Therefore, to compute the matrixG, we can compute the QR factoriza-tionV =QRin a standard way using e.g. Householder reflections and then compute G = RD−1. Alternatively, if we know V and P, we can solve
Polynomial Interpolation 139
for G in GP = V to get G = VP D−2. Since the two representations for the interpolating polynomialp(x)b=m(x)aboth hold, we obtain by replacing m(x) =Gp(x) the relation
Ga=b.
Summarizing all the results we obtain the following theorem.
Theorem 4.7. Letpk(x)be the orthogonal polynomial of degreek,p(x) = (p0(x), p1(x), . . . , pn(x)) andP = [p(x0),p(x1), . . . ,p(xn)]. Let D be the diagonal matrix D= diag{p0,p1, . . . ,pn}and letV =QRbe the QR decomposition of the Vandermonde matrix. Then
1. Q=P D−1 andR=D−1PV.
2. The basis transformation between the Lagrange polynomials li(x)and the orthogonal polynomialspi(x)isPl(x) =p(x).
3. The basis transformation between the orthogonal polynomialspi(x)and the monomials mi(x) = xi is Gp(x) = m(x) where G = DR = VP D−2.
4. The coefficients of the monomials of the interpolating polynomialaiand the coefficients of the orthogonal polynomialsbiare related byGa=b.