Polynomial Representation

In this section we will study different ways of interpolating data on equidistant and more general grids using polynomials. We will discuss both accuracy and efficiency and will introduce C++ arrays and other concepts to effectively implement the algorithms.

3.1.1 Vandermonde and Newton Interpolation

Assuming that we have the data available on the discrete set of points{x₀, x₁, . . . , x_N}with corresponding values {f(x₀), f(x₁), . . . f(x_N)}, then we can construct a function f(x) that passes through the pairs (x_i, f(x_i)) by the approximation

f(x)≈p_N(x) =

N k=0

a_kφ_k(x),

where p_N(x) is referred to as the interpolating polynomial, φ_k(x) are a priori known poly-nomials, anda_k are theunknown coefficients. We call φ_k(x) the basis, and its choice is very important in obtaining anefficient approximation. For example, assuming thatφ_k(x) =x^k, k = 0, . . . , N, then we have the following representation at the known pair (x_i, f(x_i))

f(x_i) =a₀+a₁x_i+a₂x²_i +. . .+a_Nx^N_i , i= 0, . . . N .

All together we have (N + 1) such equations for the (N + 1) unknowns a_i, i = 0, . . . , N. This system of equations can be recast in matrix form with the vector of unknowns a^T = (a₀, a₁, a₂, . . . , a_N) as follows

where the matrix V is known as the Vandermonde matrix. This matrix is non-singular because we assume that all {x₀, x₁, . . . x_N} are distinct points, and therefore there exists a unique polynomial of order N that represents this data set. We could obtain the vector of coefficients a from

a=V⁻¹f,

by inverting the Vandermonde matrix V and subsequently performing matrix-vector mul-tiplications with f(x_i). This, however, is an expensive operation with a cost of O(N³) to invert the matrix V (see chapter 9), and it is rarely used in practice.

One approach in reducing the computational complexity is to simply change the basis to φ_k(x) = Π^k_i=0⁻¹(x−x_i),

so f(x) is now approximated by

f(x)≈a₀+a₁(x−x₀) +a₂(x−x₀)(x−x₁) +. . .+a_N(x−x₀)(x−x₁). . .(x−x_N−1). (3.1) Notice that we still use a polynomial basis, but we have simply shifted it with respect to the coordinates of the data points. This simple shift turns out to have a dramatic effect since now the newunknown coefficients can be computed by inverting the following system



which is a lower triangular matrix and requires only O(N²) operations in order to obtain the vector of unknown coefficients. This is done by simple forward substitution, and can be implemented readily using BLAS2.

Remark: It is instructive to compare this method, which is called Newton interpolation, with the Vandermonde interpolation. Assuming that we use Gauss elimination to obtain the vector of unknown coefficients (see chapter 9), we see that the change of basis in the Newton approach takes us directly to the second stage of Gauss elimination, which is the forward substitution, while in the Vandermonde approach we have to essentially perform an LU decomposition of the matrix V, which is an O(N³) operation. However, the Vander-monde matrix is a special one, and its inversion can also be done in O(N²) operations (e.g.

using FFTs, see section 3.2). Thus, the two approaches discussed here are computationally equivalent.

Newton Interpolation: Recursive Algorithm

There is a nice recursive property that we can deduce from Newton’s interpolation method, and which can be used for writing compact C++ code as we shall see in the next section.

Solving for the first few coefficients, we obtain a₀ = f(x₀)

so we see that the coefficient

a_k =F(x₀, x₁, . . . , x_k),

that is the k^th coefficient is a function of the first k function valuesf(x_k). F is a function of both the x_k variables and the f(x_k) data (and hence, in the end, since f(x) is a function of x, then really F is just a function of the x_k’s as given above).

To obtain a recursive relation for the coefficient a_k we need to write the approximation in the grid

G^k₀ ≡ {x_i}, i= 0, . . . k ,

where the subscript denotes the starting index and the superscript denotes the ending index.

To this end, we consider the two subsets

G^k−1₀ ≡ {x₀, x₁, . . . , x_k−1}, and G^k₁ ≡ {x₁, x₂, . . . , x_k},

ofkgrid points each. We also denote the corresponding polynomial approximations byp^k₀(x), p^k₀⁻¹(x) and p^k₁ formed by using the grids G^k₀, G^k₀⁻¹ and G^k₁, respectively. We then observe that

(x₀−x_k)p^k₀(x) = (x−x_k)p^k₀⁻¹(x)−(x−x₀)p^k₁(x), (3.2) as the polynomial p^k₀(x) passes through all the pairs

(x_i, f(x_i)), i= 0, . . . , k.

Next, upon substitution ofp^k₀(x),p^k₀⁻¹(x) andp^k₁(x) in equation (3.2) by their full expansions, which are

p^k₀(x) = a₀+a₁(x−x₀) +. . .+a_k(x−x₀). . .(x−x_k−1) p^k₀⁻¹(x) = a₀+a₁(x−x₀) +. . .+a_k−1(x−x₀). . .(x−x_k−2)

p^k₁(x) = b₁+b₂(x−x₁) +. . .+b_k(x−x₁). . .(x−x_k−1).

and comparing the coefficients of highest polynomial power, x^k, we obtain:

(x₀−x_k)a_k =a_k−1−b_k or

(x₀−x_k)F(x₀, x₁, . . . x_k) =F(x₀, x₁, . . . x_k−1)− F(x₁, x₂, . . . x_k) and therefore

F(x₀, x₁, . . . x_k) = F(x₀, . . . x_k−1)− F(x₁, . . . x_k)

x₀−x_k . (3.3)

We thus obtain the higher divided differences (i.e., coefficients) from the lower ones from equation (3.3).

We illustrate this procedure on a gridG²₀ containing three grid points (x₀, x₁, x₂), so that F(x₀) =f(x₀); F(x₁) =f(x₁); F(x₂) =f(x₂),

then at the next level

F(x₀, x₁) = F(x₀)− F(x₁) x₀−x₁ F(x₁, x₂) = F(x₁)− F(x₂)

x₁−x₂ and

F(x₀, x₁, x₂) = F(x₀, x₁)− F(x₁, x₂) x₀ −x₂ , and so on, for grids with more points.

3.1.2 Arrays in C++

So far, when we have discussed variables in C++, we have referred to single variables, such as the variablesmynode and totalnode presented in section 2.3.4. Now, mathematically, we just introduced a collection of variables in the form of a sequence: x₀, x₁, x₂, ...x_N. If you were to write a program which involved such a sequence of numbers, how would you declare these variables? Of course, to start with, you may use the knowledge you gained from section 2.1.2 to decide how to declare the variables. The variable declaration would look like the following (for N = 5):

double x0,x1,x2,x3,x4,x5;

This does not seem too difficult. However, imagine that you want to use 100 points!

Do you want to type x0, x1, ..., x99? And even more annoying, suppose that you want to compare the results of running a program using 50 points compared to 1000 points! Do not be dismayed; C++ has a solution to your problem! The C++ solution to this problem is the concept of arrays. In C++, you can allocate a block of memory locations using the concepts of arrays. There are two means of accomplishing this: static allocationanddynamic allocation. We will discuss both briefly.

Static Allocation of Arrays

The first means by which you can allocate an array is to staticallyallocate the array. For our purposes, we will take this to mean that prior to both compilation and execution, the size of the array is known. In the previous section, we discussed the idea of using a discrete set of points {x₀, x₁, . . . , x_N} for interpolation. For a specific example, let us take N = 99 (so that the total number of points is 100 points), and let us assume that we want our grid points to be evenly spaced in the interval [0,1].

Software Suite

The following piece of code would statically allocate an array of 100 doubles, and would fill in those variables with their appropriate positions in the interval [0,1]:

#include <iostream.h>

int main(int argc, char * argv[]){

int i;

double x[100];

double dx = 1.0/99.0;

for(i=0;i<100;i++) x[i] = i*dx;

for(i=0;i<100;i++)

cout << "x[" << i << "] = " << x[i] << endl;

}

Let us now examine in detail the statements in this program. First, notice the syntax used for allocating static arrays:

<type> <variable name>[ size ]

Here, size is the number of memory positions that you want allocated. In our example, we wanted 100 doubles to be allocated. Once the allocation is done, how do we access these variables? C++ uses [ ] for accessing variables in an array. In the above allocation, x[0] is the first element, x[1] is the second element, etc. There are several key points for you to realize:

• C++ array indexing always begins at 0. Hence, the first position in an array is always the position denoted by [0].

• C++ does not verify that you do not overrun an array. To overrun an array is to attempt to access a memory location which has not been allocated to the array. In the above example, trying to access x[100] would be illegal because we only allocated an array containing 100 elements (indexed 0, . . . ,99). C++ will not complain when compiling, but may cause a segmentation fault (or even far worse, it may run normally but give the wrong results!). You should be very careful not to overrun arrays!

• When statically allocating arrays, you cannot use a variable for the size parameter.

Hence the following C++ code isinvalid:

int npts = 100;

double x[npts];

Your C++ compiler will complain that this is illegal! This is because it is not until the program is actually executed that the value of npts is known to the program (recall that upon executionnptsis both allocated in memory, and then intialized to the value 100). This type of operationcan be done with dynamic memory allocation, which will also be discussed below.

• We can, however, index the array using variables. In the above example, we are able to iterate through all the values of the array using a forloop.