HAL Id: hal-01311725
https://hal.archives-ouvertes.fr/hal-01311725
Submitted on 4 May 2016
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Distributed under a Creative Commons Attribution - ShareAlike| 4.0 International
Wavelet-Galerkin solution of boundary value problems
Kevin Amaratunga, John Williams
To cite this version:
Kevin Amaratunga, John Williams. Wavelet-Galerkin solution of boundary value problems. Archives
of Computational Methods in Engineering, Springer Verlag, 1997, �10.1007/BF02913819�. �hal-
01311725�
Problems
K. Amaratungaand J.R.Williams
IntelligentEngineeringSystemsLab oratory,
MassachusettsInstituteofTechnology,
Cambridge,MA02139,USA
In this pap er wereview the application of waveletsto the solutionof partial dierential equations. We
considerindetailb oththesinglescaleandthemultiscaleWaveletGalerkinmetho d. Thetheoryofwavelets
isdescrib edhereusingthelanguageandmathematicsofsignalpro cessing. Weshowametho dofadapting
wavelets to an interval using an extrap olation technique called Wavelet Extrap olation. Wavelets on an
interval allow b oundary conditions to b e enforced in partial dierential equations and image b oundary
problems to b e overcome in image pro cessing. Finally,we discuss the fast inversion of matrices arising
from dierential op erators by preconditioning the multiscale waveletmatrix. Wavelet preconditioningis
shownto limitthe growth ofthe matrix conditionnumb er, suchthatKrylov subspaceiterationmetho ds
canaccomplishfastinversionoflargematrices.
1 ORGANIZATION OF PAPER
Wehave foundthatwaveletscanbemosteasilyunderstoodwhentheyareviewed aslters.
Inthis paperwe seektoprovidearoadmap forthoseincomputationalmechanics whowish
tounderstand howwavelets can be usedtosolve initial andboundaryvalueproblems.
Theproperties ofwavelets can bededuced from considerations offunctional spaces, as
wasshownbyMorlet[1][2],Meyer[4][5],andDaubechies[6][7]. However,waveletscanalso
be viewed from a signalprocessing perspective, as proposedbyMallat[8],[9]. The wavelet
transformisthenviewedasalterwhichactsuponaninputsignaltogiveanoutputsignal.
By understandingthepropertiesoflters we candevelop thenumerical toolsnecessary for
designing wavelets which satisfy conditionsof accuracyand orthogonality.
First,weremindthereaderofhowafunctioncanbeapproximated byprojectionontoa
subspace spanned byasetof basefunctions. Wenotethatsome subspacescanbespanned
bythetranslates of asingle function. This iscalleda Reisz basis.
Theprojectionof a functionontoa subspace can be viewed asaltering process. The
lter is determined by the lter coeÆcients and our goal is to choose good coeÆcients.
We discuss the properties of lters which are desirable in the context of solving partial
dierentialequations. In Section3 we revise some key conceptsof signalprocessing which
indicate how to design these lter coeÆcients. Using signal processing concepts we show
thatwaveletltersshouldbeconstrainedtobelineartimeinvariant,stable,causalsystems.
Theseconstraintsdeterminetheformthatthewaveletfrequencyresponsemusttakeandwe
can deducethatthepolesandthezeros ofawavelet system mustliewithin theunitcircle.
Withthis backgroundinplace,we proceedtodescribe themethodusedbyDaubechies for
designing orthogonalwavelets.
We then address the solution of partial dierential equations using wavelets as the
basis of our approximation. We note that the frequency responses of the wavelet lters
are intimately linked totheir approximation properties, via theStrang-Fix condition. We
examine the imposition of boundary conditions which leads to the problem of applying
wavelets on a nite domain. A solution tothis problem called theWavelet Extrapolation
Method is described. Finally results using wavelet based preconditioners for the solution
0 1 2 3
−2
−1 0 1 2
0 1 2 3
−2
−1 0 1 2
0 1 2 3
−2
−1 0 1 2
0 1 2 3
−2
−1 0 1 2
Figure1. HaarFunction. (a)(x). (b)(x 1). (c)(2x 1). (d) (x 1)
of partial dierentialequations are presented. It is shown that these preconditioners lead
toorderO(N)algorithmsformatrix inversion.
2 BASISFUNCTIONS DEFINING A SUBSPACE
In this section we show how we can project a function onto a subspace spanned by a
given set of basis functions. Given this approximation, we then show how to lter it
into two separate pieces. One lter picks out the low frequency content and the other
the high frequency content. This ltering into two subbands is the key to developing a
multiresolution analysis.
First, lets see how to project a function onto a subspace spanned by a set of basis
functions. WeintroducetheHaarfunction(Figure1)asanexamplewaveletscalingfunction
whichis easily visualized. The Haarfunction isrelatively simple but illustratesadmirably
mostofthe propetiesof wavelets.
(x)=f
1 0x<1
0 otherwise:
(1)
In our discussion of wavelets we shall be concerned with two functions, namely the
scaling function (x) and its corresponding wavelet function (x). The lter related to
(x) lters out the low frequency part of a signal and the lter related to (x) lters
outthe high frequency part. As we shall see by examining a simple example, the scaling
function and wavelet are intimately related. Once one is dened, then the other can be
easily deduced. TheHaar functioncorresponds toascaling function.
n;k
be denedas,
n;k
(x)=2 n
2
(2 n
x k): (2)
Consider rst thegeneralscaling functionatleveln=0,
0;k ,
0;k
=(x k): (3)
These functions, for integer k, are orthogonal under translation, as illustrated in Figure
1(a), which shows
0;0
and (b), which shows
0;1
. We note that the
0;k
span the whole
functionspace and we can approximate any function f(x) using
0;k
as a basis, as given
below:
f 1
X
k = 1 c
k
0;k
= 1
X
k = 1 c
k
(x k) (4)
The approximation at the level n = 0 is called the projection of f onto the subspace V
0
spannedby
0;k
and iswritten,
P
0 f =
1
X
k = 1 c
0k
0;k
(5)
The accuracy of the approximation depends on thespan of theHaar function. In the
caseof the
0;k
, thespan is1, and the structure of the functioncannot be resolved below
thislimit.
Now, we can change thescale of thefunction by changing n from n=0 tosay n=1.
Figure 1(c) shows the Haar function at this scale for the translation k = 1 i.e.
1;1 .
Once again we note that thefunction spans the space but at a higher resolution. We can
improvetheaccuracyoftheapproximationbyusingnarrowerandnarrowerHaarfunctions.
However, the apporximationis always piecewise constant and the order of convergence is
xed. Laterweshall seek waveletswith betterconvergence properties.
Thefunctionswhichwenormallydealwithinengineeringbelongtothespaceofsquare
integrable functions, L 2
(R), and arereferred toasL 2
-functions. TheP
n
f samplingof the
L 2
-function f belongs to a subspace, which we shall call V
n
. The P
n
f form a family of
embedded subspaces,
V
1 V
0 V
1 V
2
(6)
For example, suppose we have a function f(x) dened in [0;1]. We can sample the
functionat2 n
uniformlyspaced pointsand createa vector
f
0 f
1 f
2 f
3
::: f
2 n
2 f
2 n
1
T
where
f
k
=f
k
2 n
: (7)
Thisvectorrepresentsadiscretesamplingofthefunctionandissaidtobetheprojection
ofthefunctionon thespace represented bythevector. Asnotedpreviously theprojection
iswrittenP
n
f and thespace is writtenasV
n .
Suppose nowwesampleathalfthenumberofpoints(2 n 1
). Wecall thisistheprojec-
tionof f onV
n 1
andwe writeit asP
n 1
f. Foruniformly spaced points thisis essentially
n 1
V
n
. This leads tothesequence ofembedded subspaces
V
1 V
0 V
1 V
2
(8)
In this concrete example we have chosen the f
k
tobe the value of the functionat the
point x= k
2 n
,sothatthe projectionof thefunctionon V
n is
P
n f(x)=
2 n
1
X
k =0 f
k 2
n
Æ(2 n
x k); k Z (9)
Here, the basis functions are delta functions. In general we can choose to expand the
function interms of any set of basis functions we please. If the basisfunctions for V
n are
chosen to be the functions
n;k
dened by the notation
n;k
= 2 n
2
(2 n
x k), then the
projectionon V
n is
P
n f(x)=
2 n
1
X
k =0 c
k
n;k
; k Z (10)
andthe vector is
c
0
; c
1
; c
2
; c
3
; ::: c
2 n
2
; c
2 n
1
T
We note that the translates (dened by (x k)) of such a function span and dene
a subspace. The factor of 2 n
which multiplies the free variable x in a scaling function
determinesspan ofeach function.
We have seenthat wecan projecta functionontoa subspace spanned by asetof basis
functions. Belowweshallrequire thatthebasisfunctionssatisfy certainproperties. Atthis
stage we do not know if such basis functions can be found. However, if they do exist we
shallrequire thatthey satisfytheconstraintswhich we nowdevelop. (To convince yourself
thatthey doexist you may want tosubstituteintheHaar function.)
We now consider the property which gives wavelets their multiresoltion character,
namely the scaling relationship. In general, a scaling function (x) is taken as the so-
lutiontoadilation equationof theform
(x)= 1
X
k = 1 a
k
(Sx k) (11)
Aconvenient choice of thedilationfactoris S =2,inwhich casethe equationbecomes
(x)= 1
X
k = 1 a
k
(2x k) (12)
Thisequationstatesthatthefunction(x)canbedescribedintermsofthesame function,
but ata higher resolution. (Of course, itremains forus tond functions forwhich this is
true.) The constant coeÆcientsa
k
are calledltercoeÆcients and itis oftenthecase that
only a nite number of these arenon zero. The lter coeÆcients are derived byimposing
certain conditions on the scaling function. One of these conditions is that that scaling
functionand itstranslates should form anorthonormal set i.e.
R
1
1
(x)(x+l )dx=Æ
0;l
l Z
Æ
0;l
=f
1; l=0;
0; otherwise;
Thewavelet (x)ischosentobeafunctionwhichisorthogonaltothescalingfunction.
A suitable denition for (x) is
(x)= 1
X
k = 1 ( 1)
k
a
N 1 k
(2x k) (13)
where N isan even integer.
Thissatises orthogonalitysince
h(x); (x)i = R
1
1 P
1
k = 1 a
k
(2x k) P
1
l = 1 ( 1)
l
a
N 1 l
(2x l )dx
= 1
2 P
1
k = 1 ( 1)
k
a
k a
N 1 k
=0
Thescaling relationshipdenestwo subspaces, V
n and V
n 1
ontowhichwecan project
a given function. The question now arises as to what is the dierence between these two
projections. Letus postulate a subspace W
n 1
that isorthogonaltoV
n 1
such that:
V
n
=V
n 1
W
n 1
We are atliberty toprojectthe functionontothe subspace W
n 1
. Consider a wavelet
basis
n;k
= 2 n
2
(2 n
x k) which spans the subspace W
n
. Then, the projection of f on
W
n 1 isQ
n 1
f. Since
P
n f =Q
n 1 f +P
n 1 f ,
andthebases
n;k and
n;k
areorthogonal,W
n 1
isreferredtoastheorthogonal complement
of V
n 1 inV
n .
Now we know that W
n
is orthogonal to V
n
, i.e. W
n
? V
n
. Therefore, since W
n
?
(V
n 1
W
n 1
) we can deduce that W
n
is also orthogonalto W
n 1
. Each level of wavelet
subspace isorthogonaltoevery other. Thus
n;k
areorthogonalforall nand all k.
Multiresolution analysis therefore breaks down the original L 2
space into a series of
orthogonalsubspacesatdierentresolutions. Theproblemwenowfaceishowtodesignthe
basisfunctionsand withtherequisiteproperties. Thisistheproblem thatDaubechies
solved andthat we shallnowtackle.
3 INTRODUCTION TO SIGNAL PROCESSING
The design of the wavelet basis functions is most easily understood in terms of signal
processing lter theory. We now indicate some of the key concepts of lter theory. The
interested reader is referred to the excellent book by Gilbert Strang and Truong Nguyen
[11].
Sometexts denethe lterco eÆcientsofthewaveletas( 1) k
a1 k. Equation(13),however,is more
convenient touse whenthereare onlya nitenumb er of lter co eÆcientsa
0 a
N 1
,since itleads to a
waveletthathassupp ortoverthesameinterval,[0;N 1],asthescalingfunction.
Acontinuoustimedependentsignalisrepresentedasx(t)wheretisacontinousindependent
variable.
Adiscrete timesignalisrepresentedasasequenceofnumbersinwhichthenth. number
is denoted by x[n],suchthat,
x=x[n]; 1<n<1 (14)
It is a function whose domain is the set of integers. For example a continuous signal
sampled atintervals ofT can be represented by thesequence x[n],where,
x[n]=x(nT) (15)
3.2 Impulse and Step Function Signal
Two special signals aretheunit impulsedenoted byÆ[n], where,
Æ[n]=f
1 n=0
0 n6=0:
(16)
and the unit step functiondenoted byu[n],where
u[n]=f
0 n<0
1 n0:
(17)
Forlinear systems (see discussion later) an arbitrary sequence can be represented asa
sum of scaleddelayed impulses.
x[n]= 1
X
k = 1
a[k]Æ[n k] (18)
3.3 Discrete Time System
A discrete time system is an operator or transformation T that maps an input sequence
x[n] intoan output sequencey[n], such that,
y[n]=T(x[n]) (19)
3.3.1 Linear systems
A subclass of all such discrete time systems are linear systems dened by the principle of
superposition. If y
k
[n] is theresponse of the system tothe kth input sequence x
k
[n] and
the system islinear, then
T N
X
k =1 x
k [n]
!
= N
X
k =1 T(x
k [n])=
N
X
k =1 y
k
[n] (20)
T(a:x[n])=a:y[n] (21)
Another subclass of all such discrete time systems are time invariant systems. These are
systems forwhich a timeshift of the input x[n n
0
] causes a corresponding time shift of
the outputy[n n
0 ].
An example ofa system which isnot timeinvariant isa system denedby,
y[n]=x[M:n] (22)
In this system the output takes every Mth. input. Suppose we shift the input signal
byM thenthe outputsignal shiftsby1.
y[n+1]=x[M:(n+1)]=x[M:n+M] (23)
3.3.3 Causal systems
Acausalsystem isoneforwhich theoutputy[n
0
]dependsonlyontheinputsequence x[n],
where nn
0 .
3.3.4 Stablesystems
A stable system is called Bounded Input Bounded Output (BIBO) if and only if every
bounded input sequence produces a bounded output sequence. A sequence is bounded if
there exists a xed positive nite valueA, suchthat
jx[n]jA<1 ; 1<n<1 (24)
An example of an unstable systemis given below, where the input sequence u[n] is
the stepfunction.
y[n]= N
X
k =1
u[n k]=f
0 n<0
n+1 n0:
(25)
3.3.5 Linear Time Invariant (LTI) systems
These two subclasseswhencombinedallowanespeciallyconvenient representationofthese
systems. Consider theresponseof anLTI system toasequence of scaledimpulses.
y[n]=T 0
@ 1
X
k = 1
x[k]Æ[n k]
1
A
(26)
Then, using thesuperposition principle we can write
y[n]= 1
X
k = 1
x[k]T(Æ[n k])= 1
X
k = 1 x[k]h
k
[n] (27)
Now here we take h
k
[n] as the response of the system to the impulse Æ[n k]. If the
system is only linear then h
k
[n] depends on both n and k. However, the time invariance
propertyimpliesthatifh[n]istheresponsetotheimpulseÆ[n]thentheresponsetoÆ[n k]
is h[n k]. Thus, we can write theabove as
y[n]= 1
X
k = 1
x[k]h[n k] (28)
−80 −6 −4 −2 0 2 4 6 8 2
4 6
h[k]
−80 −6 −4 −2 0 2 4 6 8
2 4 6
h[−k] = h[0−k]
−80 −6 −4 −2 0 2 4 6 8
2 4 6
h[3−k] : in general h[n−k]
Figure 2. ConvolutionSequencetoformH[n-k]
An LTIsystem iscompletely characterized by its response toan impulse. Once this is
knownthe response ofthesystem toanyinputcan becomputed.
We note thatthis isa convolution and can be writtenas
y[n]=x[n]h[n] (29)
3.3.6 Reminder - convolution
Theconvolutionisformedforsayy[n]bytakingthesumofall theproductsx[k]h[n k]as
kfor 1<k<1. Thesequence x[k]is straightforward. Noticethe sequence h[n k]is
justh[ (k n)]. Thisis obtainedby1)reecting h[k]abouttheorigin togive h[ k]then
shiftingthe origintok=n, asshowninFigure 2.
3.3.7 Linear constant-coeÆcientdierence equations
These systems satisfy the Nth-order linear constant coeÆcient dierence equation of the
form
M
X
k =0 a
k
y[n k]= N
X
k =0 b
k
x[n k] (30)
Thiscan be rearranged intheform
y[n]= N
X
k =1 a
k
y[n k]+ M
X
k =0 b
k
x[n k] (31)
3.3.8 Signal energy
Theenergy ofa signalis given by
E = 1
X
n= 1 jx[n]j
2
= 1
2 Z
jX(e
j!
)j 2
d! (32)
where jX(e )j is the energy density spectrumof thesignal. Equation (32) is well known
asParseval'stheorem.
3.3.9 Convolution theorem
Using the shiftingproperyit can be shown thatif
y[n]= 1
X
k = 1
x[k]h[n k]=x[n]h[n] (33)
thentheFouriertransformoftheoutputis thetermbytermproductoftheFouriertrans-
forms oftheinput and thesignal.
Y(e j!
)=X(e j!
):H(e j!
) (34)
Thisleads toan important propertyof LTI systems. Given aninput x[n]=e j!n
then
the outputis given by
y[n]= X
k
x[k]h[n k]= X
k e
j!k
h[n k]=( X
p h[p]e
j!p
)e j!n
=H(e j!
)e j!n
(35)
where we have changed variables n k!p. We rewrite this as
y[n]=H(e j!
)e j!n
=H(e j!
)x[n] (36)
Thus, forevery LTIsystem thesequenceof complex exponentials isan eigenfunction
with an eigenvalue of H(e j!
).
3.4 The z-Transform
3.4.1 Denition
The z-transform of asequence isa generalization oftheFouriertransform
X(z)= 1
X
k = 1 x[n]z
n
(37)
where z isingenerala complex number z=re j!
.
Writing
X(z)= 1
X
k = 1 (x[n]r
n
)e j!n
(38)
We can interpret the z-transform as the Fourier transform of the product of x[n] and
r n
. Thus when r=1 thez-transform isequivalent totheFouriertransform.
3.4.2 Region of convergence
As withtheFourier transformwe must considerconvergence ofthe serieswhich requires
jX(z)j 1
X
n= 1 jx[n]r
n
j<1: (39)
The values of z forwhich thez-transform converges is called theRegion of Conver-
gence (ROC). The region of convergence is a ring in the z-plane, as shown in Figure 3.
Weshallseewhen discussingthegeneration ofwaveletsthat thepropertiesofLTIsystems
that are both stable and causal provide the constraints that we need to calculate wavelet
coeÆcients.
