The Discrete Duality Finite Volume method in the context of weather prediction models

Thesis

Reference

The Discrete Duality Finite Volume method in the context of weather prediction models

MOODY, Sandie

Abstract

This work originated from a collaboration with the swiss weather prediction service MeteoSwiss which computes its predictions using the COSMO model and uses a terrain-following grid on a non-convex domain. The goal of this thesis is to present a numerical method adapted to the grid to improve the quality of the predictions. The chosen method is the DDFV method, which originated in the late 90s. We first analyse the way turbulent diffusion is treated in the COSMO model, as well in space as in time. We then introduce the DDFV method and prove its convergence on non-convex domains. In order to compensate for the loss of regularity of the solution near the nonsmooth part of the boundary, we introduce a refinement of the grid at the reentrant corner. Finally, we analyse the stability of several time-schemes and compute stability criterions when the DDFV method is used as a space discretisation.

MOODY, Sandie. The Discrete Duality Finite Volume method in the context of weather prediction models . Thèse de doctorat : Univ. Genève, 2019, no. Sc. 5304

DOI : 10.13097/archive-ouverte/unige:114243 URN : urn:nbn:ch:unige-1142432

Available at:

http://archive-ouverte.unige.ch/unige:114243

Disclaimer: layout of this document may differ from the published version.

UNIVERSIT´ E DE GEN` EVE FACULT´ E DES SCIENCES

Section de math´ ematiques Martin J. Gander

The Discrete Duality Finite Volume Method in the Context of Weather Prediction Models

TH` ESE

pr´ esent´ ee ` a la Facult´ e des Sciences de l’Universit´ e de Gen` eve pour obtenir le grade de Doctor ` es Sciences, mention Math´ ematiques

par Sandie Moody

de

Essertines-sur-Yverdon (VD)

Th` ese N

5304

Gen`eve

Atelier d’impression ReproMail 2019

Contents

I Remerciements 3

II R´ esum´ e de la th` ese en fran¸ cais 6

III Introduction 8

III.1 The Equations . . . . 9

III.2 Chapter Summaries . . . . 9

1 Basic Numerical Analysis Notions 11 1.1 The Finite Difference Method . . . . 11

1.2 Convergence and Consistency . . . . 20

1.3 Runge-Kutta Methods . . . . 22

1.3.1 Partitioned Runge-Kutta Methods . . . . 25

1.4 The Finite Volume Method . . . . 25

1.4.1 The Finite Volume Method is not Consistent . . . . 26

1.4.2 Error estimate of FV . . . . 27

1.4.3 Convergence of FV . . . . 28

1.5 Sobolev Spaces . . . . 32

1.6 Weighted Sobolev Spaces . . . . 35

2 A Very Short Introduction to Numerical Weather Prediction 40 2.1 A Historical Introduction to NWP . . . . 40

2.2 Basic Atmospheric and NWP notions . . . . 47

2.3 Tendencies . . . . 50

3 Discretisation and Numerical Integration in the COSMO Model 53 3.1 Time Integration Scheme . . . . 53

3.2 Space Discretisation of Diffusion . . . . 57

3.3 Test Case: Simplified Domain with constant ∆z . . . . 59

3.3.1 Residual for constant metric terms . . . . 60

3.3.2 Numerical Results . . . . 61

3.3.3 Truncation Error . . . . 62

3.4 Test Case: Simplified Domain with variable ∆z . . . . 64

3.4.1 Residual for variable metric terms . . . . 65

3.4.2 Numerical Results . . . . 66

3.4.3 Truncation Error . . . . 67

3.5 Test Case: Domain with Mountain . . . . 69

3.5.1 Test Case: Non-Convex Domain . . . . 69

3.6 Conclusion . . . . 74

4 The Discrete Duality Finite Volume Method 75 4.1 Notations . . . . 75

4.2 Definition and Properties of Discrete Operators . . . . 77

4.3 DDFV for the Poisson Equation . . . . 81

4.4 General Results . . . . 84

4.5 Error Estimates in the discrete H

semi-norm for Convex Ω . . . . 94

4.5.1 Convex Ω and Convex Diamond Cells . . . . 94

4.5.2 Convex Ω and Non-Convex Diamond Cells . . . . 99

4.6 Error Estimate in the discrete H

semi-norm for non-convex Ω . . . 102

4.6.1 Non-convex Ω and convex diamond cells . . . 102

4.6.2 Non-Convex Ω and Non-Convex Diamond Cells . . . 107

4.7 Refinements . . . 109

4.7.1 Attempts to Prove the Second Inequality of Conjecture 4.7.1 . . 118

4.8 Conclusion . . . 121

5 Discrete Duality Finite Volume Method for NWP 123 5.1 Test Case: Simplified Domain with constant ∆z . . . 123

5.1.1 Residual for constant ∆z . . . 123

5.1.2 Numerical Results . . . 124

5.1.3 Truncation Error . . . 124

5.2 Test Case: Simplified Domain with variable ∆z . . . 126

5.2.1 Residual for variable ∆z . . . 126

5.2.2 Numerical Results . . . 126

5.2.3 Truncation Error . . . 127

5.3 Test Case: Domain with Mountain . . . 128

5.3.1 Continuous solution . . . 128

5.3.2 Solution with singularity at the reentrant corner . . . 128

5.4 Coupling of DDFV and FV . . . 133

5.5 Conclusion . . . 135

6 Solving the Heat Equation 136 6.1 Stability Analysis for One-step Methods . . . 136

6.1.1 Numerical Results . . . 142

6.2 Splitting Methods . . . 144

6.2.1 Lie-Trotter splitting . . . 144

6.2.2 Strang splitting . . . 146

6.2.3 Particular θ-splitting . . . 147

6.3 Stability Analysis for Splitting Methods . . . 148

6.4 Conclusion . . . 158

III Conclusion 159

A Code for Section 3.5.1 161

B Code for Section 3.4.1 166

Bibliography 167

Chapter I

Remerciements

Je me d´etourne avec effroi et horreur de cette plaie lamentable des fonctions continues qui n’ont point de d´eriv´ees.

Lettre d’Hermite `a Stieljtes du 20 mai 1893

Cette th`ese n’aurait pas vu le jour sans l’aide et le soutien de nombreuses personnes. Si vous vous sentez concern´e, votre nom devrait figurer dans le mot-crois´e `a la page 5 (mes excuses les plus plates

`

a ceux que j’aurais pu oublier). Quelques personnes m´eritent cependant une mention sp´eciale ! Merci infiniment `a Martin Gander, qui a ´et´e un professeur de th`ese id´eal ! Non seulement sur le plan professionnel, mais aussi dans l’organisation des sorties de groupe au Sal`eve, au Rhˆone et des grillades/trampoline dans ton jardin !

Mille mercis `a Oliver Fuhrer et Philippe Steiner, qui me permirent de m’int´eresser `a l’application de l’analyse num´erique `a la m´et´eorologie sous la forme d’un stage `a M´et´eoSuisse, et qui me soutinrent dans ce projet de th`ese.

Un grand merci `a Florence Hubert et Gilles Vilmart d’avoir accept´e de faire partie du jury. Encore merci Gilles de m’avoir permis de m’int´eresser `a l’int´egration num´erique g´eom´etrique et raide des

´

equations diff´erentielles et surtout d’avoir r´epondu `a mes questions tout au long de ma th`ese.

Je tiens encore `a remercier toutes les personnes de la section de Math´ematiques (coll`egues, ´el`eves, professeurs, ...) qui font que cette communaut´e ressemble plus `a une grande famille qu’`a un institut universitaire! Avec une mention sp´eciale pour Adrien (un des rares `a avoir lu cette th`ese, ce h´eros) et Marco qui sont des co-bureaux exceptionnels !

Plusieurs organisations m’ont aid´ee `a rester (plus ou moins) saine d’esprit pendant ces ann´ees de th`ese : les sports universitaires (Antonio Latella, Markos Michaelides, Robin Fayan, les fr`eres Huwiler, Miguel Valencia, ...), le Erster Z¨urcher Badminton Club, le Badminton Club Rousseau et le Volleyball Club Etoile.

Merci `a mes coll`egues de Drize et de l’IUFE pour leur soutien/aide durant les derniers mois de cette th`ese.

Un ´enorme merci `a Fran¸cois sans qui je n’aurais sans doute pas surv´ecu aux moults derni`eres lignes droites !

Finally, thanks a million to my family for always being supportive and encouraging throughout my never-ending studies.

Across3My friend and only academic sister! 7Ex beau-fr`ere physicien mais que j’adore quand mˆeme! 10Half academic brother, who gets the best Lebanese Loukoums! 14Beach, natation, sauna, glaces, mathscopes, soir´ees, ...

17Meine Spartan Race Freundin! 19Ma petite soeur de filleule pr´ef´er´ee! 20Academic brother from Iran who once came sailing with me! 22L’homme qui nous emm`ene grimper en ´ecoutant Renaud! 24Cousine s’exilant aux quatre coins du monde! 25Meilleur co-bureau de la rue du Li`evre (ex æquo) 26Probabiliste qui m’initia au ski de fond et aux puzzles ! 27Celle qui fit de la Chambre 310 `a Zermatt ”the place to be”! 28Academic step-father, who works with incredibly large matrices! 29Official proofreader of this thesis! 30Compagnon de bad et de mots-fl´ech´es, avec qui j’ai failli mourir noy´ee en Bretagne! 32Celle sans qui je n’aurais pas surv´ecu `a l’UniNe et tellement plus encore! 35Fr`ere acad´emique qui m’aida pour analyse num et maths pour informaticiens! 39Ex-voisin, partenaire de beach et de baignade! 40Sexy matheuse qui m’encouragea `a faire plus de badminton! 42Celle qui m’initia `a la vulgarisation! 43Mon amie depuis le temps des Cerisiers d´ej`a! 45So much more than a sister! 46Violoniste, nageuse et excellente cliente du Mathscope! 48Ma premi`ere amie du J-P, toujours prˆete `a aller `a une disco sur glace!

49Academic brother from India, a symbol in the group! 51Celle sans qui les soir´ees du BC Rousseau seraient bien moins fun! 53Ex office mate, who left us for Stanford! 54L’homme qui lisait un UNION cach´e dans un Picsou dans un TGV! 56Ma filleule pr´ef´er´ee! 59Celui qui pr´ef´era s’exiler `a Durham plutˆot que de faire sa th`ese `a Gen`eve en 7 ans comme tout le monde! 61Half academic brother, thanks to whom the office 610C hosts a lucky cat! 63Ma petite-cousine qui parlait suisse-allemand mieux que moi `a 2 ans `a peine! 66Beau-p`ere acad´emique qui a toujours pris le temps de r´epondre `a mes questions (stupides, souvent)! 69Copine de course `a pied et de soir´ees! 70D’abord patron, puis ami, puis p`ere de ma filleule! 71Le meilleur joueur de badminton que je connaisse! 72G´eniteur de mes neveux et interpr`ete renomm´e de ”Tout nu et tout bronz´e”! 75Excellent d´etecteur de pluie, l’homme le plus blas´e de l’uniNe! 76Co´equipi`ere de rˆeve, 3`eme place aux CG18! 77Sexy matheuse et maintenant mˆeme coll`egue `a Drize! 78 Partenaire de mots-fl´ech´es `a Z¨urich! 80Meilleur colloc de la Kanzlei; plein de surprises! 82La femme de ma vie!

83Ma marraine, qui m’encouragea `a me lancer dans les math´ematiques! 84Mon premier coach de badminton, qui m’incita `a tester tous les sports universitaires! 85El´ement indispensable de la team Mathscope et partenaire de beach!

86Genevois et japonais qui ne se lasse pas de tourmenter mon coll`egue de bureau cˆot´e fenˆetre!

Down 1 Former company which still tolerates me in their running team! Go Miranners! 2 Photographe et conducteur officiel de toutes nos sorties sportives! 3Le fondateur du Mathscope, qui devra encore me supporter dans le cadre de MATh en Jeans! 4Erste Badminton Club, der mich akzeptiert hat! 5Partenaire de badminton, g´enie du Sudoku mais surtout, l’homme qui implanta l’id´ee de faire une th`ese dans ma tˆete! 6SBB Freund und viel mehr! 8 Copine de course qui participe `a toutes les courses les plus extrˆemes! 9La mamma italienne des Sexy matheuses! 11 Ma princesse pr´ef´er´ee! 12Celui qui s´eduisit mon acolyte `a la Bionoce! 13Mon amie itin´erante en voie de devenir qu´eb´ecoise! 15Celui qui me poussa `a me lancer dans cette folle aventure! 16Mon joueur d’Ultimate et rappeur pr´ef´er´e! 18L’homme qui boit son caf´e froid! 21My dear friend from Z¨urich, always ready to go on crazy adventures with me, no questions asked! 23Meilleur club de badminton genevois! 25Ma cousine avec qui j’ai tant ri en parlant d’Euler! 29Ex belle-soeur et amie formidable! 31Ma cousine z¨urichoise pr´ef´er´ee! 33Mon amie et maman de ma filleule! 34Le seul genevois du groupe d’analyse num! 36Celui qui me fit porter un v´elo sur mon ´epaule pendant des heures `a St-John’s! 37Tant de fois il est rentr´e chez lui et m’a trouv´ee en train de squatter ses pantoufles (et sa femme)! 38My favourite nephew (ex æquo)! 41Un des piliers des sexy matheuses, qui perdit `a Z¨urich son Labello!

44Toujours prˆet `a faire un beach, un bad, de la voile et toutes sortes d’autres jeux! 47Meilleur co-bureau de la rue du Li`evre (ex æquo)! 48Une des sexy matheuses que l’Australie faillit nous voler! 50My favourite nephew (ex æquo)!

52Maman adoptive `a Henri-Bordier! 55Ma libraire pr´ef´er´ee! 57Meilleure co-bureau de la rue du Li`evre (ex æquo)!

58Fr`ere acad´emique, source in´epuisable de Fun facts! 60Papa adoptif `a Henri-Bordier! 62Grand chef de APN, gˆace `a qui cette th`ese fut possible! 64Ex-voisine parfaite! Petit-d´ejs dans le jardin, soir´ees Gin, parcours Vita... 65 Biologiste et grimpeur qui m’a soutenue et encourag´ee tout au long du Bachelor! 67Meilleure co-bureau de la rue du Li`evre (ex æquo)! 68Only italian academic brother! 69Genevoise exil´ee `a Neuchˆatel, beacheuse en devenir! 73Mon papa acad´emique qui a rendu cette aventure merveilleuse! 74Coll`egue de M´et´eoSuisse papa des M&Ms! 79Celui qui est `a l’origine de ce projet! 81The one who always believed in me! Simply the best!

12345678 9101112 13141516 171819 20212223 2425 262728 293031 3233 3435 363738 394041424344 454647 48 4950515253 54555657 58596061 626364 65666768 6970717273 747576 77 787980 81 828384 8586

Chapter II

R´ esum´ e de la th` ese en fran¸ cais

Tout d’abord pouss´e par ce qui se fait en aviation, j’ai appliqu´e aux insectes les lois de la r´esistance de l’air, et je suis arriv´e avec M. Sainte-Lagu¨e `a cette conclusion que leur vol est impossible.

Antoine Magnan,Le vol des insectes, 1934

Ce travail trouve ses origines dans le contexte de la m´et´eorologie et plus particuli`erement dans la pr´evision num´erique du temps. Les services m´et´eorologiques basent leurs pr´evisions sur des observa- tions, ainsi que sur des mod`eles num´eriques d´ecrivant les processus se d´eroulant dans l’atmosph`ere.

Le service m´et´eorologique suisse M´et´eoSuisse utilise le mod`ele num´erique COSMO - acronyme de Consortium for Small-Scale Modelling (Consortium pour la mod´elisation `a petite ´echelle) - et fournit des pr´evisions sur l’arc alpin. Les calculs sont effectu´es sur des grilles tridimensionnelles couvrant le domaine consid´er´e. Au cours des derni`eres ann´ees, l’´evolution de la capacit´e de calcul des superordin- ateurs a permis de fortement r´eduire la distance entre les points de la grille. Utiliser une grille plus fine permet d’augmenter la pr´ecision des pr´evisions m´et´eorologiques. Cependant, cela implique que l’importance des processus qui agissent `a une ´echelle inf´erieure `a la maille est grandissante. Le but de cette th`ese est d’analyser l’op´erateur traitant l’un de ces processus, la diffusion turbulente, et de le comparer `a un nouvel op´erateur plus adapt´e `a la topographie complexe pour laquelle M´et´eoSuisse d´elivre des pr´evisions m´et´eorologiques.

La premi`ere partie de cette th`ese est d´edi´ee `a l’analyse du traitement de la diffusion turbulente dans le mod`ele COSMO. On montre que le sch´ema temporel ainsi que le sch´ema spatial sont d’ordre un. N´eanmoins, les exp´eriences num´eriques sur un domaine simplifi´e utilisant une grille ´equidistante montrent que lorsque les fonctions consid´er´ees sont suffisamment lisses, le sch´ema est d’ordre deux.

Lorsque la grille consid´er´ee est plus r´ealiste i.e. lorsque l’espacement vertical est r´eduit aux niveaux bas, on observe des instabilit´es au niveau du sol. Lorsque le domaine consid´er´e est plus r´ealiste, c’est-

`

a-dire que le niveau le plus bas du domaine varie au lieu d’ˆetre constant, le sch´ema n’est pas bien d´efini puisqu’il repose sur une transformation de coordonn´ees qui n’est pas d´erivable aux points o`u la pente n’est pas constante (voir Section 3.5.1).

Dans la deuxi`eme partie de cette th`ese on introduit une m´ethode de volumes finis DDFV (acronyme de Discrete Duality Finite Volume), adapt´ee `a des grilles presque arbitraires. On prouve la convergence de la m´ethode DDFV l´eg`erement modifi´ee sur des domaines non-convexes munis de grilles dont le maillage diamant associ´e comprend des ´el´ements non-convexes. Une impl´ementation de cette m´ethode

´

equivalente `a DDFV sur toutes les cellules diamants convexes serait d´esirable dans des travaux futurs.

On montre que DDFV est plus adapt´e que l’op´erateur actuel de COSMO pour traiter la diffusion, puisqu’on observe une convergence d’ordre deux quel que soit le domaine et la grille associ´ee. De plus, on propose deux raffinements de grilles facilement impl´ementables, qui permettent de recouvrir l’ordre optimal de convergence lorsque la solution du probl`eme consid´er´e poss`ede une singularit´e aux coins rentrants. De tels raffinements ne peuvent cependant pas ˆetre impl´ement´es dans le mod`ele COSMO.

Finalement, on analyse la stabilit´e de diff´erents sch´emas temporels utilisant les deux m´ethodes spatiales mentionn´ees plus haut (les sch´emas DDFV et COSMO). Des crit`eres de stabilit´e sont calcul´es sous certaines hypoth`eses simplificatrices qui reviennent `a supposer que le domaine consid´er´e est un

parall´elogramme sur lequel on d´efinit une grille ´equidistante. On montre que le crit`ere de stabilit´e pour le sch´ema temporel utilis´e dans COSMO est moins restrictif si DDFV est utilis´e comme op´erateur spatial. On montre aussi que si un sch´ema de “Strang splitting” est utilis´e avec DDFV comme sch´ema spatial, le crit`ere de stabilit´e d´epend uniquement du maillage horizontal, alors que l’utilisation du sch´ema spatial impl´ement´e dans COSMO d´epend aussi du maillage vertical et de la pente du domaine.

Chapter III

Introduction

“Begin at the beginning,” the King said very gravely, “and go on till you come to the end: then stop.”

Lewis Caroll,Alice’s Adventures in Wonderland (said by the King to the White Rabbit), 1865 This work originated from a practical question which arose recently in the context of weather forecasting. Weather prediction models attempt to describe the evolution of weather in the future by solving equations which govern the atmosphere. These equations are of chaotic nature, which makes them impossible to solve exactly. The goal of Numerical Weather Prediction (NWP) is to approximate solutions to these equations, by making some simplifying assumptions and using certain mathematical techniques. A weather prediction model looks to simulate the evolution of weather on a specific domain. If the domain is the entire planet Earth, the model is said to be global. Otherwise, it is said to be regional. In both cases, the domain is gridded and the governing equations are approximated for each point of the grid. This entails the use of powerful computers for the solution of the systems of equations. The computational power of supercomputers has increased tremendously in the past decades, which has the effect that NWP models can not only compute predictions further ahead but can also increase their grid resolutions.

The model domain used by MeteoSwiss, the Swiss weather forecasting institution, covers the entire Alpine region with Switzerland at the centre of the domain (see Figure III.0.1). From 2008 to 2016, MeteoSwiss ran a forecasting model called COSMO-2 every 3 hours which used grid boxes of size 2.2 kilometers (which contained 10’920’000 grid points). In 2016, MeteoSwiss launched its new high resolution forecast model COSMO-1, the resolution of which is 1.1 kilometres. This improvement in resolution allows more subtle phenomena, such as thunderstorms or Foehn, to be taken into account and produce more accurate forecasts. However, reducing the grid size requires developing new sub-grid scale turbulence models. Finally, we come to the question which motivated this work: can we design a more robust turbulent diffusion operator?

Figure III.0.1: Region for which MeteoSwiss produces short-range weather forecasts. (Source: ht- tps://www.meteoswiss.admin.ch)

III.1 The Equations

The goal of this thesis was to analyse the properties of the Discrete Duality Finite Volume (DDFV) scheme in the context of the weather prediction model used by MeteoSwiss. MeteoSwiss is part of the Consortium for Small-Scale Modelling (COSMO) and thus uses and conducts research on its model:

the COSMO model. As mentioned earlier, the focus was on sub-grid scale atmospheric turbulent diffusion. The diffusion equation is written as

∂φ

∂t =∇ ·(D(φ)∇φ),

whereD is a diffusion coefficient. IfD is assumed to be constant, this reduces to the heat equation

∂φ

∂t = ∆φ.

In the first part of this thesis, we focus on the spatial dimension of the equation i.e. we consider the two-dimensional Poisson problem

∆φ=f.

We consider two types of boundary conditions for this problem. The first are, for reasons of simplicity, Dirichlet boundary conditions. The second are mixed boundary conditions, as in the COSMO model setup. On the lateral boundaries we impose Dirichlet boundary conditions and at the top and bottom of the domain, Neumann boundary conditions are implemented.

In the second part of this thesis, we consider the heat equation and study different methods to solve it. We consider some one-step methods and some time splitting methods. We conduct von Neumann stability analyses in order to compare the performance of the spatial discretisations considered.

III.2 Chapter Summaries

• Chapters 1 and 2:

The interdisciplinary aspect of this project motivated the two first chapters of this thesis. Both of these chapters are introductory and attempt to follow a historical approach to each subject.

The first chapter retraces the history of numerical analysis (the branch of applied mathematics which develops and analyses techniques to give approximate solutions to complex problems).

The second chapter focuses on weather prediction and the emergence of mathematics, or more precisely numerical analysis, as a tool to forecast the behaviour of the atmosphere.

• Chapter 3:

The third chapter describes the mathematical aspects of the dynamical core of the COSMO model implemented by MeteoSwiss. It gives a brief overview of the time stepping scheme which is implemented and then focuses on the spatial discretisation of the diffusion operator. We consider four test cases. The first consists in solving a Poisson problem on a very simplistic domain with a regular grid. The second is almost the same with one exception: the grid is not as regular. The third test case takes place on a domain which is a vertical cut of a landscape with a mountain in the middle of it, and the grid is close to equidistant. The fourth test case uses the same domain but the grid is again less regular.

• Chapter 4:

The fourth chapter is devoted to the Discrete Duality Finite Volume (DDFV) method. We first give the definition of the DDFV method. We then prove a property which is important in the context of numerical weather prediction: the fact that the method is curl-free. We then give some convergence results for convex domains and extend them to non-convex domains, by using some refinement techniques.

• Chapter 5:

In the fifth chapter, we consider the same test cases as in Chapter 3, using the DDFV method to approximate the diffusion operator instead of the COSMO method, which was analysed in Chapter 3. We also introduce a new method which is a coupling of FV and DDFV and which is less costly than DDFV.

• Chapter 6:

In the last chapter, we focus on time-dependent problems. We first analyse explicit schemes which use the DDFV method and the method implemented in the COSMO model for the space discretisation. We give some convergence results and analyse the stability of the schemes con- sidered when used to solve the heat equation. We then study the time discretisation implemented in the COSMO model again using DDFV and COSMO as space discretisations. Finally, we study time splitting operators and their stability.

Chapter 1

Basic Numerical Analysis Notions

Gravity must be caused by an agent, acting constantly according to certain laws; but whether this agent be material or immaterial, I have left to the consideration of my readers.

Sir Isaac Newton,Letter to Richard Bentley, 1693

In this chapter, we recall some elementary numerical analysis notions. We attempt to give a brief historical context to these notions, in the hope that this chapter might interest even the most fervent numerical analysts. The main sources for the first part of this chapter are “A History of Numerical Analysis” by Goldstine [Gol77], “A Short Account of the History of Mathematics” by Ball [Bal1908]

and “Mathematical Thought from Ancient to Modern Times” by Kline [Kli90]. The second part of this chapter considers notions which were introduced in the twentieth century and is based mostly on [Tho01]. The third part does not concern numerical analysis per se but recalls functional analysis notions that are essential in numerical analysis.

1.1 The Finite Difference Method

Before treating the well known concept of the finite difference method (FDM), we start with a historical introduction of finite differences, partial differential equations and other basic notions needed to introduce the FDM properly. This section is based mainly on [FLT16], [Wal14] and [Str89].

Finite differences seem to have appeared as early as in 1592, in the manuscript Fundamentum Astronomiaepresented by the Swiss mathematician Jost B¨urgi to Emperor Rudolf II. This manu- script was found by German mathematics historian Menso Folkerts in 2013 in the University Library of Wroclaw (Poland), after being thought lost for many years. Its discovery finally enlightened the scientific world as to what Jost B¨urgi’sKunstweg, which is mentioned by B¨urgi and others in many documents of that time, really is. B¨urgi developed a procedure to calculate the sine of an angle to any desired accuracy in a relatively short time. He called this procedure hisKunstweg and created his own table of sines, aCanon Sinuum (see Figure 1.1.1). However, he kept it to himself as well as the method he used to create it, for fear of being published without his permission. Other scientists of that time cite B¨urgi and use his work without ever explaining the method in detail. The Kunstweg uses first and higher order differences of consecutive sine values to derive a simple relation for producing the further sines of minutes of arc. In hisFundamentum Astronomiae, B¨urgi does not mention how he discovered theKunstweg and why it works. A modern proof can be found in [FLT16].

The creation of logarithmic canons led other famous scientists such as Briggs, Napier and Newton to develop finite differences. Newton produced many of our ideas on finite differences, and his notations were very close to the one used nowadays.

Let us now introduce a quite conceptual and controversial notion which seems obvious to most people; the notion of the function. It had to wait until 1734 to obtain its current notation and until the nineteenth century to be defined as it is nowadays accepted. The study of functions started with the analysis of explicit functions such as the trigonometric functions, polynomials and rational functions.

Figure 1.1.1: First page of B¨urgi’sCanon Sinuum(out of 18) in his Fundamentum Astronomiae, 1592.

However, when working on physical problems, mathematicians were obliged to broaden the concept of function. This is illustrated by the non-exhaustive list of the different definitions of the concept of function which were given in the scientific world in the seventeenth and eighteenth century.

(Newton, 1665) If variables can be related in some way, they arefluent.

(Gregory, 1667) Afunctionis a quantity obtained from other quantities.

(Leibniz, 1673) Afunctionis a quantity varying from point to point of a curve.

(Bernouilli, 1697) Afunctionis a quantity formed, by any manner whatever, of vari- ables and of constants.

(Euler, 1748) Afunctionis an analytical expression formed in any manner from a variable quantity “x” and constants. It is denoted by: f(x).

(Euler, 1755) If some quantities depend on others in such a way as to undergo vari- ation when the latter are varied, then the former are calledfunctions of the latter.

(Lagrange, 1797) A function of one or several variables is an expression useful for calculation in which these variables enter in any manner (see Figure 1.1.2).

Definition 1.1.1 Afunctionf from a setX to a setY is defined by a setGof ordered pairs(x, y), such thatx∈X,y∈Y, and every element ofX is the first component of exactly one ordered pair in G.

Let us now recall the definition of finite differences using modern notations. Let x₀ ∈ R be an arbitrary fixed value and let h > 0 be the spacing between two adjacent points. The points to be considered are

xj=x0+jh, j= 0,±1,±2, ...

Definition 1.1.2 The forward differenceassociated with equally spaced points is defined by

∆f(x) =f(x+h)−f(x). (1.1.1)

Higher order differences are defined as

∆ⁿ⁺¹f(x) = ∆ⁿ[∆f(x)] = ∆ⁿf(x+h)−∆ⁿf(x), n= 1,2,3, ... (1.1.2)

Figure 1.1.2: Definition of a function inTh´eorie des fonctions analytiques by Lagrange (1797).

Definition 1.1.3 Thek^th order divided differenceassociated with equally spaced points is defined by

f[x₀] =f(x₀),

f[x0, ..., xn] = f[x0, x1, ..., x_n−1]−f[x1, x2, ..., xn]

x₀−x_n , n= 1,2,3, ... (1.1.3) Theorem 1.1.1 The relation between divided differences with equally spaced arguments and forward differences is given by

∆ⁿf(x₀) =n!hⁿf[x₀, x₁, ..., x_n].

Closely related to differences is the notion of derivatives. Derivatives find their origin in the problem of calculating the instantaneous velocity as a function of the time, knowing the distance traveled. This can be done by calculating the tangent to a curve, a problem on which many mathematicians worked throughout history. Archimedes (287-212 B.C.) created a method to find the tangent at any point on what is called the Archimedean Spiral(see Figure 1.1.3). In 1629, Fermat gave a method to find the tangent to a curve which is relatively close to the one we consider standard nowadays, but he does not take into account the difficult theory of limits. His method relies mostly on properties of similar triangles (see Figure 1.1.4). Descartes also developed his own method, but it was only useful for curves described by a simple polynomial (he was nonetheless convinced that his method was better than Fermat’s). At the same period, scientists were also interested in finding areas, volumes and lengths of curves. Kepler (1571-1630) is said to have been attracted to this problem because he found wine dealer’s methods to calculate the volume of kegs inaccurate. Kepler identifies curvilinear areas and volumes with the sum of an infinite number of infinitesimal elements of the same dimension.

This inspired other scientists, such as Cavalieri (1598-1647) and Roberval (1602-1673), to compute the area under a given curve using rectilinear approximating figures. The relationship between the tangent, the derivative and the integral as a limit of the sum was known by scientists of that time for some specific cases. Cavalieri showed in 1639 in his bookCenturia di varii problemi that, in our

Figure 1.1.3: Archimedean Spiral, from The Archimedes Codex: How a Medieval Prayer Book Is Revealing the True Genius of Antiquity’s Greatest Scientist, 2007.

Figure 1.1.4: Fermat’s tangent method, from Methodus ad Disquirendam Maximam et Minimam, 1629.

y= 1/x a

x0 b

x1 x2 x3 c

x4 d

Figure 1.1.5: Method used by Gregory of St. Vincent to connect the rectangular hyperbola and the logarithm function.

notation, ˆ a

0

xⁿ dx= aⁿ⁺¹

n+ 1 forn∈ {1,2, ...,9}.

In 1647, Gregory of St. Vincent showed in hisOpus Geometricum that the area under a rectangular hyperbola is the same over an interval [x₀, x₁] as over another interval [x₂, x₃] if^x_x⁰

1 =^x_x²

3. In particular (see Figure 1.1.5), he showed that if the curve considered isy= 1/xand thexi are chosen so that the areasa, b, c, d, ...are equal, then theyiwhich are the images of thexiare in geometric progression. So the sum of the areas fromx0toxiis proportional to the logarithm of theyivalues. This allowed him to give the basis to the connection between the rectangular hyperbola and the logarithm function.

This is equivalent to say, in our notation, ˆ xi

x₀

dx

x =klogy_i, wherek∈R.

In 1668, James Gregory proved that the tangent and area problems are inverse problems but his book went unnoticed. At that stage, most of the work was restricted to specific cases. Greater generality of methods was supplied by the achievements of Newton and Leibniz. They saw calculus as a new general method and made it a science capable of handling a wide range of problems, no longer being simply an extension of Greek geometry. One of their main contributions is the reduction to antidifferentiation of area, volume and other problems which used to be treated using summations. They formalised the notions of derivativeand integraland introduced notations that we still use today, as can be seen in the following examples.

Example 1.1.1 (Newton, Methodus Fluxionum et Serierum Infinitarum, written in 1671): A variable quantity is called a “fluent” and denoted by x. The rate of change of xis called the “fluxion” and is denoted byx. Let˙ x andy be two fluents such that y=xⁿ. The relation between their fluxions is given by

˙

y=nxⁿ⁻¹x.˙ Example 1.1.2 (Leibniz, manuscript written in October 1675):

ˆ

x= x² 2 .

Mathematicians of the eighteenth century used calculus to solve more and more physical problems.

Most of those came from astronomy. The subjects of ordinary and partial differential equations arose in the study of those problems. Although some specific differential equations had been solved before, James Bernouilli was one of the first to solve problems of ordinary differential equations analytically.

In 1690, he published the solution of the following problem: “Find a curve along which a pendulum takes the same time to make a complete oscillation whether it swings through a wide or small arc”, which Bernouilli wrote as

dyp

b²y−a³=dx

√ a³.

Figure 1.1.6: Table from Johann Bernoulli’s article in Acta Eruditorium, 1697. Figure I shows the cycloid.

The solution of this first order ordinary differential equation is the cycloid (see Figure 1.1.6). In 1691, James Bernoulli faced a second order ordinary differential equation when studying the shape of a sail under the pressure of the wind. In 1728, Euler also gained interest in higher order equations and later solved the n^th-order nonhomogeneous linear ordinary differential equation.

Definition 1.1.4 An Ordinary Differential Equation (ODE) is a differential equation for a function of one independent variable and its derivatives. An ODE of ordernhas the form

y⁽ⁿ⁾=f(t, y, y⁰, ..., y⁽ⁿ⁻¹⁾).

The ODE is said to beautonomousif it does not depend ont. It is calledlinearif it can be written as

y⁽ⁿ⁾=

n−1

X

i=0

α_iy⁽ⁱ⁾+β(t).

Let us now continue with the concept of partial differentiation. Unsurprisingly, Newton was one of the first to define partial differentiation. In 1736, in his book Method of Fluxions he explains the differentiation of an implicit function in x and y. Another notable mathematician is worth mentioning when it comes to partial differential equations; in the third Volume ofInstitutiones Calculi Integralis published in 1770, Euler gives a uniform approach to partial differential equations. This was one of the first papers devoted to mathematical work on partial differential equations without necessarily being a solution of a physical problem.

Definition 1.1.5 APartial Differential Equation (PDE)is a differential equation that contains unknown multivariable functions and their partial derivatives. Letu(t, x₁, .., x_n)be a function ofn+ 1 independent variables. A PDE is an equation of the form

f(t, x1, ..., xn, u,∂u

∂t, ∂u

∂x₁, ..., ∂u

∂x_n,∂²u

∂t², ∂²u

∂tx₁, ..., ∂²u

∂tx_n,∂²u

∂x²₁, ..., ∂²u

∂x₁x_n, ...) = 0.

APDE of order nonly involves derivatives up to ordernof the unknown function.

An important class of PDEs are linear second-order PDEs. But before going into more detail, let us introduce four equations which all play a fundamental role in the history of mathematics. The first one came from a popular problem during the eighteenth century which consisted in studying the displacement of a vibrating string. This problem was studied separately as a function of time and of distance. Studying this displacement as a function of both variables led to a well known partial differential equation; the wave equation.

Definition 1.1.6 The one-dimensional wave equationis given by

∂²φ(t, x)

∂t² =c∂²φ(t, x)

∂x² .

The first mention of it was by Jean Le Rond d’Alembert in 1746, when he solved the wave equation with initial conditions y(0, x) being the shape of the string at timet= 0 and each particle starts with zero initial velocity i.e. ∂_ty(0, x) = 0. Theboundary conditions

y(t,0) = 0, y(t, l) = 0,

represent the fact that the string is fixed at the endpoints x= 0 and x=l. The general solution to the one-dimensional wave equation is named after him.

Definition 1.1.7 The d’Alembert’s formulais the general solution to the initial-value problem utt−c²uxx= 0, u(x,0) =g(x), ut(x,0) =h(x)

forx∈Randt >0. It is given by

u(x, t) =g(x−ct) +g(x+ct)

2 + 1

2c ˆ x+ct

x−ct

h(ξ)dξ. (1.1.4)

The second and third equations we want to highlight arose during the eighteenth century, when scientists wanted to determine gravitational attraction of one mass on another. An important function is the potential function, usually denoted byV. Its partial derivatives represent the components of the force exerted by a body on a unit mass. The notion of potential was present in Daniel Bernouilli’s work Hydrodynamica published in 1738. For pointsoutside the attracting body, the following differential equation (which is known as the potential equation) holds

∂²V

∂x² +∂²V

∂y² +∂²V

∂z² = 0.

This differential equation appears for the first time in one of Euler’s papers in 1752. Euler writes in this paper that it is not known how to solve this equation. Lagrange reproduces Euler’s work without acknowledgement in a paper published in 1762. Legendre also is interested in the attraction exerted by solids of revolution. He does extensive work on the subject, which greatly inspires Laplace.

Between 1798 and 1825, Laplace published a five-volume treatise on celestial mechanics; his famous Trait´e de m´echanique c´eleste. Without mentioning Legendre’s work, he takes over the problem and gives a solution to the potential equation. This leads to the potential equation also being known as the Laplace equation.

Definition 1.1.8 The Laplace equationis given by

∆φ= 0,

whereφis a twice-differentiable real-valued function and∆is theLaplace operator which is defined as the divergence of the gradient

∆φ=∇ · ∇φ.

Unfortunately, one of Laplace’s assumptions was wrong; he assumed that this equation holds when the point mass being attracted by a body lies inside the body. This error was corrected by Poisson in 1813, when he stated that if a point lies inside the attracting body, then the potential satisfies

∂²V

∂x² +∂²V

∂y² +∂²V

∂z² =−4πρ,

where ρis the density of the attracting body. Although his proof was far from being rigorous, this led to the following equation bearing his name.

Definition 1.1.9 The Poisson equationis given by

∆φ=f, whereφis a twice-differentiable real-valued function.

The fourth equation arose at the beginning of the nineteenth century, when scientists were inter- ested in the flow of heat. This interest was motivated by practical reasons, for instance the handling of metals in industry, but also to determine the temperature in the interior of the earth and how temperature varies with time. Some of those questions were answered by Fourier in 1822, in his book Th´eorie analytique de la chaleur, in which he proves that the temperatureT must satisfy the following partial differential equation

∂²T

∂x² +∂²T

∂y² +∂²T

∂z²

=k²∂T

∂t,

where k is a constant which depends on the material of the body. This is now known as the heat equation in three space dimensions.

Definition 1.1.10 Theheat equation is given by

∂φ

∂t =c∆φ,

whereφis a twice-differentiable real-valued function and c is a real number.

Let us now get back to the classification of second-order PDEs. Laplace and Poisson both made efforts to classify PDEs but the standard classification for linear second-order PDEs was introduced by Du Bois-Reymond in 1889.

Definition 1.1.11 Linear second-order PDEs in two independent variablesxandy have the general form

auxx+buxy+cuyy+dux+euy+f u+g= 0.

They are classified according to the discriminantb²−4ac.

If b²−4ac <0 : Elliptic PDE (time-independent) e.g. uxx= 0.

If b²−4ac >0 : HyperbolicPDE (time-dependent and wavelike) e.g. utt=uxx. If b²−4ac= 0 : ParabolicPDE (time-dependent and diffusive) e.g. ut=uxx.

This classification only made it clearer to the mathematicians of that time that they were missing methods to solve many differential equations. So they started searching for proofs of existence of such solutions. One of the goals also was to understand what initial and boundary conditions ensure a (unique) solution. Cauchy established that sometimes, existence can be proved even when an explicit solution is not available. These questions lead to what is called theDirichlet Problem and which consists in establishing the existence of a solution to the Laplace equation. Dirichlet suggested a general method for solving this class of problems. However, the first proof of the Dirichlet problem in two dimensions under general assumptions about the bounding curve was given in 1870 by H.

A. Schwarz, and the proof of the problem in three dimensions was given in the same year by Carl G. Neumann. Due to the extensive work that Dirichlet and Neumann realized on the existence of solutions to PDEs, two types of boundary conditions are named after them.

Definition 1.1.12 Consider a boundary value problem on a domainΩwhere the boundary condition specifies the values of a solution at the boundary of the domain. This is a Dirichlet boundary condition and is written as

φ(x) =f(x)∀x∈∂Ω, wheref is a given scalar function.

Definition 1.1.13 Consider a boundary value problem on a domainΩwhere the boundary condition specifies the values of the normal derivative of a solution at the boundary of the domain. This is a Neumann boundary condition and is written as

n(x)^T ∇φ(x) =f(x)∀x∈∂Ω, wherenis the normal to the boundary and f is a given scalar function.

A boundary-value problem with boundary conditions which determine a unique solution is said to be well-posed.

We now come to the goal of this chapter; the finite difference scheme. It was first introduced by Runge in 1908 and was rapidly adopted by many other scientists. Suppose you have a well-posed initial-boundary value problem and that you wish to determine the values of the solution. We first discretisethe (t, x) plane i.e. we define a grid of points in the (t, x) plane. Lethand δbe positive numbers, then the grid will be the points (tn, xi) = (nδ, ih) for arbitrary integers n and i. For a functionudefined on the grid, we writeuⁿ_i for the value ofuat the grid point (tn, xi).

The basic idea of finite difference schemes is to replace derivatives by finite differences. One way to do so is to use forward differences

∂u

∂t(nδ, ih)≈ u((n+ 1)δ, ih)−u(nδ, ih)

δ .

We now give a few common finite difference schemes for the one-way wave equation (1.1.6). The forward-time forward-spacescheme for the one-way wave equation

uⁿ⁺¹_i −uⁿ_i

δ =cuⁿ_i+1−uⁿ_i

h . (1.1.5)

Similarly, theforward-time backward spacescheme is given by uⁿ⁺¹_i −uⁿ_i

δ =cuⁿ_i −uⁿ_i−1

h . (1.1.6)

Theforward-time centered-spacescheme is given by uⁿ⁺¹_i −uⁿ_i

δ =cuⁿ_i+1−uⁿ_i−1

2h . (1.1.7)

The last example we give is known as theLeapfrogscheme uⁿ⁺¹_i −uⁿ⁻¹_i

2δ =cuⁿ_i+1−uⁿ_i−1

2h . (1.1.8)

When considering the two-dimensional Laplace equation (1.1.8), it is standard to consider centered differences on half cells to approximate first order derivatives. This leads to the standardfive-point approximationfor the Laplace operator

∆u≈ u_i+1j+u_i−1j+u_ij+1+u_ij−1−4u_ij

h² . (1.1.9)

1.2 Convergence and Consistency

In this section, we recall the concepts of convergence and consistency of numerical schemes. The notion of convergence is present in many other fields of mathematics. However, mathematicians of the eighteenth century resented the need to think about convergence. In 1715, Brook Taylor proved the following formula

f(a+h) =f(a) +f⁰(a)h+f⁰⁰(a)h²

2! +f⁰⁰⁰(a)h³ 3! +...

Although his proof was not rigorous and in spite of the fact that he did not consider the question of convergence, the following theorem bears his name.

Theorem 1.2.1 Taylor’s theoremLet k≥1 be an integer and letf :R−→Rbek times differen- tiable at the point a∈R. Then there exists a functionhk:R−→Rsuch that

f(x) =f(a) +f⁰(a)(x−a) +f⁰⁰(a)

2! (x−a)²+...+f^(k)(a)

k! (x−a)^k+hk(x)(x−a)^k and lim

x→ahk(x) = 0.

Definition 1.2.1 The Taylor seriesof an infinitely differentiable function f is the power series f(a) +f⁰(a)(x−a) +f⁰⁰(a)

2! (x−a)²+f⁰⁰⁰(a)

3! (x−a)³+...

In a letter to Bernouilli in 1743, Euler says that “Any series, divergent or convergent, has a definite sum or value”. By value, he means “the algebraic expression from which the series comes” and gives as an example that the series

1−1 + 1−1 + 1...

has value 1/2. He obtains this value by using the fact that 1

1−x= 1 +x+x²+x³+...

and settingx=−1 leads to

1

2 = 1−1 + 1−1 + 1...

It was not before the nineteenth century that mathematicians insisted on restricting the use of series to convergent ones.

Let us now come back to the question of convergence for numerical schemes. Convergence had been treated before that, but in 1928, Courant, Friedrichs and Lewy published a paper which had a great influence on numerical analysis [CFL28]. In the first part of this paper, they consider the Dirichlet problem for Laplace’s equation (1.1.8) with Dirichlet boundary conditions (Definition 1.1.12),

−∆φ= 0 in Ω, withφ=gon∂Ω. (1.2.1)

They show that the solution u_h to the five-point approximation (1.1.9) converges to a solution uof (1.2.1), which corresponds to the now standard definition of a convergent scheme.

Definition 1.2.2 A numerical scheme approximating a partial differential equation is a convergent scheme if any solution uh to the numerical scheme converges to a solution u(x) of the differential equation when h→0.

It is not easy in general to prove that a scheme is convergent. A related concept is the one of consistency, which is easier to check. A scheme is said to be consistent if the discrete operator converges to the continuous operator. Before giving the formal definition of consistency, we introduce the notions of residual and truncation error.

h

δ

X_δ(x_i, t_n) (xi, tn)

(x_i−t_n^h_δ,0) (x_i+t_n^h_δ,0)

Figure 1.2.1: Domain of dependence of scheme (1.2.3).

Definition 1.2.3 Let S_h be a numerical scheme approximatingSu= 0. Let ube the exact solution to this problem. The residualat the point x_i is computed by introducing the exact solution at that point into the numerical scheme i.e.

r(Sh(ui)) =Sh(u(xi)).

Definition 1.2.4 Let Sh be a numerical scheme approximatingSu= 0. Let ube the exact solution to this problem. Thelocal truncation errorτi of the scheme at the pointxi is given by the leading terms of the Taylor expansion ofSh(u(xi)). Thetruncation error is given by

τ(h) = max

i |τi|.

Definition 1.2.5 A scheme isconsistentif its truncation error tends to zero whenh→0.

Definition 1.2.6 Consider a scheme approximating a PDE. It is said to be oforder pif its truncation error verifies

τ(h) =O(h^p).

Remark 1.2.1 If a scheme has order p, itsresidualcan be written as r(Sh(ui)) =h^pR(Sh, xi) +O(h^p+1) =τi+O(h^p+1).

Let us now focus on time-dependent differential equations. For such problems, an additional notion which is the one of stability comes into play.

Definition 1.2.7 A numerical scheme is stable if for any time t, there exists a constant ct and a constant δ0>0 such that

kuⁿk∆≤c_tku⁰k∆,

for alln≤t/δ and for allh, δ such that0< h, δ≤δ₀. Here, k · k∆ denotes a chosen discrete norm.

Let us come back to Courant, Friedrichs and Lewy’s paper of 1928. In the second part of the paper, they treat the Dirichlet problem of the initial-boundary value problem for the wave equation

utt−uxx= 0 forx∈R², t≥0, withu(x,0) andut(x,0) given. (1.2.2) They use the centered-time centered-space finite difference scheme which is given by

uⁿ⁺¹_i −2uⁿ_i +uⁿ⁻¹_i

δ² =uⁿ_i+1−2uⁿ_i +uⁿ_i−1

h² . (1.2.3)

They show that the discrete solution at (x_i, t_n) only depends on initial data in the interval [x−t/λ, x+t/λ] where λ=δ/h (see Figure 1.2.1) and thus introduce the notion of domain of de- pendence.

Definition 1.2.8 The numerical domain of dependence for a fixed value of δ of the discrete approximation of a partial differential equation, denoted by Xδ(xi, tn) is the set of points xj whose initial datau⁰_i enter into the computation ofuⁿ_i.

The d’Alembert formula (1.1.7) with c = 1 shows that the exact solution at (x, t) only depends on data in [x−t, x+t] which is the domain of dependence of the PDE.

Definition 1.2.9 The mathematical domain of dependence of a PDE at the point (x, t) is de- noted by X(x, t) and is the set of all points in space where the initial data at t = 0 may have some effect on the solutionu(x, t).

Back to our example. We have the numerical domain [x−t/λ, x+t/λ] and the mathematical domain [x−t, x+t]. So ifλ >1, some initial necessary information is not being used and the discrete solution will not converge to the exact solution. Hence the conditionλ <1 is needed. We now give the more general definition of that condition, and the conclusion made by Courant, Friedrichs and Lewy in the form of a theorem.

Definition 1.2.10 The Courant-Friedrichs-Lewy (CFL) condition. For each (x, t), the math- ematical domain of dependence must be contained in the numerical domain of dependence:

X(x, t)⊆Xδ(x, t).

Theorem 1.2.2 The CFL condition is a necessary condition for the convergence of a numerical approximation of a partial differential equation, linear or nonlinear.

A lot of progress was made in the field of numerical analysis during and after the Second World War due to the development of computers. The work of von Neumann, Crank, Nicolson, Lax and others still plays a dominating role. In 1959, Lax and Richtmyer proved the following theorem.

Theorem 1.2.3 (Lax Equivalence Theorem)For a consistent finite difference method for a well- posed linear initial-value problem, the method is convergent if and only if it is stable.

This combined with the CFL theorem gives the last theorem of this section

Theorem 1.2.4 The CFL condition is a necessary condition for the stability of a consistent finite difference method for a well-posed linear initial-value problem.

1.3 Runge-Kutta Methods

Let an initial value problem be specified as follows :

∂u

∂t =f(t, u), u(t0) =u0.

Euler described a method to solve this initial value problem in 1768 in his Institutiones Calculi Integralis (method which is now known as the Euler method, see Definition 1.3.5). In 1895 Carl David Tolm´e Runge constructed similar methods, and so did Karl Heun in 1900. In 1901, Martin Wilhelm Kutta formulated the general scheme of what is now called an explicit Runge- Kutta method.

Definition 1.3.1 Let b_i,a_ij (i, j= 1, ..., s)be real numbers and letc_i be defined by c_i=

i−1

X

j=1

a_ij. The method

k_i=f(t₀+c_iδ, uⁿ+δ

s

X

j=1

a_ijk_j), i= 1, ..., s,

uⁿ⁺¹=uⁿ+δ

s

X

i=1

b_ik_i

(1.3.1)

is called an s-stage Runge-Kutta method. When aij = 0 forj ≥i we have anexplicit method.

An equivalent definition is given by

Yi=uⁿ+δ

s

X

j=1

aijf(Yj), i= 1, ..., s

uⁿ⁺¹=uⁿ+δ

s

X

i=1

bif(Yi).

(1.3.2)

In 1964, John Charles Butcher published a paper in which he symbolised Runge-Kutta methods using a now customary notation.

Definition 1.3.2 The Butcher Tableauof an s-stage Runge-Kutta method is

c A

b^T

whereA= (aij)^s_i,j=1,b= (b1, ..., bs)andc= (c1, ..., cs) are the coefficients of the method.

These Runge-Kutta methods fall in a more general class of methods which are one-step methods:

Definition 1.3.3 A one-stepmethod is a method of the form uⁿ⁺¹=uⁿ+δnΦ(tn, uⁿ, δn).

Definition 1.3.4 A one-step method has order p if for all sufficiently regular problems (4.3.1) the local truncation erroru₁−u(t₀+δ) satisfies

u₁−u(t₀+δ) =O(δ^p+1).

Theorem 1.3.1 Order conditions for Runge-Kutta Methods A Runge-Kutta Method has order 1 if and only if

s

X

i=1

b_i= 1. (1.3.3)

It has order 2 if and only if (1.3.3) holds and

s

X

i,j=1

biaij = 1/2. (1.3.4)

A Runge-Kutta Method is of order 3 if and only if (1.3.3) and (1.3.4) hold and

s

X

i,j,l=1

b_ia_ija_il= 1/3,

s

X

i,j,l=1

b_ia_ija_jl= 1/6.

The number of order conditions for Runge-Kutta methods grows rapidly (see Table 1.3.1) which explains why finding higher order methods is not easy.

order p 1 2 3 4 5 6 7 8 9 10

no. of conditions 1 2 4 8 17 37 85 200 486 1205 Table 1.3.1: Number of order conditions for Runge-Kutta Methods.

We now give some common Runge-Kutta methods and their order.

Definition 1.3.5 The Forward Euler methodis given by uⁿ⁺¹=uⁿ+δf(t_n, uⁿ) and has the Butcher tableau 0 0

1 and has order 1.

Definition 1.3.6 The Backward Euler methodis given by uⁿ⁺¹=uⁿ+δf(t_n+1, uⁿ⁺¹) and has the Butcher tableau 1 1

1 and has order 1.

Definition 1.3.7 “The” Runge-Kutta method (also known as RK4) is given by the Butcher

tableau

0 0 0 0 0

1/2 1/2 0 0 0

1/2 0 1/2 0 0

1 0 0 1 0

1/6 2/6 2/6 1/6

and has order 4.

We now define a Runge-Kutta method which was introduced in 2002 by Wicker and Skamarock ([WLJ02]) and which is now used by most weather forecasting models. In the numerical weather prediction community, this scheme is referred to as the third-order Runge-Kutta method. However, this is only true for linear equations.

Definition 1.3.8 The Runge-Kutta 3 (RK3) method is defined by the Butcher tableau

0 0 0 0

1/3 1/3 0 0

1/2 0 1/2 0

0 0 1

. and has order 2.

Theorem 1.3.2 The RK3 method has order 3 for linear equations.

Proof Order conditions for Runge-Kutta schemes can be derived by computing the Taylor series of the exact solution of the ODE and identifying its coefficients to those of the Taylor series of the numerical solution (see [HNW13] Chapter II.2). The third order term of the Taylor series of the exact solution is given by

h³

6 (f⁰⁰(f, f) +f⁰f⁰f). (1.3.5)

The third order term of the Taylor series of the numerical solution of a Runge-Kutta method is given by

h³ 2





s

X

i,j,l=1

biaijailf⁰⁰(f, f) + 2

s

X

i,j,l=1

biaijajlf⁰f⁰f



. In the case of the RK3 method, the third order term is

h³

8 f⁰⁰(f, f) +h³ 6 f⁰f⁰f.

We see that the second term of equation above is the same as the second term in Equation (1.3.5).

The first terms have different coefficients. However, if the ODE considered is linear we have that f⁰⁰(f, f) = 0. So the third order terms vanish when the numerical solution is subtracted from the exact solution and the method is of order 3.

We now introduce what is known as theθ-scheme.

Definition 1.3.9 The θ-scheme is defined forθ∈[0,1]as

uⁿ⁺¹=uⁿ+δ θf(tn+1, uⁿ⁺¹) + (1−θ)f(tn, uⁿ)

. (1.3.6)

When θ = 0, it is the forward Euler scheme. When θ = 1, it is the backward Euler scheme. Fi- nally, when θ = 1/2, it is the Crank-Nicolson scheme, which is given by the Butcher tableau

0 0 0

1 1/2 1/2 1/2 1/2

and is of order 2.