Numerical tools for the large eddy simulation of incompressible turbulent
flows and application to flows over re-entry capsules
Michel Rasquin
Promoteurs: Th` ese pr´ esent´ ee en vue de
G´ erard Degrez l’obtention du grade acadmique de
Herman Deconinck Docteur en Sciences de l’Ing´ enieur
de l’Universit´ e Libre de Bruxelles
29 avril 2010
Acknowledgements
First of all, I would like to express my deepest gratitude to my supervisor, Professor G´ erard Degrez, for his trust, his constant availability, his guidance and his invaluable advices throughout this work. I take this opportunity to also thank the cafeteria of the University of Brussels for providing the napkins that were used to jot down some formu- lae and sudden ideas G´ erard had during our lunch times. I cannot remember either the number of times I knocked at his door with a problem and left a few minutes later with a solution. He is one of the main reasons why I decided to start this adventure and I would certainly encourage anybody to join his department, not only for his deep under- standing and intuition in fluid mechanics, but also for his personality and human qualities.
The chapter related to the implementation of variational multiscale methods for the large eddy simulation of turbulent flows in SFELES is entirely dedicated to Eve Anger- hausen, who was a brilliant student I had the chance to advise during her final year project in engineering and who left us tragically after starting her PhD thesis in our department.
The constant motivation she showed during her project, despite the health problems she faced, will always be a source of inspiration.
During this PhD, I had the opportunity to spend an eight months research stay at the Rensselaer Polytechnic Institute (RPI). I would really like to acknowledge Professor Kenneth Jansen for his warm welcome in his research group at the Scientific Computation Research Center (SCOREC) and for the time he devoted to me despite his impressive workload. It was a pleasure to join his team. Meanwhile, I have been contaminated by his enthusiasm for computational fluid dynamics in general and for some challenging ap- plications in particular and I am really happy to have the chance to live this experience again a second time soon.
I am also very grateful to Professor Herman Deconinck and David Vanden Abeele from the von Karman Institute (VKI) for their precious advices about the development of our solver library called FlexMG . The first contribution of Todd White (former diploma course student at VKI) for this library was a very valuable step. Fabien Delalondre from RPI is warmly acknowledged for his suggestions about some possible smoothing strategies for the aggregation-type multigrid and we appreciated the review and helpful comments of Professor Yvan Notay from ULB on our article about FlexMG .
I would like to thank Professors Kenneth Jansen (RPI), Daniele Carati (ULB), Herman
Deconinck (VKI), Chris Lacor (Vrije Universiteit Brussel - VUB) and Gregory Coussement
(Universit´ e de Mons-Hainaut - UMONS) for accepting to take place in this PhD thesis jury.
It is a pleasure to thank all the researchers and people I had the opportunity to meet in the Aero-Thermo-Dynamic department at ULB. The increasingly number of people would make the list too long to enumerate but I am sure they will recognize themselves.
I wish everybody to work in such a friendly atmosphere and I will always feel a little bit nostalgic when I think about all the good times and roars of laughter we had during our lunch times or coffee breaks. In addition to their good humor, Yves Detandt, Thomas Cordaro and Axel Coussement are particularly acknowledged for their precious help to understand, optimize and run SFELES on our new cluster. I am also grateful to Shirley Wayne for her efficiency to find a quick solution to any administration problem that could arise.
I would also like to thank all the members of the PHASTA team at RPI for welcoming me in their research group and for guiding my first steps in the PHASTA code. In particular, Onkar Sahni and his help to settle in Troy and run some computations related to the channel flow was really appreciated.
Last but not least, I would like to thank my family and my parents for their constant support and encouragements, not only during this PhD thesis, but also during all my stud- ies. I realize how their presence since the beginning is invaluable in the accomplishment of this work. I am also thankful to Elizabete for sharing my life for the last four years.
When nothing works and when the situation seems desperate, her smile and her presence is always the best remedy to all my worries. I also want to apologize to them for the too long past and upcoming periods of separation due to Atlantic Ocean and thank them for their understanding.
This thesis was primarily supported by a FRIA (‘Fonds pour la formation ` a la Recherche dans l’Industrie et dans l’Agriculture’) fellowship from the French Community of Belgium.
As part of this work, the eight months research stay at RPI was made possible thanks to a traveling fellowship granted by the French Community of Belgium and to a comple- mentary grant from the SCOREC. The Van Buuren Fund and Renard Fund are also both acknowledged for providing their support during the last months of this work.
Developments in SFELES were supported by the Higher Education and Scientific Research Division of the French Community Government at ULB (ARC project 02/07- 283). Computations were performed amongst other on the Linux clusters ANIC4 and ANIC5, funded by several contracts (http://anic4.ulb.ac.be and http://anic5.ulb.ac.be).
In addition to its grant, the SCOREC is also acknowledged for its impressive computer
resources made available.
Contents
List of figures v
List of tables xii
1 Abstract 1
2 Background 5
2.1 SFELES : a Spectral/Finite Element flow solver for the Large Eddy Simu-
lation of turbulent flows . . . . 6
2.1.1 Navier-Stokes equations . . . . 6
2.1.2 Discrete representation . . . . 7
2.1.3 Stabilized finite element discretization . . . . 7
2.1.4 Fourier expansion . . . . 9
2.1.5 Matrix form . . . . 10
2.1.6 Time integration . . . . 11
2.1.7 Boundary conditions . . . . 11
2.1.8 Important features and overview . . . . 12
2.1.9 Parallelization . . . . 13
2.2 PHASTA : a Parallel Hierarchic Adaptive Stabilized Transient Analysis software . . . . 13
2.2.1 Navier-Stokes equations . . . . 15
2.2.2 Stabilized finite element discretization . . . . 15
2.2.3 Parallelization . . . . 16
2.3 Comparison between SFELES and PHASTA . . . . 17
3 Development of FlexMG: a new library of multigrid preconditioners for a spectral/finite element incompressible flow solver 19
3.1 Abstract . . . . 19
3.2 Introduction . . . . 19
3.3 Fixed-point methods . . . . 21
3.3.1 Jacobi iterative method . . . . 22
3.3.2 Gauss-Seidel iterative method . . . . 22
3.3.3 Successive over relaxation iterative method . . . . 22
3.3.4 Incomplete LU factorization . . . . 23
3.3.5 Convergence analysis . . . . 26
3.4 Multigrid preconditioners . . . . 29
3.4.1 Introduction . . . . 29
3.4.2 Piecewise constant aggregation-type multigrid (MG-Agg) . . . . 32
3.4.3 Smooth aggregation-type multigrid . . . . 32
3.4.3.1 Smooth aggregation-type multigrid based on a diffusion equation (MG-AggSmthD) . . . . 35
3.4.3.2 Smooth aggregation-type multigrid based on a constant least-square fitting technique (MG-AggSmthCLSF) . . . . 36
3.4.3.3 Smooth aggregation-type multigrid based on a linear least- square fitting technique (MG-AggSmthLLSF) . . . . 37
3.4.4 Non-nested finite element-type multigrid (MG-FE) . . . . 42
3.5 Krylov methods . . . . 46
3.5.1 GMRES method . . . . 52
3.5.2 Preconditioners for GMRES . . . . 56
3.5.3 Variants of GMRES . . . . 58
3.6 Settings and results . . . . 61
3.6.1 Problem description . . . . 62
3.6.2 Accuracy . . . . 62
3.6.3 Performance . . . . 64
3.6.3.1 Convergence rate . . . . 64
3.6.3.2 Computation time . . . . 69
3.6.3.3 Memory consumption . . . . 72
3.6.4 Robustness . . . . 74
3.7 Conclusions and perspectives . . . . 77
4 Scale-separation and variational multiscale methods for the large eddy simulation of incompressible and turbulent flows 81
4.1 Abstract . . . . 81
4.2 Introduction to turbulent flows . . . . 81
4.2.1 Basic notions and illustrations . . . . 81
4.2.2 Turbulence phenomenology . . . . 84
4.3 The large eddy simulation approach . . . . 89
4.3.1 Classical eddy viscosity models . . . . 89
4.3.2 Dynamic eddy viscosity models . . . . 91
4.3.3 Scale-separation eddy viscosity models . . . . 92
4.3.4 Variational multiscale formulation for eddy viscosity models . . . . 93
4.4 Implementation of LES subgrid scale models in PHASTA . . . . 95
4.4.1 Filtering strategy for a scale-separation model . . . . 95
4.4.2 Projection strategy for a variational multiscale model . . . . 97
4.4.3 Numerical experiments with the turbulent channel flow test case . . 100
4.4.3.1 Subgrid scale models . . . 100
4.4.3.2 Turbulence statistics . . . 103
4.4.4 Conclusions . . . 111
4.5 Implementation of LES subgrid scale models in SFELES . . . 112
4.5.1 Filtering strategy for a scale-separation model . . . 112
4.5.1.1 Top hat filter in Cartesian coordinates . . . 112
4.5.1.2 Top hat filter in cylindrical coordinates . . . 115
4.5.2 Projection strategy for a variational multiscale model . . . 117
4.5.2.1 Implementation of the hierarchical P1h finite element . . . 117
4.5.2.2 Implementation of a variational multiscale LES model based
on the P1h element . . . 122
4.5.2.3 Validation of the P1h/P1h implementation . . . 124
4.5.3 Numerical experiments with the turbulent channel flow test case . . 124
4.5.3.1 Dissipation and kinetic energy . . . 125
4.5.3.2 Subgrid scale models . . . 126
4.5.3.3 Turbulence statistics . . . 127
4.5.4 Conclusions . . . 134
4.6 Conclusions and perspectives . . . 134
5 Large eddy simulations of the
3D turbulent flow over a re-entry capsule
at low speed 1355.1 Abstract . . . 135
5.2 Introduction . . . 135
5.3 Numerical simulations . . . 139
5.3.1 Flow visualizations . . . 142
5.3.2 Spectral properties . . . 150
5.3.3 Aerodynamic coefficients . . . 156
5.4 Conclusions and perspectives . . . 163
6 Conclusions 165 Bibliography 167 A Turbulent statistics for the channel flow test case computed with PHASTA at
Re
τ= 587
175 B Validation of the P1h/P1h implementation 183B.1 Manufactured solution . . . 183
B.2 Lid-driven flow in a square cavity at Re = 100 and Re = 1 000 . . . 188
B.3 Flow past a circular cylinder at Re = 100 . . . 192
B.4 Channel flow at Re
τ= 1 . . . 193
C Implementation of passive scalars in SFELES 195
C.1 Discretization . . . 195
C.2 Parallelization . . . 196
C.3 Illustration . . . 197
List of Figures
2.1 Sequence of operations during one time step for the P1/P1 formulation in SFELES . . . . 12 2.2 Data structure illustrated in both physical and Fourier space for parallel
computations with the Cartesian version of SFELES [1]. . . . . 14 2.3 Parallel procedure with its two global communications for the resolution
of each time step in SFELES [2]. . . . 14 3.1 Typical asymptotic convergence for a fixed-point method determined by
the norm of the eigenvalue of I
−AM
−1closest to one. . . . 27 3.2 Reduction functions P
k(x) for a fixed-point method. . . . 28 3.3 Multilevel correction scheme. . . . . 29 3.4 Aggregation process for the construction of the prolongators in MG-Agg.
Each color zone represents an aggregate of nodes (or aggregate of aggre- gates). . . . . 33 3.5 Aggregation process for the construction of the prolongators in MG-Agg.
Fives fine nodes are clustered into two aggregates (2D example). . . . . . 34 3.6 Piecewise constant and smooth prolongators for the aggregation-type multi-
grid in 1D. . . . 34 3.7 Definition of two sets of vertices,
Giand
Fi, associated with a vertex i in
2D. A vertex is a node if the finest level of the multigrid correction scheme is considered or an aggregate if a coarser level is considered. . . . 36 3.8 Aggregation-type multigrid. Comparison of two smoothing strategies based
on a least-square fitting technique and applied after the transfer of the so- lution from a coarse grid with two aggregates to a fine grid with four nodes (1D example). The first aggregate includes the two left nodes and the sec- ond aggregate the two right ones. In both cases, the solution is smoothed at the two middle nodes by means of fitting functions represented by the dashed lines. . . . 38 3.9 Aggregation-type multigrid. Comparison of two smoothing strategies based
on a least-square fitting technique and applied after the transfer of the so- lution from a coarse grid with four aggregates to a fine grid with 16 nodes (2D example). . . . 41 3.10 Increasingly coarser meshes for the construction of the prolongators in
MG-FE. . . . 42 3.11 Finite element interpolation for the construction of the prolongators in
MG-FE. . . . 43 3.12 Global and natural coordinate systems of an element used as part of the
search algorithm for the host element T of a fine node p in MG-FE. . . . 43
3.13 Determination of the new candidate element hosting node p in MG-FE. . 45 3.14 Way followed by the search algorithm in MG-FE to find the coarse element
(marked with a circle) containing a given fine node (marked with a black point). . . . . 45 3.15 Ways followed by the search algorithm in MG-FE to find the closest coarse
element (marked with a circle) to a given fine node (marked with a black point) that lies outside of the domain. . . . 46 3.16 Degradation of the superlinear convergence of a GMRES method due to
a restart of the algorithm on an illustrative example. . . . 56 3.17 Pressure field and streamlines around a circular cylinder at Re = 100 on
the 80 000 nodes mesh. Visualization of von Karman vortices. . . . 62 3.18 Validation of the coupling between SFELES and FlexMG and verifica-
tion of the accuracy of the solutions on the 20 000 nodes mesh (Re = 100). 63 3.19 Locations around the circular cylinder where some flow variables are mon-
itored, in reference to Figure 3.18. . . . . 64 3.20 Typical convergence curves for GMRES procedures preconditioned by ei-
ther ILUT or a fixed-point method on the 80 000 nodes mesh. . . . 65 3.21 Typical convergence curves for multigrid preconditioners used as stand-
alone solvers on the 80 000 nodes mesh. . . . 67 3.22 Typical convergence curves for different iterative procedures precondi-
tioned by ILUT and multigrid (fixed-point method, stand-alone multigrid cycle and GMRES) on the 80 000 nodes mesh. . . . 68 3.23 Typical convergence curves for the GMRES, FGMRES and GMRES? pro-
cedures preconditioned by ILUT and multigrid on the 80 000 nodes mesh. 69 3.24 Linear system solution time for multigrid preconditioners normalized with
the solution time of the direct sparse LU solver on the 20 000 nodes mesh (normalization factor = 0.22 s). . . . 70 3.25 Linear system solution time for Krylov methods normalized with the so-
lution time of the direct sparse LU solver on the 20 000 nodes mesh (nor- malization factor = 0.22 s). . . . 72 3.26 Memory usage for multigrid preconditioners normalized with the memory
of the direct sparse LU solver on the 20 000 nodes mesh (normalization factor = 766.5 MB). . . . . 73 3.27 Memory usage for Krylov methods normalized with the memory of the
direct sparse LU solver on the 20 000 nodes mesh (normalization factor
= 766.5 MB). . . . 74 4.1 Illustration of turbulent phenomena with their intrinsic wide range of scales. 82 4.2 Energy transfer from the potential range to the dissipative range, through
the inertial range. . . . 86 4.3 Kolmogorov energy spectrum. . . . 87 4.4 Top hat filter. Results for two harmonic functions with a low or high
wavelength on a 2D rectangular uniform structured mesh with 33
×33 nodes in the x and z directions. . . . 96 4.5 Unstructured mesh, dual mesh and agglomeration of some nodal cells of
the dual mesh into a macro-cell [3, 4]. . . . . 97
4.6 Agglomeration-based projector. Results for two harmonic functions with a low or high wavelength on a 2D rectangular uniform structured mesh
with 33
×33 nodes in the x and z directions. . . . . 99
4.7 Sketch of the channel flow test case, with its hexahedral finite element mesh.101 4.8 Eight nodes hexahedral aggregates applied to the 33
×65
×33 nodes mesh and associated with the 3D channel flow test case. Each color zone represents an aggregate of nodes. . . 102
4.9 Re = 392 - Mean eddy viscosity profiles through the channel. . . 104
4.10 Re = 392 - Mean eddy viscosity profiles in the boundary layer. . . . 105
4.11 Re = 392 - Viscous stresses, Reynolds stresses and total stresses through the channel. . . 106
4.12 Re = 392 - Mean velocity profiles through the channel. . . 107
4.13 Re = 392 - Mean velocity profiles in the boundary layer. . . 108
4.14 Re = 392 - u, v and w root mean square velocity fluctuations. . . 110
4.15 2D example of a structured mesh. The squares are finite elements, whereas the circles correspond to aggregates. One finite element in gray out of four shares exactly the same nodes as an aggregate, leading to
∂w∂xi0 j=
∂w∂xi jin the gray regions. . . . 111
4.16 Top hat filter. Results for two harmonic functions with a low or high wavelength on a 2D uniform triangular structured mesh with 33 nodes in the x direction and 64 Fourier modes (32 active) in the z direction. . . 114
4.17 Triangular finite element Ω
ein 2D, with its three vertices v
i. . . . 116
4.18 Three combinations of linear shape functions defined for a macro-mesh and a micro-mesh in 1D. . . 118
4.19 Three combinations of linear shape functions defined for a macro-mesh and a micro-mesh in 2D. . . 119
4.20 Contribution of the micro-nodes to the macro-nodes for the construction of the linear systems in the case of a P1h finite element formulation. . . . 122
4.21 Sequence of operations during one time step for the P1h/P1h formulation in SFELES . . . 123
4.22 Re = 392 - Mean eddy viscosity profiles through the channel. . . 128
4.23 Re = 392 - Mean eddy viscosity profiles in the boundary layer. . . . 129
4.24 Re = 392 - Reynolds stresses through the channel. . . . 130
4.25 Re = 392 - Mean velocity profiles through the channel. . . 131
4.26 Re = 392 - Mean velocity profiles in the boundary layer. . . 132
4.27 Re = 392 - u, v and w root mean square velocity fluctuations. . . 133
5.1 Illustration of atmospheric re-entry capsules. . . 138
5.2 Apollo capsule geometry (α = 180˚) [5]. . . . 139
5.3 2D finite element mesh used for the simulations of the 3D flow around an Apollo-type re-entry capsule with SFELES . . . 140
5.4 3D instantaneous views of turbulent structures in the wake of the capsule with isosurfaces of the Q-criterion at 20 (classical WALE model). . . 144
5.5 3D instantaneous views of turbulent structures in the wake of the capsule with isosurfaces of the Q-criterion at 120 (classical WALE model). . . 145
5.6 2D instantaneous slices of the Q-criterion in the wake of the capsule (clas-
sical WALE model). . . . 146
5.7 2D instantaneous slices of the vorticity field in the wake of the capsule
(classical WALE model). . . 147
5.8 Instantaneous azimuthal velocity fields and their corresponding stream- traces in a 2D plane (classical WALE model). . . 148
5.9 Time and azimuthal average of the pressure fields after ˜ t = 100 for the classical and scale-separation versions of the WALE model, with their corresponding streamtraces. . . 149
5.10 Power spectrum density of physical variables at a point located in the wake of the capsule (classical WALE model). . . . 152
5.11 Power spectrum density of physical variables at a point located in the wake of the capsule (scale-separation WALE model). . . 153
5.12 Power spectrum density of physical variables at a point located in the shear layer near the heat shield of the capsule (classical WALE model). . . 154
5.13 Power spectrum density of physical variables at a point located in the shear layer near the heat shield of the capsule (scale-separation WALE model). . . . 155
5.14 Temporal evolution of the aerodynamic coefficients (force and moment) of the capsule. . . 159
5.15 Global mean (straight horizontal line) and cumulative mean for the aero- dynamic coefficients (force and moment) of the capsule. . . . 160
5.16 Power spectrum density of the aerodynamic coefficients (force and mo- ment) of the capsule (classical WALE model). . . 161
5.17 Power spectrum density of the aerodynamic coefficients (force and mo- ment) of the capsule (scale-separation WALE model). . . . 162
A.1 Re = 587 - Mean eddy viscosity profiles through the channel. . . 176
A.2 Re = 587 - Mean eddy viscosity profiles in the boundary layer. . . . 177
A.3 Re = 587 - Viscous stresses, Reynolds stresses and total stresses through the channel. . . 178
A.4 Re = 587 - Mean velocity profiles through the channel. . . 179
A.5 Re = 587 - Mean velocity profiles in the boundary layer. . . 180
A.6 Re = 587 - u, v and w root mean square velocity fluctuations. . . 181
B.1 Manufactured solution obtained with the P1h/P1h formulation and with the excited mode ˆ k = 2. The mesh includes 64 finite element planes in the z direction and 81
×81 nodes uniformly spaced in each finite element plane. . . . 185
B.2 Validation of the P1h/P1h formulation. Error graphs for a manufactured solution. . . 186
B.3 Validation of the P1h/P1h formulation. Planar error for a manufactured solution in the spectral direction. . . 187
B.4 Validation of the P1h/P1h formulation. Relevant geometry and boundary conditions for the lid-driven cavity. . . 188
B.5 Validation of the P1h/P1h formulation. Lid-driven cavity and related 2D fields at Re = 100. . . 189
B.6 Validation of the P1h/P1h formulation. Lid-driven cavity and related 2D
fields at Re = 1 000. . . . 190
B.7 Validation of the P1h/P1h formulation. Lid-driven cavity at Re = 100.
Comparison of the velocities u and v along the lines x = 0.5 and y = 0.5 respectively with Ghia [6]. . . 191 B.8 Validation of the P1h/P1h formulation. Lid-driven cavity at Re = 1 000.
Comparison of the velocities u and v along the lines x = 0.5 and y = 0.5 respectively with Ghia [6]. . . 191 B.9 Validation of the P1h/P1h formulation with the computation of the lam-
inar flow past a circular cylinder on a 20 000 nodes mesh at Re = 100. . . 192 B.10 Validation of the P1h/P1h formulation with the velocity profile in a peri-
odic channel flow at Re = 1. . . 193 C.1 Parallel procedure for the resolution of the hydrodynamic Navier-Stokes
equations in SFELES . . . 196 C.2 Parallel procedure for the resolution of the hydrodynamic Navier-Stokes
equations coupled to two passive scalars in PASCALSFELES . . . 197 C.3 Axisymmetric flow between two coaxial finite counter-rotating disks en-
closed by a cylinder vessel and illustration of two passive scalars (e.g. two
temperature fields) coupled to this flow. . . 198
List of Tables
2.1 Axis boundary conditions for the axisymmetric version of SFELES . . . . 12 3.1 Number of iterations for various GMRES procedure preconditioned by
ILUT or fixed-point method on the 80 000 nodes mesh. . . . 66 3.2 Number of iterations for various multigrid preconditioners used as stand-
alone solvers on the 80 000 nodes mesh. . . . 66 3.3 Number of iterations for multigrid preconditioners used as stand-alone
solvers or coupled with GMRES on the 80 000 nodes mesh. . . . . 69 3.4 Slopes associated with the graphs of the multigrid preconditioners and the
SuperLU package presented in Figure 3.24 and representing the increase of linear system solution time per thousand of nodes. . . . 71 3.5 Slopes associated with the graphs of the multigrid preconditioners and the
SuperLU package presented in Figure 3.26 and representing the increase of memory consumption per thousand of nodes. . . . 74 3.6 Influence of the time step on the efficiency and the robustness of some
multigrid preconditioners for 1 000 time steps on the 20 000 nodes mesh and with an absolute convergence threshold equal to 10
−12. . . . 75 3.7 Influence of the time step on the efficiency and the robustness of some
multigrid preconditioners for 1 000 time steps on the 20 000 nodes mesh and with an absolute convergence threshold equal to 10
−6. . . . 77 4.1 Typical values of Reynolds number. . . . 85 4.2 Abbreviations for LES models summarizing the scales used to compute
the eddy viscosity (small or all), the strain rate tensor (small or all) and the weight functions (projected or not) of the Galerkin finite element formulation. . . 102 5.1 Characteristic Strouhal numbers at a point located in the far wake of the
capsule and related to the three components of the velocity, the pressure, the vorticity and the Q-criterion. . . 151 5.2 Characteristic Strouhal numbers at a point located in the shear layer and
related to the three components of the velocity, the pressure, the vorticity and the Q-criterion. . . . 151 5.3 Experimental data from [5] for the force and moment coefficients of the
capsule with a 180˚ angle of attack. . . 158 5.4 Numerical results obtained with SFELES for the force and moment co-
efficients of the capsule with a 180˚ angle of attack. ‘STDV’ stands for
‘standard deviation’. . . 158
5.5 Characteristic Strouhal numbers of the force and moment coefficients of the capsule obtained with SFELES for a 180˚ angle of attack. . . 163 A.1 Values of the parameter b for the hyperbolic tangent mapping function
used to cluster the nodes in the y direction near the walls. . . 175 B.1 Validation of the P1h/P1h formulation. Slopes of the error graphs for a
manufactured solution. . . 187
Chapter 1
Abstract
The context of this thesis is the numerical simulation of turbulent flows at moderate Reynolds numbers and the improvement of the capabilities of an in-house 3D unsteady and incompressible flow solver called SFELES to simulate such flows. In addition to this abstract, this thesis includes five other chapters.
The second chapter of this thesis presents the numerical methods implemented in the two CFD solvers used as part of this work, namely SFELES and PHASTA .
The third chapter concentrates on the implementation of a new library called FlexMG . This library allows the use of various types of iterative solvers preconditioned by algebraic multigrid methods, which require much less memory to solve linear systems than a direct sparse LU solver available in SFELES . Multigrid is an iterative procedure that relies on a series of increasingly coarser approximations of the original ‘fine’ problem. The underlying concept is the following: low wavenumber errors on fine grids become high wavenumber errors on coarser levels, which can be effectively removed by applying fixed-point methods on coarser levels.
Two families of algebraic multigrid preconditioners have been implemented in FlexMG , namely smooth aggregation-type and non-nested finite element-type. Unlike pure gridless multigrid, both of these families use the information contained in the initial fine mesh. A hierarchy of coarse meshes is also needed for the non-nested finite element-type multigrid so that our approaches can be considered as hybrid. Our aggregation-type multigrid is smoothed with either a constant or a linear least square fitting function, whereas the non- nested finite element-type multigrid is already smooth by construction. All these multigrid preconditioners are tested as stand-alone solvers or coupled with a GMRES (Generalized Minimal RESidual) method. After analyzing the accuracy of the solutions obtained with our solvers on a typical test case in fluid mechanics (unsteady flow past a circular cylinder at low Reynolds number), their performance in terms of convergence rate, computational speed and memory consumption is compared with the performance of a direct sparse LU solver as a reference. Finally, the importance of using smooth interpolation operators is also underlined in this work.
The fourth chapter is devoted to the study of subgrid scale models for the large eddy
simulation (LES) of turbulent flows. It is well known that turbulence features a cascade
process by which kinetic energy is transferred from the large turbulent scales to the smaller
ones. Below a certain size, the smallest structures are dissipated into heat because of the
effect of the viscous term in the Navier-Stokes equations. In the classical formulation of
LES models, all the resolved scales are used to model the contribution of the unresolved
scales. However, most of the energy exchanges between scales are local, which means that
the energy of the unresolved scales derives mainly from the energy of the small resolved scales.
In this fourth chapter, constant-coefficient-based Smagorinsky and WALE models are considered under different formulations. This includes a classical version of both the Smagorinsky and WALE models and several scale-separation formulations, where the re- solved velocity field is filtered in order to separate the small turbulent scales from the large ones. From this separation of turbulent scales, the strain rate tensor and/or the eddy viscosity of the subgrid scale model is computed from the small resolved scales only.
One important advantage of these scale-separation models is that the dissipation they introduce through their subgrid scale stress tensor is better controlled compared to their classical version, where all the scales are taken into account without any filtering. More precisely, the filtering operator (based on a top hat filter in this work) allows the de- composition u
0= u
−u, where u is the resolved velocity field (large and small resolved scales), u is the filtered velocity field (large resolved scales) and u
0is the small resolved scales field. At last, two variational multiscale (VMS) methods are also considered. The philosophy of the variational multiscale methods differs significantly from the philosophy of the scale-separation models. Concretely, the discrete Navier-Stokes equations have to be projected into two disjoint spaces so that a set of equations characterizes the evolution of the large resolved scales of the flow, whereas another set governs the small resolved scales. Once the Navier-Stokes equations have been projected into these two spaces as- sociated with the large and small scales respectively, the variational multiscale method consists in adding an eddy viscosity model to the small scales equations only, leaving the large scales equations unchanged. This projection is obvious in the case of a full spectral discretization of the Navier-Stokes equations, where the evolution of the large and small scales is governed by the equations associated with the low and high wavenumber modes respectively. This projection is more complex to achieve in the context of a finite element discretization. For that purpose, two variational multiscale concepts are examined in this work. The first projector is based on the construction of aggregates, whereas the second projector relies on the implementation of hierarchical linear basis functions.
In order to gain some experience in the field of LES modeling, some of the above-mentioned models were implemented first in another code called PHASTA and presented along with SFELES in the second chapter. Finally, the relevance of our models is assessed with the large eddy simulation of a fully developed turbulent channel flow at a low Reynolds num- ber under statistical equilibrium. In addition to the analysis of the mean eddy viscosity computed for all our LES models, comparisons in terms of shear stress, root mean square velocity fluctuation and mean velocity are performed with a fully resolved direct numerical simulation as a reference.
The fifth chapter of the thesis focuses on the numerical simulation of the 3D turbulent
flow over a re-entry Apollo-type capsule at low speed with SFELES . The Reynolds number
based on the heat shield is set to Re = 10
4and the angle of attack is set to 180˚, that is the
heat shield facing the free stream. Only the final stage of the flight is considered in this
work, before the splashdown or the landing, so that the incompressibility hypothesis in
SFELES is still valid. Two LES models are considered in this chapter, namely a classical
and a scale-separation version of the WALE model. Although the capsule geometry is
axisymmetric, the flow field in its wake is not and induces unsteady forces and moments
acting on the capsule. The characterization of the phenomena occurring in the wake of
the capsule and the determination of their main frequencies are essential to ensure the
static and dynamic stability during the final stage of the flight.
Visualizations by means of 3D isosurfaces and 2D slices of the Q-criterion and the vorticity field confirm the presence of a large meandering recirculation zone characterized by a low Strouhal number, that is St
≈0.15. Due to the detachment of the flow at the shoulder of the capsule, a resulting annular shear layer appears. This shear layer is then affected by some Kelvin-Helmholtz instabilities and ends up rolling up, leading to the formation of vortex rings characterized by a high frequency. This vortex shedding depends on the Reynolds number so that a Strouhal number St
≈3 is detected at Re = 10
4.
Finally, the analysis of the force and moment coefficients reveals the existence of a lateral force perpendicular to the streamwise direction in the case of the scale-separation WALE model, which suggests that the wake of the capsule may have some preferential orientations during the vortex shedding. In the case of the classical version of the WALE model, no lateral force has been observed so far so that the mean flow is thought to be still axisymmetric after 100 units of non-dimensional physical time.
Finally, the last chapter of this work recalls the main conclusions drawn from the previous chapters.
Some of the contributions presented in this work have been presented in conferences or published in refereed journals, proceedings, or technical reports. This includes, in counter-chronological order,
•
Sahni O, Zhou M, Rasquin M, Shephard M, Jansen K. Adaptive computational fluid dynamics at extreme scale. Technical Report FZJ-JSC-IB-2010-02, J¨ ulich Su- percomputing Centre, Extreme Scaling Workshop 2009.
•
Rasquin M, Deconinck H, Degrez G. FlexMG : A new library of multigrid precondi- tioners for a spectral/finite element incompressible flow solver. International Journal for Numerical Methods in Engineering 2010;
82(12):1510–1536.•
Rasquin M, Degrez G, Deconinck H. FlexMG : A new library of multigrid precon- ditioners for large-scale CFD calculations. Proceedings of the 8th National Congress on Theoretical and Applied Mechanics (NCTAM2009), Brussels, Belgium, 28-29 May 2009.
•
Rasquin M, Jansen K, Degrez G. Scale-separation and variational multiscale meth- ods coupled to a stabilized finite element formulation for large eddy simulation of incompressible and turbulent flows. Proceedings of the 8th National Congress on Theoretical and Applied Mechanics (NCTAM2009), Brussels, Belgium, 28-29 May 2009.
•
Krivilyov M, Rasquin M, Laguerre L, Degrez G, Fransaer J. Modeling of mass trans- fer in (electro)chemical reactors using a hybrid spectral/finite-elements method. Pro- ceedings of the 8th National Congress on Theoretical and Applied Mechanics (NC- TAM2009), Brussels, Belgium, 28-29 May 2009.
•
Rasquin M, Jansen K, Degrez G. Variational multiscale methods coupled to a sta- bilized finite element formulation for large eddy simulation of incompressible and turbulent flows. 15th International Conference on Finite Elements in Flow Prob- lems (FEF09), Tokyo, Japan, 1-3 April 2009.
•
Rasquin M, Vir´ e A, Djoudi Z, Detandt Y, Degrez G. Numerical simulations of un-
steady flows around reentry capsules. Proceedings of the 2nd European Conference
for AeroSpace Sciences (EUCASS), Brussels, Belgium, 1-6 July 2007.
•
Rasquin M, White T, Deconinck H, Degrez G, Vanden Abeele D. Development of an
aggregation/geometric multigrid solver for large-scale CFD calculations. Proceedings
of the 15th Annual Conference of the CFD Society of Canada, Toronto, Canada, 27-
31 May 2007.
Chapter 2
Background
Two CFD solvers have been used as part of this work and are presented in the next three sections. The first one, called SFELES has been developed jointly at Brigham Young University (USA), the von Karman Institute for Fluid Dynamics and the University of Brussels (Belgium) [2, 7–9]. The second one is called PHASTA and is developed at the Rensselaer Polytechnic Institute and the University of Colorado at Boulder (USA) [10–
13]. This work is first dedicated to the improvement of the computational capabilities of SFELES with, on the first hand, the implementation of a new external library of multigrid preconditioners for the resolution of the linear systems arising from the discretization in SFELES and, on the other hand, the development of new turbulence models for the large eddy simulation of turbulent flows. SFELES is then used to simulate the turbulent flow over a re-entry capsule at low speed during the final stage of the flight. The second code taken in hand is called PHASTA and has been used in particular to gain some experience in turbulence modeling and explore some new approaches for the large eddy simulation of turbulent flows. While SFELES and PHASTA are both presented in Sections 2.1 and 2.2 respectively, the main features of these two CFD solvers are compared and outlined in Section 2.3.
The governing Navier-Stokes equations for laminar and incompressible flow, imple- mented in both SFELES and PHASTA , are
( ∇ ·u
= 0,
∂u
∂t + (u
· ∇)u=
−∇p+ ν∇
2u+
f,(2.1) where
uis the 3D velocity field, p is the kinematic pressure (pressure divided by density), ν is the kinematic viscosity and
fis the external force also divided by density.
In Cartesian coordinates, the fully developed form of Equation (2.1) becomes
∂u
x∂x + ∂u
y∂y + ∂u
z∂z = 0,
∂u
x∂t + u
x∂u
x∂x + u
y∂u
x∂y + u
z∂u
x∂z =
−∂p
∂x + ν
∂
2u
x∂x
2+ ∂
2u
x∂y
2+ ∂
2u
x∂z
2+ f
x,
∂u
y∂t + u
x∂u
y∂x + u
y∂u
y∂y + u
z∂u
z∂z =
−∂p
∂y + ν ∂
2u
y∂x
2+ ∂
2u
y∂y
2+ ∂
2u
y∂z
2+ f
y,
∂u
z∂t + u
x∂u
z∂x + u
y∂u
z∂y + u
z∂u
z∂z =
−∂p
∂z + ν ∂
2u
z∂x
2+ ∂
2u
z∂y
2+ ∂
2u
z∂z
2+ f
z.
(2.2)
In cylindrical coordinates, we obtain
∂u
z∂z + 1 r
∂ru
r∂r + 1 r
∂u
θ∂θ = 0,
∂u
z∂t + u
z∂u
z∂z + u
r∂u
z∂r + u
θr
∂u
z∂θ =
−∂p
∂z + ν ∂
2u
z∂z
2+ 1 r
∂
∂r
r ∂u
z∂r
+ 1 r
2∂
2u
z∂θ
2+ f
z,
∂u
r∂t + u
z∂u
r∂z + u
r∂u
r∂r + u
θr
∂u
r∂θ
−u
2θr =
−∂p
∂r + ν ∂
2u
r∂z
2+ ∂
∂r 1
r
∂ru
r∂r
+ 1 r
2∂
2u
r∂θ
2 −2 r
2∂u
θ∂θ
+ f
r,
∂u
θ∂t + u
z∂u
θ∂z + u
r∂u
θ∂r + u
θr
∂u
θ∂θ + u
ru
θr =
−1 r
∂p
∂θ + ν ∂
2u
θ∂z
2+ ∂
∂r 1
r
∂ru
θ∂r
+ 1 r
2∂
2u
θ∂θ
2+ 2 r
2∂u
r∂θ
+ f
θ. (2.3)
2.1 SFELES: a Spectral/Finite Element flow solver for the Large Eddy Simulation of turbulent flows
SFELES is an unsteady and incompressible Navier-Stokes flow solver based upon a combi- nation of triangular finite element in-plane discretization and a spectral discretization (i.e.
a truncated Fourier series) in the transverse direction if working in Cartesian coordinates or in the azimuthal direction if working in cylindrical coordinates. Therefore, this code is designed to simulate 3D flows associated with planar or axisymmetric geometries, which are more frequently encountered in practical engineering problems. The main asset of this code is its ability to exploit the existence of a direction of periodicity in the geometry to boost the computational efficiency, as described below.
2.1.1 Navier-Stokes equations
Since the discretization method is different for the in-plane and the periodic direction, it is convenient to reformulate Equations (2.2) and (2.3) and introduce, on the first hand, an in- plane ˜
∇operator, and on the other hand, an in-plane velocity defined by ˜
u= u
xex+ u
yeyfor the Cartesian coordinates and ˜
u= u
zez+ u
rerfor the cylindrical coordinates. Con- sequently, the laminar and incompressible Navier-Stokes equations for planar geometries and without forcing term become
∇ ·
˜
u˜ + ∂u
z∂z = 0,
∂
u˜
∂t + (˜
u·∇)˜˜
u+ u
z∂˜
u∂z =
−∇p˜ + ν( ˜
∇2u˜ + ∂
2u˜
∂z
2),
∂u
z∂t + (˜
u·∇)u˜
z+ u
z∂u
z∂z =
−∂p
∂z + ν ( ˜
∇2u
z+ ∂
2u
z∂z
2).
(2.4)
For axisymmetric geometries, we obtain
∇ ·
˜
u˜ + 1 r
∂u
θ∂θ = 0,
∂u
z∂t + (˜
u·∇)u˜
z+ u
θr
∂u
z∂θ =
−∂p
∂z + ν
∇
˜
2u
z+ 1 r
2∂
2u
z∂θ
2,
∂u
r∂t + (˜
u·∇)u˜
r+ u
θr
∂u
r∂θ
−u
2θr =
−∂p
∂r + ν
∇
˜
2u
r+ 1 r
2∂
2u
r∂θ
2 −2 r
2∂u
θ∂θ
,
∂u
θ∂t + (˜
u·∇)u˜
θ+ u
θr
∂u
θ∂θ + u
ru
θr =
−1
r
∂p
∂θ + ν
∇
˜
2u
θ+ 1 r
2∂
2u
θ∂θ
2+ 2 r
2∂u
r∂θ
.
(2.5)
2.1.2 Discrete representation
The continuous representation of any flow field variable λ is then given by
λ
h(x, y, z, t) = 1 N
mNm/2
X
k=−Nm/2+1
Nn
X
j=0
Λ
bkj(t)ψ
j(x, y)
| {z }
Λbk(x,y,t)
exp(2πIk z L )
(2.6)
or
λ
h(z, r, θ, t) = 1 N
mNm/2
X
k=−Nm/2+1
Nn
X
j=0
Λ
bkj(t)ψ
j(z, r)
| {z }
Λbk(z,r,t)
exp(Ikθ)
(2.7)
for Cartesian and cylindrical coordinates respectively. In Equations (2.6) and (2.7), N
mis the number of Fourier modes in the discrete Fourier series, Λ
bkj(t) is the k-th mode of λ at node j, N
nis the number of nodes in each finite element plane, ψ
j(x, y) or ψ
j(z, r) stands for the P1 piecewise linear basis function associated with node j, L is the span of the domain in the periodic direction (L = 2π in cylindrical coordinates) and I
2=
−1.2.1.3 Stabilized finite element discretization
Equations (2.4) and (2.5) are then first discretized using a stabilized finite element vari- ational formulation. In Cartesian coordinates and
∀w, q∈H
1, this formulation leads to the following weak form:
0 =
ZΩ
˜
w·∂
u˜
∂t + (˜
u·∇)˜˜
u+ u
z∂
u˜
∂z
−∇ ·
˜
w˜ p + ν
∇
˜
w :˜
∇˜˜
u−w˜
·∂
2u˜
∂z
2+ w
z∂u
z∂t + (˜
u·∇)u˜
z+ u
z∂u
z∂z + p
+ ν
∇w
˜
z·∇u˜
z−w
z∂
2u
z∂z
2+ q
∇ ·
˜
u˜ + ∂u
z∂z
dΩ +
Z
Γn
h
˜
w·(pI
−ν
∇˜˜
u)·n−
νw
z∇u˜
z·nidΓ + ST. (2.8)
ST in Equation (2.8) stands for the stabilization terms of the finite element formulation, which can be written as
ST =
Xe
Z
Ωe
h
τ
supg(˜
u·∇) ˜˜
w+ τ
pspg∇q˜
·
˜
r+ τ
supg(˜
u·∇w˜
z)r
zi
dΩ, (2.9) with the definition of the residuals
˜
r= ∂˜
u∂t + (˜
u·∇)˜˜
u+ u
z∂
u˜
∂z + ˜
∇p−ν
∇
˜
2u˜ + ∂
2u˜
∂z
2, r
z= ∂u
z∂t + (˜
u·∇)u˜
z+ u
z∂u
z∂z + ∂p
∂z
−ν
∇
˜
2u
z+ ∂
2u
z∂z
2.
(2.10)
In cylindrical coordinates, the variational formulation yields,
∀w, q∈H
1, 0 =
Z
Ω
˜
w·∂
u˜
∂t + (˜
u·∇)˜˜
u+ u
θr
∂u
z∂θ
ez+ ∂u
r∂θ
er
−∇ ·
˜
w˜ p + ν
∇
˜
w :˜
∇˜˜
u−w˜ r
2 ·∂
2u
z∂θ
2 ez+ ∂
2u
r∂θ
2 er−
w
rr
u
2θ−ν
r
2 ∂u
θ∂θ + ∂
2u
r∂θ
2+ w
θ∂u
θ∂t + (˜
u·∇)u˜
θ+ u
θr
∂u
θ∂θ + u
ru
θr + 1
r
∂p
∂θ
+ ν
∇w
˜
θ·∇u˜
θ−w
θr
2∂
2u
θ∂θ
2+ 2 ∂u
r∂θ
−u
θ
+ q
∇ ·
˜
u˜ + 1 r
∂u
θ∂θ
dΩ +
Z
Γn
h
˜
w·(pI
−ν
∇˜˜
u)·n−
νw
θ∇u˜
θ·nidΓ + ST, (2.11)
with
ST =
Xe
Z
Ωe
h
τ
supg(˜
u·∇) ˜˜
w+ τ
pspg∇q˜
·
˜
r+ τ
supg(˜
u·∇w˜
θ)r
θidΩ (2.12) and
˜
r= ∂
u˜
∂t + (˜
u·∇)˜˜
u+ u
θr
∂u
z∂θ
ez+ ∂u
r∂θ
er−
u
2θerr + ˜
∇p−ν
∇
˜
2u˜ + 1 r
2∂
2u
z∂θ
2 ez+ ∂
2u
r∂θ
2 er
−
2 r
2∂u
θ∂θ
er
, r
θ= ∂u
θ∂t + (˜
u·∇)u˜
θ+ u
θr
∂u
θ∂θ + u
ru
θr + 1
r
∂p
∂θ
−ν
∇
˜
2u
θ+ 1 r
2∂
2u
θ∂θ
2+ 2 r
2∂u
r∂θ
. (2.13) Like in most finite element codes, the viscous terms characterized by second order derivatives with respect to the in-plane coordinates are integrated by part in both Equa- tions (2.8) and (2.11). This prevents these terms to become identically equal to zero in the case of linear basis functions. The pressure gradient, which represents a term of the same nature as the viscous terms, is also integrated by part. Different expressions for the Petrov-Galerkin parameters τ
pspgand τ
supgare proposed in the literature. In this work, the following expressions are used [14]:
•
τ
pspg= 1
r1
τ
c2+ 1 τ
t2+ 1
τ
ν2, (2.14)
with
τ
c= h
e2U , τ
t= ∆t
2 , τ
ν= h
2e4ν ,
(2.15)
and
h
e=
r
4Ω
eπ ,
U = a reference velocity;
(2.16)
•
τ
supg= 1
r1
τ
c2+ 1 τ
t2+ 1
τ
v2, (2.17)
with
τ
c= h
e2
kuesupg k, τ
t= ∆t
2 , τ
ν= h
2e4ν ,
(2.18)
and
h
e= 2
kuesupgk3
X
j∈e
uesupg· ∇wj
,
uesupg
= 1 3
3
X
j∈e
uj
.
(2.19)
The parameter h
ein the PSPG stabilization term is the hydraulic diameter of the element, whereas h
ein the SUPG stabilization term is the maximum length of the considered element according to its average velocity direction.
2.1.4 Fourier expansion
The full space discretization is obtained by taking the discrete Fourier transform of the semi-discretized expressions in Equations (2.8) and (2.11). For planar geometries in Carte- sian coordinates, we obtain
0 =
ZΩ
"
˜
w·∂
Uˆ ˜
k∂t
−∇ ·˜
w˜ P ˆ
k+ ν
∇˜
w :˜
∇˜
Uˆ ˜
k+ 2πk
L
2˜
w·Uˆ ˜
k!
+ w
z∂ U ˆ
zk∂t + 2πIk L P ˆ
k!
+ ν
∇w˜
z·∇˜ U ˆ
zk+ 2πk
L
2w
zU ˆ
zk!
+ q
∇ ·
˜
Uˆ ˜
k+ 2πIk L
U ˆ
zkdΩ +
hˆ ˜
k+ ˆ h
kz+
Z
Γn
h
˜
w·( ˆ P
kI
−ν
∇˜
Uˆ ˜
k)
·n−
νw
z∇˜ U ˆ
zk·nidΓ + ˆ ST
ksupg+ ˆ ST
kpspg,c+
Xe
Z
Ωe
τ
pspg∇q˜
·"
∂
Uˆ ˜
k∂t + ˜
∇P ˆ
k+ ν 2πk
L
2ˆ ˜
Uk#
dΩ,
∀w,˜ w
z, q
∈H
1,
∀k.(2.20)
The non-linear convective terms, the SUPG stabilization terms and the contribution of
the convective terms to the PSPG stabilization are treated by means of a pseudo-spectral
approach in Equation (2.20), that is
ˆ ˜
hk
= DFT
R
Ωw
˜
·(˜
u·∇)˜˜
u+ u
z∂
u˜
∂z
dΩ
, ˆ h
kz= DFT
R
Ω
w
z
(˜
u·∇)u˜
z+ u
z∂u
z∂z )
dΩ
, ST ˆ
ksupg= DFT
"
X
e
τ
supg ZΩe
(˜
u·∇) ˜˜
w·˜
r+ ˜
u·∇w˜
zr
zdΩ
#
, ST ˆ
kpspg,c= DFT
"
X
e
τ
pspg ZΩe
∇q
˜
·(˜
u·∇)˜˜
u+ u
z∂˜
u∂z
dΩ
#
.
(2.21)
Similarly to Equations (2.20) and (2.21), the full space discretization for axisymmetric geometries can be easily derived from Equations (2.11) to (2.13).
This pseudo-spectral treatment implies that the non-linear terms are computed in physical space and transfered back to Fourier space using a Fast Fourier Transform. To avoid aliasing errors in the pseudo-spectral approach, the
23dealiasing rule is applied. That is, if N
mmodes are used for the continuous representation of a variable, only N
am=
23N
mactive modes at most are kept during the transformation from spectral to physical space, the other modes being set to zero. Therefore, the non-linear terms are evaluated in physical space at N
mlocations in the periodic direction and then transformed back to Fourier space, where the upper
13N
mmodes are thrown out.
2.1.5 Matrix form Taking
( ˜
w, wz, q) = (ψ
iex, 0, 0), (ψ
iey, 0, 0), (0, ψ
i, 0), and (0, 0, ψ
i) (2.22) successively in Equations (2.20) and (2.21), the space-discretized equations in Cartesian coordinates can finally be cast in a matrix form, that is
M
ijd ˆ Φ
kjdt + L
kijΦ ˆ
kj=
−Cikφ
−Nm 2 +1 j
, . . . , φ
Nm 2
j
, (2.23)
where
•
Φ ˆ
kjis the eight components vector in Fourier space, which contains the k-th mode for the real and imaginary parts of the velocity vector and the pressure at node j;
•
