The value of hydrological information in multireservoir systems operation

The value of hydrological information in

multireservoir system operation

Thèse

Jasson Piña Fulano

Doctorat en Génie des Eaux Philosophiæ doctor (Ph. D.)

Québec, Canada

The value of hydrological information in

multireservoir system operation

Thèse

Jasson Piña Fulano

Sous la direction de:

Amaury Tilmant, directeur de recherche Pascal Côté, codirecteur de recherche

Résumé

La gestion optimale d’un système hydroélectrique composé de plusieurs réservoirs est un problème multi-étapes complexe de prise de décision impliquant, entre autres, (i) un compromis entre les consé-quences immédiates et futures d’une décision, (ii) des risques et des incertitudes importantes, et (iii) de multiple objectifs et contraintes opérationnelles. Elle est souvent formulée comme un problème d’optimisation, mais il n’existe pas, à ce jour, de technique de référence même si la programmation dynamique (DP) a été souvent utilisée. La formulation stochastique de DP (SDP) permet la prise en compte explicite de l’incertitude entourant les apports hydrologiques futurs. Différentes approches ont été développées pour incorporer des informations hydrologiques et climatiques autres que les ap-ports. Ces études ont révélé un potentiel d’amélioration des politiques de gestion proposées par les formulations SDP. Cependant, ces formulations sont applicables aux systèmes de petites tailles en raison de la célèbre « malédiction de la dimensionnalité ».

La programmation dynamique stochastique duale (SDDP) est une extension de SDP développée dans les années 90. Elle est l’une des rares solutions algorithmiques utilisées pour déterminer les politiques de gestion des systèmes hydroélectriques de grande taille. Dans SDDP, l’incertitude hydrologique est capturée à l’aide d’un modèle autorégressif avec corrélation spatiale des résidus. Ce modèle analytique permet d’obtenir certains des paramètres nécessaires à l’implémentation de la technique d’optimisation. En pratique, les apports hydrologiques peuvent être influencés par d’autres variables observables, telles que l’équivalent de neige en eau et / ou la température de la surface des océans. La prise en compte de ces variables, appelées variables exogènes, permet de mieux décrire les processus hydrologiques et donc d’améliorer les politiques de gestion des réservoirs. L’objectif principal de ce doctorat est d’évaluer la valeur économique des politiques de gestion proposées par SDDP et ce pour diverses informations hydro-climatiques.

En partant d’un modèle SDDP dans lequel la modélisation hydrologique est limitée aux processus Makoviens, la première activité de recherche a consisté à augmenter l’ordre du modèle autorégressif et à adapter la formulation SDDP. La seconde activité fut dédiée à l’incorporation de différentes variables hydrologiques exogènes dans l’algorithme SDDP. Le système hydroélectrique de Rio Tinto (RT) situé dans le bassin du fleuve Saguenay-Lac-Saint-Jean fut utilisé comme cas d’étude. Étant donné que ce système n’est pas capable de produire la totalité de l’énergie demandée par les fonderies pour assurer pleinement la production d’aluminium, le modèle SDDP a été modifié de manière à considérer les

décisions de gestion des contrats avec Hydro Québec. Le résultat final est un système d’aide à la décision pour la gestion d’un large portefeuille d’actifs physiques et financiers en utilisant diverses informations hydro-climatiques. Les résultats globaux révèlent les gains de production d’énergie auxquels les opérateurs peuvent s’attendre lorsque d’autres variables hydrologiques sont incluses dans le vecteur des variables d’état de SDDP.

Abstract

The optimal operation of a multireservoir hydroelectric system is a complex, multistage, stochastic decision-making problem involving, among others, (i) a trade-off between immediate and future con-sequences of a decision, (ii) considerable risks and uncertainties, and (iii) multiple objectives and operational constraints. The reservoir operation problem is often formulated as an optimization prob-lem but not a single optimization approach/algorithm exists. Dynamic programming (DP) has been the most popular optimization technique applied to solve the optimization problem. The stochastic formulation of DP (SDP) can be performed by explicitly considering streamflow uncertainty in the DP recursive equation. Different approaches to incorporate more hydrologic and climatic information have been developed and have revealed the potential to enhance SDP- derived policies. However, all these techniques are limited to small-scale systems due to the so-called curse of dimensionality. Stochastic Dual Dynamic Programming (SDDP), an extension of the traditional SDP developed in the 90ies, is one of the few algorithmic solutions used to determine the operating policies of large-scale hydropower systems. In SDDP the hydrologic uncertainty is captured through a multi-site periodic au-toregressive model. This analytical linear model is required to derive some of the parameters needed to implement the optimization technique. In practice, reservoir inflows can be affected by other observ-able variobserv-ables, such snow water equivalent and/or sea surface temperature. These variobserv-ables, called exogenous variables, can better describe the hydrologic processes, and therefore enhance reservoir operating policies. The main objective of this PhD is to assess the economic value of SDDP-derived operating policies in large-scale water systems using various hydro-climatic information.

The first task focuses on the incorporation of the multi-lag autocorrelation of the hydrologic variables in the SDDP algorithm. Afterwards, the second task is devoted to the incorporation of different exoge-nous hydrologic variables. The hydroelectric system of Rio Tinto (RT) located in the Saguenay-Lac-Saint-Jean River Basin is used as case study. Since, RT’s hydropower system is not able to produce the entire amount of energy demanded at the smelters to fully assure the aluminum production, a portfolio of energy contacts with Hydro-Québec is available. Eventually, we end up with a decision support system for the management of a large portfolio of physical and financial assets using various hydro-climatic information. The overall results reveal the extent of the gains in energy production that the operators can expect as more hydrologic variables are included in the state-space vector.

Contents

Résumé iii

Contents vi

List of Tables vii

List of Figures viii

Acknowledgments ix Preface x Introduction 1 0.1 Objectives . . . 2 0.2 Specific objectives . . . 2 0.3 Outline . . . 3 1 Literature Review 4 1.1 The reservoir operation problem . . . 4

1.2 The main solution strategies . . . 6

2 Methods 12 2.1 Outline . . . 12

2.2 Optimization problem. . . 12

2.3 Linear Programming . . . 13

2.4 Stochastic Dynamic Programming . . . 14

2.5 Hydrologic information and large scale systems . . . 15

2.6 Stochastic Dual Dynamic Programming . . . 17

2.7 Climatological information as exogenous variable . . . 25

2.8 Hydropower scheduling and contract management . . . 25

2.9 SDDPX formulation . . . 27

3 Case study 31 3.1 Outline . . . 31

3.2 The Gatineau River Basin and hydropower system. . . 31

3.3 The Saguenay-Lac-Saint Jean River Basin and Rio Tinto system . . . 33

4 Overview of Results 38 4.1 Outline . . . 38

4.2 Incorporation of multi-lag autocorrelation . . . 38

4.3 Incorporation of exogenous variables . . . 43

4.4 Joint optimization of physical and financial assets . . . 45

Conclusion 50

Future work 52

Bibliography 53

A Paper I: Horizontal Approach to asses the Impact of Climate Change on Water

Resources Systems 60

B Paper II:Optimizing multireservoir system operating policies using exogenous

hy-drologic variables 72

C Autoregressive modeling 88

List of Tables

3.1 Gatineau hydropower system . . . 32

3.2 Rio Tinto hydro-power system characteristics . . . 35

4.1 Extrapolation techniques . . . 42

4.2 SDDP formulations . . . 44

4.3 Average annual results - Differences with respect to the SDDP(1) model . . . 45

4.4 Average annual results - Differences with respect to the SDDP(1) model . . . 47

List of Figures

1.1 Illustration of reservoir system optimization as sequential decision process. Modified

from Labadie (2004) . . . 4

1.2 Decision tree in reservoir operation problem (adapted from Pereira et al. (1998)) . . 5

1.3 Reservoir optimization classification (Ahmad et al., 2014). . . 5

1.4 SDP principle when maximizing the sum of immediate and future benefits functions 6 1.5 Future benefit function - FBF . . . 7

1.6 Dynamic programing Optimization procedure. Adapted from Labadie (2004) . . . . 9

1.7 SDP and SDDP principles . . . 10

1.8 SDDP and exogenous variables principles . . . 11

2.1 Multistage decision problem scheme . . . 13

2.2 Piecewise linear approximation of FBF function Ft+1 . . . 17

2.3 Example reservoir system configuration and connectivity matrix (adapted from Labadie (2004)) . . . 19

2.4 Normalized Convex-Hull approximation at Passes-Dangereuses power station- Rio Tinto system, Quebec, Canada . . . 20

2.5 Backwards openings and the approximation of the FBF (adapted from Tilmant and Kelman(2007)) . . . 24

2.6 The ARX model structure (adapted from Ljung (1999)) . . . 25

2.7 Schematization of energy trade . . . 26

2.8 Schematic SDDPX toolbox . . . 30

3.1 Gatineau River Basin and hydropower system . . . 32

3.2 Weekly average (µ) and standard deviation (σ ) of inflow series - Gatineau River Basin 33 3.3 Rio Tinto hydropower system. . . 34

3.4 Weekly average and standard deviation of inflow series - Saguenay-Lac-St-Jean River Basin . . . 35

3.5 Schematic of minimum and maximum weekly levels at Lac-Saint Jean . . . 36

3.6 Weekly average and standard deviation of Snow Water Equivalent and precipitation -Saguenay-Lac-St-Jean River Basin . . . 37

4.1 Draw-down refill cycle Baskatong and Cabonga Reservoirs. Lag-1 SDDP (Left panel) and multi-lag SDDP(p) (right panel) . . . 39

4.2 (a) Statistical distribution of annual spillage losses (b) spillage deviation respect SDDP(1) (c) Statistical distribution of annual energy production (d) energy production deviation with respect SDDP(1) . . . 40

4.4 Annual energy generation (a) cumulative distribution functions (b) relative differences

between the distribution functions . . . 42

4.5 Baskatong drawdown-refill cycle - Climate change and current conditions . . . 43

4.6 Accumulated drawdown-refill cycle Passes Dangereuses and Lac-Saint-Jean reservoirs 45 4.7 Statistical distribution of the annual spillage losses (left panel) and the marginal value of water (right panel) . . . 46

4.8 Statistical weekly distribution of energy purchases from the portfolio of contracts . . 48

4.9 % of difference in the power efficiency respect SDDP(1) formulation for both config-uration . . . 48

C.1 Mean and standard deviation . . . 93

C.2 (1) Partial correlogram PACF (2) Mean square error MSE (3) AIC (4) BIC . . . 95

C.3 Cabonga and Baskatong . . . 96

Acknowledgments

I would like to acknowledge various people who have been part of this adventure.

Firstly, I would like to thank my advisor, Amaury Tilmant for guiding and supporting me over the years of my Ph.D study. I thank his time, patience and motivation in all the time of research and writing of this thesis.

I thank also my co-supervisor, Pascal Côté for his kind support, knowledge and generosity.

I would like to thank my thesis committee members, Professors François Anctil, Fabian Bastin and Marcelo Oliveros, for all of their valuable comments.

Special thanks to my friends in these latitudes: Nicolas and his patience with my French; Charles, his chocolates and technical discussions; Thibaut and his patience with my Spanish. I would also like to thank Charles-Hubert for his support and motivation. Thanks to Alex, Hector, Coraline, Bruno, Béné, Sara, Diane, Maria Natalia, who made this experience more enjoyable.

No podría olvidar mi amada familia que desde la distancia fue un apoyo incondicional. Mil palabras de agradecimiento por estar junto a mi en todos mis proyectos: gracias Pa, Ma, Adri y Santi. También a mis amigos que siempre me acompañaron desde Colombia con memes, fotos, videos: Jenny, Moni, Raquel, Sary, Yiyi, Stiwi, Sergio, Oscar, Maria Cristina...

Preface

This PhD thesis presents the research carried out between January 2014 and August 2017 at the De-partment of Civil and Water Engineering (Université Laval-UL). This research took place within the framework of a NSERC-CRD grant with Rio Tinto (RT), under the supervision of Prof. Amaury Tilmant (UL) and co-supervision of Dr. Pascal Côté (RT). The thesis is based on the following publi-cations/presentations:

Paper I J. Pina, A. Tilmant, F. Anctil. Horizontal approach to assess the impact of climate change on water resources systems. Journal of Water Resources, Planning and Management. Published

2016: Pina et al.(2016). http://ascelibrary.org/doi/abs/10.1061/(ASCE)WR.1943-5452.

0000737.

Oral presentation at AGU Fall Meeting 2015: A spatial extrapolation approach to assess the impact

of climate change on water resource systems.https://agu.confex.com/agu/fm15/webprogram/

Paper62891.html

Paper II J. Pina, A. Tilmant, P. Côté. Optimizing multireservoir system operating policies using exogenous hydrologic variables. Journal of Water Resources Research. Published:Pina et al.(2017).

http://onlinelibrary.wiley.com/doi/10.1002/2017WR021701/abstract

Oral Presentation at EWRI-2017: Optimizing multireservoir system operating policies using

exoge-nous hydrologic variables. https://eventscribe.com/2017/ASCE-EWRI/fsPopup.asp?Mode=

presInfo&PresentationID=254515

Oral Presentation at EGU-2017: Valuing physically and financially-induced flexibility in large-scale

water resources systems.http://meetingorganizer.copernicus.org/EGU2017/EGU2017-9687.

Introduction

Reservoirs are essential for domestic and industrial uses, irrigated agriculture, energy production, etc. Even though the reservoir operation problem has been studied for decades, its solution remains challenging. Reservoir operating policies specify the amount of water to be released during a given stage (time period), or the target storage to be reached at the end of that stage. The complexity of the problem lies in the interdependence that exists between the immediate and the future consequences of a release decision. In other words, a balance must be found between outflow, i.e. depleting the reservoir, and keeping the water in storage for future uses. Since the future inflows are uncertain, the problem is essentially stochastic. Furthermore, when hydroelectric systems are analyzed, the problem is nonlinear because the hydropower production function is proportional to the product between the head (storage) and the releases through the turbines.

The examination of the scientific literature reveals that reservoir operation is often formulated as an op-timization problem but that no single opop-timization approach/algorithm exists. Dynamic programming (DP) and its extensions have been extensively used to solve the reservoir operation problem. The basic idea behind DP is to decompose the complex problem in a collection of simpler subproblems which are then solved recursively. The fact that DP can be expanded to account for the hydrologic stochas-ticity is also an interesting feature when dealing with reservoir operation problems. The stochastic extension, called Stochastic DP (SDP), performs an optimization on all discrete combinations of the state variables (storage and hydrologic). Using interpolation techniques, these optimal solutions are generalized to other points of the state-space domain. In an attempt to better describe the hydrologic processes, different approaches and extensions of SDP, such as Sampling SDP (SSDP) (Kelman et al.,

1990) and Bayesian SDP (BSDP) (Karamouz and Vasiliadis,1992) have been developed. However, those improvements quickly hit the wall: as the traditional DP-based solution strategy relies on the discretization of the state-space, the problem becomes quickly intractable due to the so-called curse of dimensionality. Since the computational effort increases exponentially with the number of state vari-ables, the researchers and practitioners were left with a inevitable trade-off between system complexity (the number of individual reservoirs) and hydrologic complexity (the number of hydrologic processes that can be considered). System complexity is desirable to identify synergies between power stations and to avoid the difficulties associated with aggregation/disaggregation techniques. Hydrologic com-plexity, on the other hand, should ultimately yield better release policies by reducing the uncertainty regarding future inflows.

This thesis attempts at removing this trade-off using an alternative technique that is not affected by the curse of dimensionality: Stochastic Dual Dynamic Programming (SDDP). In SDDP the optimal solu-tion is extrapolated to the rest of the state-space domain from a limited number of discrete points that are carefully sampled (whereas traditional techniques rely on a dense grid covering the domain and the solutions are interpolated). SDDP has been used in hydropower-dominated systems such as Nor-way (Rotting and Gjelsvik,1992;Mo et al.,2001;Gjelsvik et al.,2010), South and Central America (Pereira,1989;Homen-de Mello et al.,2011;Shapiro et al.,2013), New Zealand (Kristiansen,2004). The SDDP algorithm also constitutes the core of generic hydro-economic models that have been used to analyze a variety of policy issues in river basins: e.g. Euphrates-Tigris River basin (Tilmant et al.,

2008), the Nile River basin (Goor et al.,2011), the Zambezi River basin (Tilmant and Kinzelbach,

2012) or in Spain (Pereira-Cardenal et al.,2016;Macian-Sorribes et al.,2016).

In the traditional SDDP formulation, the hydrologic uncertainty is captured through a multi-site pe-riodic autoregressive model (MPAR). This model is required to analytically derive the extrapolating functions, and to synthetically generate the different scenarios for the simulation phase of the algo-rithm. Recent works such us Lohmann et al. (2015), Pritchard (2015), Poorsepahy-Samian et al.

(2016), andRaso et al.(2017) reveal a particular interest in improving the built-in hydrologic model. This research work follow this trend but focuses on the incorporation of various hydro-climatic infor-mation into SDDP. To achieve this, additional, exogenous hydrologic state variables must be included in the state vector, and the built-in hydrologic model must be extended to a multi-site autoregressive model with exogenous variables (MPARX). This in turn requires that the analytical formulations of the extrapolating functions be adjusted to accommodate the new exogenous hydrologic variables.

0.1

Objectives

The main objective is to assess the economic value of SDDP-derived operating policies in large-scale water systems for various hydro-climatic information.

0.2

Specific objectives

1. Develop various analytical formulations for the hydrological model embedded in the SDDP algorithm, and the respective formulation to determine the approximation of the benefit-to-go functions.

2. Evaluate the economic performances of the multireservoir system associated with the hydrolog-ical models developed in (1).

3. Assess the impact of alternative hydrologic information on the management of a hydropower portfolio including physical (hydropower plants) and financial assets (contracts) .

To address the challenges posed by these research objectives, three main activities are developed: 1) Incorporation of multi-lag autocorrelation 2) Incorporation of exogenous variables and 3) Joint opti-mization of physical and financial assets.

1. Incorporation of multilag autocorrelation

During this activity, the available Markovian SDDP toolbox (Tilmant et al., 2008), in which the hydrologic uncertainty is captured by a multi-site lag-1 autoregressive model, is modified to take into account multiple lags. The built-in hydrologic multisite periodic autoregressive model of order p (MPAR(p)) is capable of analyzing different hydrologic series, estimating the parameter of the model, selecting the order p of the periodic model, and generating the set of hydrologic information required to implement the SDDP algorithm. The mathematical formulation to couple the higher order MPAR(p) model with the SDDP toolbox is presented in Paper I. Therein, an assessment of climate change scenarios on a large scale hydropower system system using the modified toolbox is presented.

2. Incorporation of exogenous variables

In this activity, a hydrologic model capable of processing various hydrologic and climatic infor-mation is coupled with the optimization model. The built-in model now consists on a MPAR(p) with lag-b exogenous variables MPARX(p, b). The mathematical formulation to derive the ex-trapolating functions based on the MPARX model is presented in Paper II.

3. Joint optimization of physical and financial assets

For this activity, a portfolio of financial assets (i.e. sale/purchase energy contracts) is included in SDDP. A new state variable, accounting for the amount of energy remaining in the contracts, is included in the state-space vector. The mathematical formulation is presented in Paper II. Therein, a joint optimization of physical (reservoir and plants) and financial (energy contracts) assets is presented.

0.3

Outline

The present thesis summarizes the main findings of the two research papers. Chapter 1 presents a literature review on the reservoir operation problem. Chapter 2 describes the solution strategy to optimize release policies for large multireservoir systems. Chapter 3 presents different case study where the various new formulations are applied, and in chapter 4 the main findings are summarized. In chapter 5 can be found the conclusion remarks and in the appendix the research papers.

Chapter 1

Literature Review

1.1

The reservoir operation problem

The operation of a multi-reservoir system is a complex, multistage, stochastic decision-making prob-lem involving, among others, (i) a trade-off between immediate and future consequences of a release decision, (ii) considerable risks and uncertainties and (iii) multiple objectives and operational con-straints (Oliveira and Loucks,1997). The complexity of the problem lies in the interdependence that exists between the immediate and the future consequences of a decision at a given stage. In other words, a balance between storage and release decisions must be found at each stage (Figure 1.1).

Stage t Stage T Inflow1 Stage 1 Release1 Storage1 Benefits1 Storage2 State transition Inflowt Releaset Storaget Benefitst Storaget+1 InflowT ReleaseT StorageT BenefitsT StorageT+1 Allocation decision State Returns

Figure 1.1: Illustration of reservoir system optimization as sequential decision process. Modified from

Labadie(2004)

Since the future inflows are uncertain, the problem is essentially stochastic. At each stage of the decision process, reservoir operators face the future hydrologic uncertainty and the decision made can affect the availability of the resource and thus the future benefits (Figure 1.2).

Operating consequences Future hydro conditions Decision Reservoir Use reservoir Dry Deficit Wet ok Do not use reservoir Dry ok Wet Spillage losses

Figure 1.2: Decision tree in reservoir operation problem (adapted fromPereira et al.(1998))

found in Yeh (1985),Labadie (2004),Rani and Moreira (2009) and more recently inAhmad et al.

(2014). The examination of the scientific literature reveals that reservoir operation is often formulated as an optimization problem but that no single optimization approach/algorithm exists.

Reservoir Operation Linear Programming (LP) Network Flow programming Interior Point Method Non-Linear Programming (NLP) Sequential Linear Programming Sequential Quadratic Programming Method of Multiplier Generalized Reduced Gradient Method Dynamic Program (DP) Deterministic DP Stochastic DP Computational Intelligence (CI)

Fuzzy Set Theory

Artificial Neural Network

Evolutionary computer

Figure 1.3: Reservoir optimization classification (Ahmad et al.,2014)

In the stochastic case, however, two techniques exist: Implicit Stochastic Optimization (ISO) and Explicit Stochastic Optimization (ESO). ISO methods are actually deterministic methods which use a large number of historical or synthetically generated hydrological scenarios to derive optimal

eration policies: they use optimization, regression and simulation techniques to refine promising op-eration rules iteratively. ESO formulations, on the other hand, require the explicit representation of probabilistic streamflows or other uncertain parameters such us energy and water demands, spot prices for energy, etc. (Labadie,2004). To handle the complexity of the reservoir operation problem, a tem-poral decomposition approach is required (Zahraie and Karamouz,2004). This hierarchical approach relies on a chain of optimization models for long, mid and short term planning horizons. The solution strategies presented herein focus on the mid-term hydropower scheduling, which seeks to determine optimal weekly release policies. As we will see in the next section, even though various optimization techniques are available to solve the mid-term reservoir operation problem (Figure 1.3), this research relies on extension of Dynamic Programming.

Optimal decision Future benefits function Immediate benefits function

End-of-period

storage

Immediate + future benefits functions

Be

ne

fits

$

Figure 1.4: SDP principle when maximizing the sum of immediate and future benefits functions

1.2

The main solution strategies

Along with linear programming and nonlinear programming, dynamic programming and its exten-sions have been extensively studied. Dynamic programming (DP), first introduced byBellman(1957), solves the problem by breaking the multistage problem into simpler one-stage subproblems, which are

then solved recursively. With this principle in mind, the objective function of the one-stage optimiza-tion problem becomes the sum of the immediate and future benefits from system operaoptimiza-tion (Figure 1.4). As we can see, the immediate benefits decrease as the end-of-period storage increases as less water is available for immediate uses. At the same time, future benefits increase as more water is kept in storage. The derivatives of the immediate and future benefit functions correspond to the immediate and future marginal water values respectively (Tilmant et al.,2008) and, at the optimal solution, both values are identical. The marginal value of water indicate what the operator would be willing to pay to get an additional unit of water in a particular reservoir and at a given time of the year.

To account for the hydrologic uncertainty, DP can be expanded by adding hydrological variables in the state vector. Consequently, the release decisions are now function of the storage and the hydrologic state variable (e.g. previous or current inflow). This stochastic DP (SDP) formulation, often referred to as a Markov decision process, explicitly considers the streamflow lag-1 correlation found in the flow records; the recursive equation uses the fact that inflow during any given time period is related to the previous one by a conditional probability (Yeh,1985).

End-of-period storage B enefit s $ Stochastic variable

Figure 1.5: Future benefit function - FBF

SDP solves the problem by replacing the continuous domain by a grid, and by solving the one-stage DP optimization problem at each grid point. These optimal solutions are then generalized the rest of the domain using interpolation techniques (Figure 1.5). Since this optimization is performed condi-tionally on all discrete combinations of the state variables, this discrete approach is limited to small scale problems with no more than four state variables. To illustrate the so-called curse of dimen-sionality of a four-dimensional problem, let us imagine that the state variables are discretized in 10 values. Then, the one-stage optimization must be evaluated over a grid of 10 × 10 × 10 × 10 points

(104). Hence, the computational effort increases exponentially with the number of state variables as 10#variables.

Various strategies for dealing with the dimensionality issue associated with DP have been proposed in the literature. For example,Turgeon and Charbonneau(1998),Saad et al.(1996),Archibald et al.

(1997) andArchibald et al.(2006) use aggregation- disaggregation techniques to reduce the scale of the problem.Bellman and Dreyfus(1962) suggest the Dynamic Programming Successive Approxima-tion (DPSA), which decomposes the multidimensional problem into a sequence of one-dimensional problem by optimizing over one state variable at a time. Then releases are explicitly obtained from the mass balance equation as a function of specified beginning and ending storage. Incremental Dynamic Programming (IDP) and Discrete Differential Dynamic Programming (DDDP) address the dimension-ality problem by restricting the state space to a corridor around a current given solution (Karamouz et al., 2003). The methods are highly sensitive to initial storage trajectories and the discretization intervals must be carefully selected to provide accurate solutions at a reasonable computational time (Labadie,2004). Although, these efforts reduce computation time, the curse of dimensionality is not removed.

Along with the curse of dimensionality, the representation of streamflow persistence and hydrologic forecasting information is an important issue when applying SDP: better streamflow foresight is ex-pected to improve reservoir operation because it allows time for better decision making (Georgakakos,

1989). The generation of scenarios and forecast information for water resources management appli-cations relies on the use of different stochastic hydrologic models (Pagano et al.,2004;Gelati et al.,

2010). Various approaches are available to forecast reservoir inflows, from regression relationships be-tween inflows and climate observed data (e.g. snowpack, soil moisture, fall and winter precipitation), to models properly initialized with climate forecast (e.g. Ensemble Streamflow Prediction (ESP) fore-casts, forecast of ENSO, downscaled numerical climate model forefore-casts, etc.)(Anghileri et al.,2016;

Georgakakos,1989). The National Weather Service’s (NWS) ESP procedure (Day,1985) produces streamflow forecast in the form of multiple hydrographs, and the forecast of ENSO are currently available up to a year or more in advance (Gelati et al.,2014).

To exploit the potential value of these forecasts, different extensions of SDP have been developed. For example,Kelman et al.(1990) proposed a sampling SDP (SSDP) that captures the complex temporal and spatial structure of the streamflow process through the use of a large number of sample stream-flows scenarios, instead of assuming that inflow stochasticity in SDP follows a probability density function.Karamouz and Vasiliadis(1992) developed the Bayesian SDP (BSDP) which uses Bayesian decision theory to incorporate new information by updating the transition probabilities; in BSDP nat-ural and forecast uncertainties are both included in the model.

The advantage of using different hydrologic variables in SDP formulations has been presented in sev-eral references. Bras et al.(1983) presented the introduction of real time forecast with an adaptive control technique where flow transition probabilities and system objective are continuously updated.

In flo w s Time

DP

SDP

FBF

SSDP-BSDP

Simulation model

Figure 1.6: Dynamic programing Optimization procedure. Adapted fromLabadie(2004)

Stedinger et al. (1984) developed a SDP model which employs the best inflow forecast of the cur-rent period to define the policy. Georgakakos (1989) discussed the value of streamflow forecasts in reservoir operation. Kim and Palmer (1997) compared the performance of the BSDP formulation and three alternative SDP models, when the seasonal flow forecast and other hydrologic information are included in the state vector. Faber and Stedinger (2001) andKim et al. (2007) employed ESP forecasts and snowmelt volume forecasts using SSDP formulation. Côté et al.(2011), introduced in SSDP a new hydrological state variable given as a linear combination of snow water equivalent and soil moisture. More recently,Desreumaux et al.(2014) presented the effect of using various hydro-logical variables on SDP-derived policies of the Kemano hydropower system in British Columbia.

Anghileri et al.(2016) presented a forecast-based adaptive management framework for water supply reservoirs and evaluate the contribution of long-term inflow forecasts to reservoir operations.

However, most of the studies described above are limited to small-scale problems, meaning that a trade-off must be found between the complexity of system to be studied and the complexity of the hy-drologic processes that can be captured. When analyzing a large-scale system, operators must decide whether to use a simplification of the system (e.g. aggregating storage capacity), or use fewer

hy-drologic state variables to describe the hyhy-drologic process, leading somehow to a loss of information, and to a loss of spatiotemporal synergies that can be captured when analyzing the whole system. This trade-off can largely be removed by using stochastic dual DP (SDDP).

SDDP, first introduced by Pereira and Pinto(1991), is one of the few available algorithms to opti-mize the operating policies of large-scale hydropower systems. The solution approach is based on the approximation of the expected future benefit function (FBF) of SDP by piecewise linear functions (Figure 1.7). With SDDP, there is no need to evaluate the FBF over a dense grid as the function can now be derived from extrapolation (and not interpolation). The accuracy of the approximation is increased by adding new linear segments through a two-phase iterative algorithm. The set of linear segments can be interpreted as Benders cuts in a stochastic multistage decomposition algorithm, and its determination relies on the primal and dual information of the optimal solution of each subproblem. To implement the efficient decomposition scheme, each nonlinear SDP subproblem must be formu-lated as a convex problem, such as a linear program (LP). This constitutes the main drawback of the technique since all the relations associated with the problem, objective function, and constraints, must be linear. End-of-period storage B ene fit s $ Stochastic variable End-of-period storage B ene fit s $ Stochastic variable Sampling Point 2 Sampling Point 1 Piecewise linear approximation True function Hyperplane 1 Hyperplane 2

Figure 1.7: SDP and SDDP principles

SDDP has largely been used in hydropower systems, such as Norway (Rotting and Gjelsvik,1992;

Mo et al.,2001;Gjelsvik et al.,2010), South and Central America (Pereira,1989;Homen-de Mello et al.,2011;Shapiro et al.,2013), New Zealand (Kristiansen,2004) and Turkey (Tilmant and Kelman,

2007). Some improvements to deal with the nonlinear water head effects have been developed by using a convex hull approximation of the hydropower function and can be found inGoor et al.(2011) and Cerisola et al. (2012). The SDDP algorithm constitutes the core of generic hydro-economic models that have been used to analyze a variety of policy issues in the Euphrates-Tigris River basin (Tilmant et al.,2008), the Nile River basin (Goor et al.,2011), the Zambezi River basin (Tilmant and Kinzelbach,2012) or in Spain (Pereira-Cardenal et al.,2016;Macian-Sorribes et al.,2016).

In flo w s Time Hydrologic uncertainty MPAR(p) SDDP FBF approximations H yd ro -cl im a ti c in fo rm a ti o n Time Hydrologic uncertainty MPARX(p,b) SDDPX

Figure 1.8: SDDP and exogenous variables principles

In SDDP, the hydrologic uncertainty is captured through a multi-site periodic autoregressive model. This model is capable of representing seasonality, serial and spatial streamflow correlations within a river basin and among different basins. Furthermore, it is required to analytically derived the ap-proximation of the FBF, and to synthetically generate the different scenarios for the simulation phase. Recent works reveal a particular interest in improving the built-in hydrologic model. For example,

Lohmann et al. (2015) presented a new approach to include spatial information. Pritchard (2015) modeled inflows as a continuous process with a discrete random innovation,Poorsepahy-Samian et al.

(2016) proposed a methodology to estimate the cuts parameters when a Box-Cox transformation is used to normalize inflows, and more recently, Raso et al.(2017) present a streamflow model with a multiplicative stochastic component and a non-uniform time step.

This PhD thesis fits into this trend and seeks to assess the value of different hydro-climatic information when operating large-scale water resource systems. To achieve this, additional exogenous hydrologic state variables are incorporated into the SDDP algorithm. Since the modeling of the hydrologic un-certainty in SDDP is restricted to linear additive models (Infanger and Morton,1996; De Queiroz and Morton,2013), the natural extension to include climatic variability into the autoregressive model is the MPAR model with exogenous variables (MPARX) (Figure 1.8). The reader should refer to

Ljung(1999),Ltkepohl(2007) andHannan and Deistler(2012) for detailed presentations of MPARX models.

Chapter 2

Methods

2.1

Outline

This chapter explains how the exogenous hydrologic variables are incorporated into the SDDP al-gorithm. It starts with overview of the optimization techniques available for solving the reservoir operation problem with a particular attention given to Stochastic Dynamic Programming (SDP). Al-ternative SDP formulations, each employing different hydrologic information, are presented, their strengths and weaknesses discussed. Finally, the Stochastic Dual Dynamic Programming (SDDP) algorithm is described.

2.2

Optimization problem

Recall that the reservoir operation problem is a multistage decision-making problem (Figure 2.1). When it is formulated as an optimization problem, the goal is to determine a sequence of optimal allocation decisions xt (e.g. reservoir release and spillage, water withdrawals, etc) that maximizes the expected sum of benefits from system operation Z, over a planning period time T , while meeting operational and/or institutional constraints. The mathematical formulation of the multistage decision-making problem can be written as:

Z= max    E " T

∑

St+1= ft(St, qt, xt) (2.2) gt+1(St+1) ≤ 0 ∀t (2.3) at+1(xt+1) ≤ 0 ∀t (2.4) Stage t Stage T Stage 1 _State transition Allocation decision State Returns 𝑞𝑡 𝑥_𝑡 𝑞𝑇 𝑥𝑇 𝑞₁ 𝑥₁ 𝑠𝑡 𝑠𝑡+1 𝑠_𝑇 𝑠_𝑇+1 𝑠1 𝑏₁(𝑆₁, 𝑞₁, 𝑟₁) 𝑠2 𝑏_𝑡(𝑆_𝑡, 𝑞_𝑡, 𝑟_𝑡) 𝑏_𝑇(𝑆_𝑇, 𝑞_𝑇, 𝑟_𝑇)

Figure 2.1: Multistage decision problem scheme

where bt(·) is the immediate benefit function, ν(·) is the terminal value function, αt is the discount factor at stage t and E[·] is the expectation operator. In many reservoir operation problems the vector of the state variables St includes the beginning-of-period storage st and any hydrological variable ht. For a hydropower-dominated system of J reservoirs and D demand sites for off-stream uses (e.g. municipal and industrial uses, irrigated agriculture), the immediate benefit function includes the net benefits from hydropower HPt [$], the benefits from off-stream uses NBt [$], and penalties for not meeting target water demands and/or violating operating constraints:

bt(·) = HPt+ NBt− ξt|zt (2.5)

where zt (J × 1) is the vector of slack variables with the violations of operational constraints (e.g. energy deficit, environmental flows, etc.) which are penalized in the objective function by the vector ξt (J × 1) of penalties [$/unit].

2.3

Linear Programming

Linear Programming (LP) is one of the most popular optimization techniques applied in water re-sources management. Its attractiveness lies in ability to handle large scale problems, to converge to a global solution, to allow for a sensitivity analysis from the duality characteristics of linear program-ming, and the availability of generic softwares for solving LP problems. The main disadvantage is

the fact that the LP requires that all the relations associated with the problem, objective function and constraints, to be linear or linearizable. Moreover, if the stochastic formulation of LP is rarely im-plemented because of the computational burden, the two-stage stochastic linear programming with recourse, first introduced independently byDantzig(1955) andBeale(1955), can be used for solving the problem deterministically for each of the the several scenarios of future inflows. This formulation leads to an extremely large-scale linear programming problem, which can be reduced by utilizing Ben-ders decomposition. The basic idea is to express the expected value of the second stage by a scalar and to replace the second-stage constraints sequentially by cuts, which are necessary conditions expressed only in terms of the first stage variables (Infanger,1993). Another decomposition strategy for solving large-scale stochastic programs is progressive hedging (PH) (Rockafellar and Wets,1991) which is a scenario-based decomposition technique. PH has been used as an effective heuristic technique for ob-taining approximate solutions to multistage stochastic programs (Hart et al.,2012;Carpentier et al.,

2013)

2.4

Stochastic Dynamic Programming

Next to linear programming, Dynamic programming (DP) has been the most popular optimization technique applied to water resources planning and management. The method was first introduced by

Bellman (1957) and solves the problem (Equations 2.1 to2.4) by breaking the multistage problem into simpler subproblems over each stage, which are solved recursively. DP can handle non-linear relationships and discontinuous functions. DP performs an optimization on all discrete combinations of the state variables. These optimal solutions are generalized for other points of the state variables by a continuous function, using an interpolation approach (e.g. linear, cubic spline) (Johnson et al.,

1993;Tejada-Guibert et al.,1993;Kitanidis et al.,1987). Stochastic DP (SDP), often referred to a Markov decision process, solves the problem by discretizing stochastic variables, as well as the system status, to obtain an optimal policy for each discrete value of the reservoir system (Rani and Moreira,

2009). If the vector of state variables St includes the beginning-of-period storage st and any choice of hydrologic state variable ht, the recursive SDP equation can be written as (Tejada-Guibert et al.,

1995): Ft(st, ht) = E qt|ht [max xt {αtbt(·) + E ht+1|ht,qt [αt+1Ft+1(st+1, ht+1)]}] (2.6)

where E[·] is the expectation operator to observe hydrological condition ht+1 given the hydrological state ht, and it is obtained from the conditional probabilities P(ht+1| ht). SDP directly incorporates both the probability distributions of random variables and the temporal persistence between successive flows through the use of flow transition probabilities. Since the expectation operator acts on the max-imization, the release decisions are made after the hydrologic variable ht is known (Tejada-Guibert

When the temporal persistence is not modeled, the hydrologic variable is not included in the SDP and the equation2.6is reduced to the deterministic DP equation2.7. The Pseudo-code description of the DP procedure is presented in Algorithm 1.

Ft(st) = max xt

{αtbt(·) + αt+1Ft+1(st+1)} (2.7)

Initialize the FBF of the last stage; FT(sT+1) = 0;

for t = T, T − 1, ..., 1 do

for each storage level st = s1t, ..., s Kn t do solve the one-stage problem Equation2.7

Ft(st) = max xt

{αtbt(·) + αt+1Ft+1(st+1)} s.t operational constraints equations2.2to2.4

end

Create a complete FBF, Ft(st), for the previous stage by interpolating the values n

Ft(sktn), kn= 1, ..., Kn o

end

Algorithme 1 : Pseudo-code description of the DP procedure

The most common choices for hydrological state variable have been the current flow qt and the pre-vious flow qt−1. When the current flow is used the expectation Eqt|ht can be omitted and equation2.6 becomes: Ft(st, qt) = max xt {αtbt(·) + E qt+1|qt [αt+1Ft+1(st+1, qt+1)]} (2.8)

Likewise, if the state hydrological variable is the previous flow equation2.6becomes:

Ft(st, qt−1) = E qt|qt−1

[max xt

{αtbt(·) + αt+1Ft+1(st+1, qt)}] (2.9) and the Pseudo-code description of the SDP procedure is presented in algorithm 2.

2.5

Hydrologic information and large scale systems

Among several concerns, the representation of streamflow persistence and hydrologic forecasting in-formation in the decision process is a critical issue when applying SDP (Labadie,2004;Kelman et al.,

1990). Thus, better hydrologic information, incorporated as state variables, has the potential for en-hancing SDP-derived policies. For example,Bras et al.(1983) presented the introduction of real time forecast with an adaptive control technique where flow transition probabilities and system objective are continuously updated over finite transient periods before achieving steady state conditions. Ste-dinger et al. (1984) developed a SDP model which employs the best inflow forecast of the current

Initialize the FBF of the last stage; FT(sT+1, qT) = 0;

for t = T, T − 1, ..., 1 do

for each storage level st = s1t, ..., s km t , ..., s

Km t do for each past inflow qt−1= q1t−1, ..., q

kk

t−1, ..., , q Kk t−1do

for each inflow scenario of stage t conditioned to the past inflow scenario qt = qt1, ..., q

kj t , ..., q

Kj t do

solve the one-stage problem Equation2.9considering an initial storage skm

t and the inflow scenario qktj Ftkj(s km t , q kk t−1) = max_x t {αtbt(·) + αt+1Ft+1(st+1, q kj t )} s.t operational constraints equations2.2to2.4

end

calculate the expected value of the benefits obtained across the conditioned inflow scenarios Ft(sktm, q kk t−1) = ∑ kj P kj|kk Ftkj(s km t , q kk t−1) ! end

Create a complete FBF, Ft(st, qt−1), for the previous stage by interpolating the values n Ft(sktm, q kk t−1), km= 1, ..., Km; kk= 1, ..., Kk o end end

Algorithme 2 : Pseudo-code description of the SDP procedure

period to define the policy. Kelman et al.(1990) proposed a sampling SDP (SSDP) to better capture the complex temporal and spatial structure of the streamflow process. SSDP uses a large number of streamflows scenarios, instead of assuming that inflow stochasticity in SDP follows a probability density function. SSDP implementation relies on two different models: i) a decision model which chooses an optimal release that maximizes the future and current benefit for each stage, state, and scenario taking into account the streamflow uncertainty, and ii) a simulation model which uses the optimal releases for each scenario to update the future value maintaining a realistic description of streamflow series.

Karamouz and Vasiliadis(1992) developed the Bayesian SDP (BSDP) which includes inflow, storage, and forecast as state variables. BSDP describes streamflows with a discrete lag-1 Markov process, and uses Bayesian decision theory to incorporate new information by updating the transition probabilities. In BSDP, natural and forecast uncertainties are both captured in the model. When the current inflow qt and the seasonal or monthly flow forecast ft+1 are used as hydrologic state variables equation2.6 becomes: Ft(st, qt, ft+1) = E qt|qt, ft+1 [max xt {αtbt(·) + E qt+1ft+2|qt, ft+1,qt [αt+1Ft+1(st+1, qt+1, ft+2)]}] (2.10)

variables ht+1, the conditional expectations in Equation2.10can be determined for four different cases (further details seeKim and Palmer(1997)). Therein, the potential advantage of using the seasonal flow forecast and other hydrologic information is illustrated by comparing the performance of the BSDP formulation and three alternative SDP models.

As mentioned earlier, since the optimization is performed conditionally on all discrete combinations of the state variables, the problem gives rise to the curse of dimensionality: assuming k state variables, discretized into N values the computational effort required to solve equation 2.6increases exponen-tially with the number of reservoirs J as

 NkJ



. Then, if a large-scale system is studied using SDP, a balance must be found between the complexity of the system (e.g number of reservoirs) and the complexity of the hydrologic process that can be captured. Although strategies such as aggregating the storage capacity or using fewer hydrologic variables can be implemented, they can somehow lead to a loss of information, and to a loss of spatiotemporal synergies that can be captured when analyzing the whole system. This trade-off between system and hydrologic complexities can largely be removed by using stochastic dual DP (SDDP).

2.6

Stochastic Dual Dynamic Programming

Stochastic dual DP (SDDP) was first proposed byPereira and Pintoin 1991. SDDP is not affected by the curse of dimensionality and can therefore be used to optimize the operating policies of large-scale hydropower systems. The solution approach is based on the approximation of the FBF functions of SDP by piecewise linear functions (Figure 2.2). There is no need to evaluate Ft+1over a dense grid as the function can now be derived from extrapolation (and not interpolation).

Sampling Point 2 Sampling Point 1 Piecewise linear approximation True function Hyperplane 1 Hyperplane 2

𝑠

𝑞

𝐹

With SDDP the multistage optimization problem (Equations2.1to2.4) can be broken into a series of one-stage linear programming (LP) problems which are solved recursively. At stage t, describing the state of the system with the storage st and using as hydrologic state variable the previous inflow qt−1, the recursive equation can be written as:

Ft(st, qt−1) = max{αtbt(·) + αt+1Ft+1} (2.11)

The problem is bounded by L Bender’s cuts which are represented by the inequality constraints:

           Ft+1− ϕt+1l | st+1≤ γt+1l | qt+ βtl+1 .. . Ft+1− ϕt+1L | st+1≤ γt+1L | qt+ βtL+1 (2.12)

ϕt+1(J × 1), γt+1(J × 1), βt+1are the linear parameters of the approximated FBF Ft+1. Likewise, the stage to stage transformation function (i.e. the mass balance equation) is expressed as:

st+1−CMR(rt+ lt) + et(st, st+1) = st+ qt (2.13) where st+1 is the vector (J × 1) of storage at the end of the period, rt is the vector (J × 1) of the turbined flows, lt and et are the vectors (J × 1) of spillage and evaporation losses respectively, CMR is the reservoir system connectivity matrix, CMR_j,k= 1(-1) when reservoir j receives (releases) water from (to) reservoir k. Figure 2.3 displays an example of a reservoir system configuration and the connectivity matrix.

The linear segments Ft+1 are obtained from the dual solutions of the optimization problem at each stage and can be interpreted as Benders cuts in a stochastic, multistage decomposition algorithm. SDDP uses an iterative optimization/simulation strategy to increase the accuracy of the solution by adding new cuts. To implement the decomposition scheme, the one-stage optimization problem must be formulated as a convex problem, such as a linear program. Nonetheless, the power generation function depends on the product of the turbined outflow and the net head on the turbine:

Pt = η(st, st+1, rt) · ρ · g · rt· ht(st, st+1, rt) (2.14)

where Pt[W] is the power produced in the plant, η is the overall efficiency of the power plant, ρ [kg/m3] is the density of water, g [m/s2] is the acceleration due to gravity, rt [m3/s] is the release through the turbines and ht [m] is the net head which is a non linear function depending on the storage levels at the begening, st and at the end st+1of the period and the head losses.

To deal with the head effects on the hydropower production function Pt, a convex hull approximation is stored in the constraints set (2.15). The linear parameters ψ, ω and δ are determined using the procedure described inGoor et al.(2011).

1

2

3

4

Reservoir

Power Plant

𝐶𝑀

=

−1

0

0

0

0

−1

0

0

0

+1

−1

0

+1

0

+1

−1

𝑞

𝑞

𝑞

𝑞

𝑟

𝑟

𝑟

𝑟

Figure 2.3: Example reservoir system configuration and connectivity matrix (adapted from Labadie

(2004))            b Pt− ψ1st+1/2 − ω1rt≤ ψ1st/2 + δ1 .. . b Pt− ψHst+1/2 − ωHrt≤ ψHst/2 + δH (2.15)

Then, the immediate benefits of equation2.11are calculated as:

bt(st, qt, st+1, rt) = J

∑

(πh( j) − θh( j)) bPt( j)τt− ξt|zt (2.16)

where τt is the number of hours in period t, π is the energy price [$/Wh] and θ is the operation and maintenance cost [$/Wh]. As defined in Equation2.5, zt is the vector of slack variables, penalized in the objective function by the vector ξt of penalties [$/unit].

The decision variables such as storage st+1, releases rt and spillage losses lt are limited by lower and upper boundaries:

s_t+1≤ st+1 ≤ st+1 r_t ≤ rt ≤ rt

Figure 2.4: Normalized Convex-Hull approximation at Passes-Dangereuses power station- Rio Tinto system, Quebec, Canada

2.6.1 Future Benefit Function Approximation

In SDDP, the hydrologic uncertainty is typically captured through a multi-site periodic autoregres-sive model (MPAR). This model is capable of representing seasonality, serial and spatial stream-flow correlations within a river basin and among different basins. It is also needed to analytically derive the FBF approximations, and to produce synthetic streamflows scenarios for the simulation phase of the iterative procedure. Furthermore, the convexity requirement of SDDP is guaranteed because the MPAR is linear (Infanger and Morton,1996;De Queiroz and Morton,2013).

Autoregressive Model

Autoregressive (AR) models have been extensively used in hydrology and water resources since 1960’s. Its popularity and attractiveness rely on the simplest formulation and its intuitive type of time dependence: variables at time t are dependent on the preceding ones. First introduced by Thomas and Fiering (1962) and later byBox and Jenkins(1970), AR models can be represented as models with constant parameters, parameters varying with time and combination of both. The models with constant parameters are often implemented for modeling annual time series. Models with periodic parameters are usually used with time series of intervals that are fraction of the year (e.g. seasons, months, weeks, etc.) Salas et al (1980). The latter models are referred to as periodic AR (PAR)

models, and the periodicity may be in the mean, variance and/or autoregressive parameters. Since the reservoir operation analysis involves time series at various geographic locations, spatial correlation is also required. Then, at each site j the hydrologic process can be modeled by a multi-site periodic autoregressive MPAR model of order p represented by:

qt( j) − µqt( j) σqt( j) ! = p

∑

where qt is the time dependent variable for year v and time t, with t=1,2,...,52 weeks. µqt and σqt are the periodic mean and standard deviation of qt, respectively, φi,t( j) are the autoregressive parameters of the p order periodic model, and εt( j) is a time independent-spatially correlated stochastic noise. Assuming that the noise εt( j) follows a 3-parameters (µv( j), σv( j) and κt( j)) log normal distribution:

f_{εt( j)}= 1 (εt( j) − κt( j))p2πσv( j) e−0.5  log(εt ( j)−κt ( j))−µv( j) σv( j) 2 (2.19)

with mean µεt, variance σ 2 ε ,t µεt( j) = κt( j) + e  µv( j)+σ 2 v ( j) 2  σ_{ε ,t}2 ( j) = e2(µv( j)+σ 2 v( j))_{+ e}(2µv( j)+σv2( j)) _(2.20) the lower bound κt( j) which ensure non-negative inflows qt( j) > 0, is defined from equation2.18is defined as: εt( j) > − µqt( j) σqt( j) − p

∑

and the parameters µv( j) and σv( j) determined as:

µv( j) = 0.5Log σε ,t2 Λ( j)(Λ( j)−1) (2.22) σv( j) = p log(Λ( j)) (2.23) Λt( j) = 1 + σε ,t2 κt2( j) (2.24)

the standardized stochastic noise Vt is estimated as:

Vt( j) =

log(εt( j) − κt( j)) − µv( j) σv( j)

The spatial statistical dependence of reservoir inflows is introduced by the lag0-covariances and cross variances of the standardized stochastic noise. Then for the collection of independent standardized noises Vt of each node j of the system Vt = [Vt(1), . . .Vt(J)], the spatial model can be written as:

Vt= AtWt (2.26)

where Wtis a column vector of J independent elements consisting of white noises, normally distributed with zero mean and variance equal to 1. The estimation of matrix ˆAcan be obtained from the Cholesky factorization of the covariance matrix of standardized noise at each node j:

ˆ

AtAˆ|t = Cov(Vt) (2.27)

SDDP and MPAR(1)

To derive the mathematical formulation of the FBF approximation let us assume an autoregressive model of order p = 1, then equation2.18can be written as:

qt( j) − µqt( j) σqt( j) = φt( j) q_t−1( j) − µqt−1( j) σqt−1( j) ! + εt( j) (2.28)

SDDP uses a two phases strategy to increase the accuracy of the solution by adding new cuts: a backward optimization and a forward simulation. Both phases require different sets of inflows. In the backward phase, K inflows scenarios (backward openings) at each node of the system are generated by the using the MPAR model. These scenarios are needed to analytically calculate the hyperplanes’ parameters, and ultimately to derive the upper bound to the true expected FBF. In the forward phase, the MPAR model generates M synthetic reservoir inflows sequences to simulate the system behavior over the planning period.

The calculation of the linear parameters ϕtl, γtl and βtl of the approximated Ft+1(equation2.12) relies on the primal and dual information available at the optimal solution. Let us say that at stage t, s◦t and q_t−1◦ are sampled and, in order to include the stochasticity of the problem, the K vectors of inflows q_tK are generated. The one-stage SDDP subproblem 2.11 to 2.17 is solved for K reservoir inflow branches qtk. The expected FBF Ft+1, stored in the form of cuts, is the expected value of the K FBF F_tk₊₁calculated for each inflow branch (Figure 2.5) (Tilmant and Kelman,2007).

According to the Kuhn-Tucker conditions for optimality the derivative of the objective function with respect to the state variables S is given by:

∂ F ∂ Si =

_∑

being λithe dual information of the optimization problem and githe linear constraints2.13to2.15.

Then, the slopes ϕtl,kand γ l,k t of the functions Ftk F_tk≤

_∑

∑

which will be added to the expected cost-to-go function at stage (t − 1), can be calculated as:

∂ F_tk ∂ st ( j) = ϕtl,k( j) = λw,tk ( j) + H

∑

∑

The partial derivative of the current inflow respect to the previous inflow, from equation2.28is ex-pressed as: ∂ qt ∂ qt−1 = φt( j)  σt( j) σt−1( j)  (2.33)

and λ_w,tk (J × 1), λ_hp,th,k (J × H × 1) and λ_c,tl,k[L × 1] are the dual information associated to water balance (equation2.13), the L cuts of the FBF (equation2.12) and the H linear segments of the power functions (equation2.15), respectively.

Taking the expectation over the K artificially generated flows, the slope vectors ϕtl and γt,1l can be determined: ϕtl( j) = 1 K K

∑

∑

Finally, the constant term is given by: βtl = 1 K K

∑

∑

∑

As mention earlier, the backward optimization generates an outer approximation of the FBF Ft+1 (Figure 2.2). The accuracy of the approximation is evaluated at the end of the forward simulation. This

Sampled storage (period t, iteration 1) Storage (period t+1, branch 1) s_{t + 1} Cut 1,1 Storage

(period t+1, branch k) Cut 1,k

Storage (period t+1, branch K) Cut 1,K Inflow branch k F_{t + 1} Aggregated Cut 1 F_{t + 1} F_{t + 1} F_{t + 1} Sampled storage (period t, iteration 2) Storage

(period t+1, branch 1) Cut 2,1

Storage

(period t+1, branch k) Cut 2,k

Storage (period t+1, branch K) Cut 2,K Inflow branch k F_{t + 1} Aggregated Cut 2 F_{t + 1} F_{t + 1} F_{t + 1} F_{t + 1} Approximated Cut Inflow branch 1 Inflow branch 1 Inflow branch K Inflow branch K s_{t + 1} s_{t + 1} s_{t + 1} s_{t + 1} s_{t + 1} s_{t + 1} s_{t + 1}

Figure 2.5: Backwards openings and the approximation of the FBF (adapted fromTilmant and Kelman

(2007))

phase yields all the successive states and decision for each of M historical or synthetically generated hydrologic sequences. Then the expected lower bound on the optimal solution is defined as:

Z= 1 M T

∑

where bmt is the immediate benefit at stage t for the hydrologic sequence m ∈ [1, 2, . . . , M]. This forward simulation phase provides a lower bound with a 95% confidence intervals which allow us to determine whether the upper bound is a good a approximation or not. If the upper bound does not fall inside the confidence interval of the lower bound, the approximation is statistically not acceptable and a new backward recursion is implemented with a new set of hyperplanes build on the storage volumes that were visited during the last simulation phase.

The 95% confidence interval around the estimated lower bound Z is calculated as:  µZ− 1.96 σZ √ M, µZ+ 1.96 σZ √ M  (2.38)

2.7

Climatological information as exogenous variable

Incorporating exogenous hydrologic variables into the state-space vector of SDDP offers the po-tential to improve the performance of SDDP-derived release policies. The natural extension to in-clude climatic variability into the autoregressive model is the MPAR model with exogenous variables (MPARX). Using p previous inflows qt and b past exogenous variables Xt, the incremental flow at node j, qt( j), can be derived from a multisite periodic autoregressive model with exogenous variables MPARX(p, b): qt( j) − µqt( j) σqt( j) = p

∑

∑

where µXt and σXt are respectively the vectors of the periodic mean and the standard deviation of the exogenous variables, and ϑκ ,tthe vector of the exogenous regressors. As indicated in Equation (2.39) the exogenous variables may cover a different range of past input values, from ι to b, not necessarily starting from t − 1. This is significant in time-delay systems where the effect of an input may become active after a certain time period (Marmarelis and Mitsis,2014).

MPARX(p,b) Exogenous variables Endogenous variables.

𝑞

𝑞

𝑋

Figure 2.6: The ARX model structure (adapted fromLjung(1999))

2.8

Hydropower scheduling and contract management

Since the deregulation of the electricity market, in both developed and developing countries during the eighties and nineties (Boubakri and Cosset, 1998), a variety of tools and methods have been developed to jointly analyze operation scheduling and contract management (Mo et al.,2001;Gjelsvik et al.,2010;Mo and Gjelsvik,2002;Kristiansen,2004;Flach et al.,2010). The typical hydro-based producers face different types of risk such as price risk and quantity risk caused by both inflow and

demand uncertainty. In order to reduce the risk exposure, producers usually trade a variety of contracts dealing with physical (power plants) and financial assets (energy contracts). As examples of contracts, we can find future and forward contracts, option contracts, and load factor contract. The latter, usually named as flexible contract is a physical or financial contract between two parts where price, energy and maximum power (load factor) is predetermined, but its use is flexible, meaning that buyer determines the amount of energy to be bought (Mo et al.,2001).

The mathematical description of a flexible contract follows the stage to stage energy balance, ac-counting for the maximum possible withdrawal of energy ut= Pcτt [MWh], and the amount of energy remaining of the contract wt [MWh]:

wt+1= wt− ut (2.40)

with the initial energy amount of the flexible contract w0[GWh]

wt = w0 (2.41)

and

Pwτt ≤ ut≤ Pwτt (2.42)

where Pw [MW] is the instant power that can be withdrawn and τ the number of hours in period t. Moreover, the demanded load Dt [MWh] must be met with the energy produced by the system Ptτ and the sale/purchases through the contracts:

Ptτt+ up,t− us,t= Dt (2.43)

These energy purchases up,t[MWh], and energy sales us,t[MWh] can be introduced in the immediate benefit function (Equation2.16) through the vector of slack variables zt, and consequently the prices would be included in the vector ξt.

2.9

SDDPX formulation

A variant of the SDDP algorithm capable of incorporating various hydrologic information in the decision-making process is developed. This new formulation, called SDDPX, incorporates exoge-nous variables Xt, such as snow water equivalent and/or sea surface temperature in the state space vector together with the previous inflows qt. Using p previous inflows qt and b past exogenous vari-ables Xt, the vector of hydrologic state variables ht becomes [qt−1, qt−2,..., qt−p, Xt−κ,..., Xt−b]. This hydrologic information is encapsulated in SDDPX through a built-in MPARX model.

𝑠_𝑡 𝑞_𝑡 𝑟_𝑡 𝑠_𝑡+1 Stage t 𝑞𝑡+1 𝑟𝑡+1 𝑠_𝑡+2 Stage t+1 𝑠_𝑇 𝑞_𝑇 𝑟𝑇 𝑠𝑇+1 Stage T 𝑠₁ 𝑞₁ 𝑟1 𝑃₁𝜏₁ 𝑠2 Stage 1 𝑤₀ 𝑤_𝑡+1 𝑤𝑡+2 𝑤𝑇 𝑤₁ 𝑤₂ 𝐷_𝑡 𝐷_𝑡+1 𝐷_𝑇 𝐷₁ 𝑢_𝑝,1 𝑢_𝑠,1 𝑃_𝑡𝜏_𝑡 𝑢_𝑝,𝑡 𝑢_𝑠,𝑡 𝑃_𝑇𝜏_𝑇 𝑢_𝑝,𝑇 𝑢𝑠,𝑇 𝑃_𝑡+1𝜏_𝑡+1 𝑢𝑝,𝑡+1 𝑢𝑠,𝑡+1 𝑤_𝑇+1

Figure 2.7: Schematization of energy trade

If at stage t, the system status is described by the storage st, the hydrological variable, and the amount left of energy wt in C energy contracts, the one-stage SDDPX optimization problem can be written as:

Ft(st, qt−1, ..., qt−p, Xt−κ, ..., Xt−b, wt) = max{bt(st, qt, rt, st+1, ut) + Ft+1} (2.44) subject to:            Ft+1− ϕt+1l | st+1− χt+1l | wt+1≤ Γlt+1 | ht+1+ βt+1l .. . Ft+1− ϕt+1L | st+1− χt+1L | wt+1≤ ΓLt+1 | ht+1+ βt+1L (2.45) Γt+1|ht+1= γt+1,1|qt+ γt+1,2|qt−1+, ..., γt+1,p|q(t−p)+1 γt+1,p+κ|X(t−κ)+1+, ..., +γt+1,p+b|X(t−b)+1 (2.46)

the mass balance equation (2.13), the approximation of the hydropower functions 2.15, the trade energy balance (2.43), the energy balance in the purchase contracts (2.40), and the lower and upper boundaries (2.17and2.42).

In the backward phase the main modification to the traditional SDDP formulation lies in the calcula-tion of the hyperplanes’ parameters ϕt+1, χt+1, βt+1, γt+1,1, γt+1,2,..., γt+1,p, γt+1,p+κ ,..., and γt+1,p+b (see Equations 2.45and2.46). Using the Kuhn-Tucker conditions for optimality, the change of the one-stage objective function Ft respect to the state variables st, wt, qt−1, qt−2,...,qt−p, Xt−κ,..., and