A Machine Learning Approach for the Smart Charging of Electric Vehicles

(1)

A Machine Learning Approach for the Smart Charging of

Electric Vehicles

Thèse

Karol Lina Lopez

Doctorat en génie électrique

Philosophiæ doctor (Ph. D.)

(2)

A Machine Learning Approach for

the Smart Charging of Electric Vehicles

Thèse

Karol Lina Lopez

Sous la direction de: Christian Gagné

(3)

Résumé

Avec l’adoption croissante des véhicules électriques, il y a un intérêt pour utiliser des tarifs dynamiques dont le prix dépend de la demande actuelle, pour encourager les utilisateurs à recharger leurs véhicules en période de faible demande évitant les pics d’électricité pouvant dépasser la capacité installée. Le problème que devaient affronter les utilisateurs de véhicules électriques est qu’ils doivent s’assurer que l’énergie électrique présente dans les batteries est suffisante pour les déplacements et que les périodes de recharge correspondent à des périodes où le prix de l’électricité est bas. La plupart des approches actuelles de planification de recharge supposent une connaissance parfaite des futurs prix de l’électricité et de l’utilisation du véhicule, ce qui nuit à leur applicabilité dans la pratique.

Cette thèse considère la modélisation de la recharge intelligente des véhicules électriques pour déterminer, lors des sessions de connexion, les moments où le véhicule doit se recharger afin de minimiser le coût payé pour l’énergie de ses déplacements. La thèse comporte quatre principales contributions: 1) Modèle de recharge optimale des véhicules électriques pour générer une série de décisions en utilisant la connaissance a priori du prix de l’électricité et de l’énergie utilisée, en utilisant la programmation dynamique comme méthode d’optimisation. 2) Création d’un modèle de système d’information incluant des variables connexes au modèle de recharge des véhicules électriques dans un cadre guidé par des données. 3) Méthode de sélection des données pertinentes utilisant la stratification de données pouvant réduire significativement le temps requis pour entraîner les modèles de prévision avec des résultats proches de ceux obtenus en utilisant l’ensemble de données complet. 4) Modèle de classification en ligne qui permet de déterminer s’il faut charger ou non le véhicule à l’aide de modèles d’apprentissage automatique qui peuvent générer, en temps réel, une décision de recharge quasi-optimale sans tenir compte d’une connaissance de l’information future.

Nous démontrons comment la combinaison d’une méthode d’optimisation hors ligne, telle que la programmation dynamique, avec des modèles d’apprentissage automatique et un système d’information adéquat peut fournir une solution très proche de l’optimum global, sans perte d’applicabilité dans le monde réel. De plus, la polyvalence de l’approche proposée permet d’envisager l’intégration d’un plus grand nombre de variables à l’entrée du modèle, ainsi que d’autres actions comme par exemple fournir d’énergie au réseau électrique pour aider à réduire les pics de demande ce qui pourrait être utile dans un contexte de vehicle-to-grid (V2G).

(4)

Abstract

With the increasing adoption of electric vehicles, there is an interest to use dynamic tariffs where the price depends on the current demand, encouraging users to charge their vehicles in periods of low demand, avoiding electricity peaks that may exceed the installed capacity. The issue an electric vehicle user must tackle is that it should ensure that its electric power is sufficient for its trips and that the recharge periods correspond to periods where the price of electricity is low. Most current charge scheduling approaches assume a perfect knowledge of the future prices and car usage, which hinders their applicability in practice.

This thesis considers the modelling of the intelligent recharge of electric vehicles to determine, during the connection sessions, the times when the vehicle may be charged in order to minimize the overall energy cost. The thesis has four main contributions: 1) Optimum electric vehicle recharge model to generate a series of decisions using full knowledge of the price of electricity and energy used using dynamic programming as a method of optimization. 2) Creation of an information system model which includes variables relevant to the recharging model of electric vehicles in a framework data-driven. 3) Method of selecting relevant data using the stratification by clusters which can significantly decrease the time required to train forecasting models with results close to those obtained using the complete dataset. 4) Classification model which allows the determination of whether or not to charge the vehicle using machine learning models that can generate, in real time, a near-optimal recharge decision without considering perfect knowledge of the future information.

We demonstrated how combining an offline optimization method, such as dynamic program-ming with machine learning models and a coherent information system can provide a solution very close to the global optimum without loss of applicability in real-world. Moreover, the versatility of the proposed approach allows the consideration of the integration of a larger set of variables at the input of the model, as well as other actions such as for example supplying energy to the network to further help reducing demand peaks which could be useful in a vehicle-to-grid context (V2G).

(5)

List of Tables

1.1 Advantages and drawbacks of Electric Vehicle (EV) charging schemes. . . 17

2.1 Average gain (in %) for the 17 vehicles with respect to the basic 10-bins scheme for each ∆t, in the summer and winter training periods. . . 28

2.2 Cost and gain of Dynamic Programming (DP), Always Charge (AC), and Random Decisions (RD) models in summer and winter. . . 30

3.1 Outdoor Temperature basic statistics . . . 36

3.2 Total and single trip maximum distance made by vehicles . . . 38

3.3 Distance traveled and Energy used by vehicle basic statistics. . . 39

3.4 Hourly Ontario Energy Price basic statistics . . . 41

3.5 Hourly Ontario Demand basic statistics. . . 43

3.6 Gasoline price basic statistics . . . 45

3.7 Lineal correlation between the variables explored in the Information System.. . 45

3.8 PCA Coefficients for each variable and for the first five principal components. . 46

3.9 Influential variables on the principal components according to absolute coeffi-cient of the Information System (IS) in Summer. . . 50

3.10 Influential variables on the principal components according to absolute coeffi-cient of the IS in winter. . . 50

4.1 Forecasting errors using stratification in input space and Artificial Neural Network (ANN). . . 69

4.2 Forecasting errors using stratification in input space and Support Vector Regression (SVR). . . 69

5.1 Error in training and testing of the best configurations of the k-Nearest Neighbors (k-NN) model in summer. . . 82

5.2 Error in training and testing of the best configurations of the k-NN model in winter. . . 82

5.3 Error in validation and testing of the best configurations of Shallow Neural Network (SNN) model based on the Principal Component Analysis (PCA) input space in summer and winter. . . 85

5.4 Variables selected in the first 10 iterations of the SNN model in both summer and winter scenarios. . . 86

5.5 Classification error in training and testing of Deep Neural Network (DNN) model for all EVs in summer and winter. . . 90

5.6 Classification error in testing of Machine Learning (ML) models evaluated in summer and winter. . . 90

(9)

5.7 Mean gain ratio of cost evaluated in testing of all charging models in summer

and winter. . . 93 A.1 Specifications of battery packs, based on Chevrolet Volt 2013 and Panasonic

NRC18650 Li-Ion batteries. . . 103 A.2 Charging Standards of EVs. . . 103

(10)

List of Figures

1.1 Conceptual map of Smart Charging of Electric Vehicles (SCEV). . . 8

2.1 Illustration of two sequences of charging decisions computed with DP. . . 26

2.2 Electricity price (HOEP) and distance traveled by a vehicle over 7 days. . . 27

2.3 Average cost for the different number of bins by ∆t for summer and winter training periods. . . 29

3.1 Outdoor temperature in o_{C from May 2008 to May 2009.} _{. . . .} ₃₇

3.2 Energy used by car 203 . . . 39

3.3 Hourly Ontario Energy Price. . . 41

3.4 Hourly Ontario Energy Price basic statistics. . . 42

3.5 Hourly Ontario Demand.. . . 43

3.6 Gasoline price. . . 44

3.7 Clusters in the PCA space . . . 47

3.8 Cumulative percentage of the total variance explained by each principal component . . . 49

3.9 PCA on the IS for EV-263 on summer. . . 51

3.10 PCA on the IS for EV-263 on winter.. . . 52

3.11 k-means clustering for EV 222 - summer shown on PCA space. . . 53

4.1 Determining the number of clusters. . . 63

4.2 Hourly Ontario Energy Price (HOEP) forecasting using stratification in input space and ANN . . . 67

4.3 HOEP forecasting using stratification in input space and SVR . . . 68

4.4 Average and dispersion of the trimmed MAPE using ANN . . . 70

4.5 Average and dispersion of the trimmed Mean Absolute Percentage Error (MAPE) using SVR . . . 70

5.1 Overview of proposed methodology.. . . 73

5.2 Stratified sample used in training for the EV 222 in summer shown on PCA space. . . 75

5.3 Stratified sample used in validation for the EV 222 in summer shown on PCA space. . . 76

5.4 Decision curves of the Threshold-Based Rule (TBR) model for both the summer and winter seasons. . . 79

5.5 k-NN model performance in a stratified validation sample. . . 81

5.6 SNN model performance in stratified validation sample for different network configurations for the EV 203 in summer. . . 83

(11)

5.7 Evolution of classification error in training and testing for DNN model according

to the number of epochs used. . . 89 5.8 Gain of different ML models on the summer and winter test periods for 17

vehicles tested with our approach. . . 92 A.1 Electric circuit equivalent of the battery model used. . . 101 A.2 Charging profile used for the experiments, based on the specifications of the

(12)

A mis amores: Violeta, Celeste y Darwin A la fortaleza de mi madre A la alegría de mi hermano A la memoria de mi padre

(13)

But instinct is something which transcends knowledge.

We have, undoubtedly, certain finer fibers that enable us to perceive truths when logical deduction, or any other willful effort of the brain, is futile.

(14)

Remerciements

La réalisation cette thèse de doctorat a été possible grâce au concours de nombreuses personnes à qui je voudrais exprimer toute ma reconnaissance. En tant que femme, immigrante, épouse et mère de deux enfants, concilier la famille et les études n’a pas été toujours facile. Mais, j’ai eu l’honneur de rencontrer et de collaborer avec de grands scientifiques et de gens passionnés par l’intelligence artificielle et par les voitures électriques, ainsi que de personnes de grand cœur qui ont cru en moi et qui m’ont donné beaucoup de bonheur et de support émotionnel, que je ressens le besoin de remercier individuellement.

Tout d’abord, je voudrais remercier mon directeur de recherche, M. Christian Gagné, pour son encadrement et son soutien tout au long de mon doctorat. Ses encouragements continus, ont été définitifs dans la réussite de cette thèse. Merci vraiment.

Je tiens à remercier Luc Vouligny et Arnaud Zinflou qui m’ont encadrée lors de mon stage à l’Institut de recherche d’Hydro-Québec (IREQ) ainsi qu’à Eric Beaulieu, président de la compagnie E Machine Learning Inc, pour m’avoir permis de poursuivre ma recherche en milieu industriel.

Je voudrais également remercier les membres du jury d’avoir accepté d’évaluer ma thèse et pour le temps consacré à la lecture du document ainsi que pour leur présence lors de ma soutenance. Toutes ses remarques, toujours constructives, m’ont permis d’améliorer la version finale du document.

Je tiens aussi à remercier tous les membres du laboratoire de vision numérique pour la bonne ambiance de travail. J’aimerais remercier spécialement à Annette Schwerdtfeger, pour la lecture et correction des articles publiés et lors de la rédaction de ma thèse, l’organisation de parties et pour son accueil si chaleureux à tous les étudiants indépendamment de leur origine. Je tiens également à remercier tous les professionnels de recherche, qui donnent beaucoup d’assistance technique et de bons conseils. Entre autres, Thierry Moszkowicz, Sylvain Comtois et Clemente Ibarra-Castanedo.

Je souhaiterais aussi adresser ma gratitude à Marc-André Gardner, qui m’a fait l’honneur d’être coauteur d’un de mes articles; il a pris le temps de m’écouter, de discuter avec moi et de m’accompagner dans les préparatifs de ma soutenance. Tout cela m’a permis d’avancer mon travail et j’en suis très reconnaissante.

(15)

Je remercie également tous les étudiants avec qui j’ai échangé, particulièrement, Yannick, Ah-med, Vahab, Billy, Matthew, Frédéric, Adel, Waël, les deux Maxime, Audrey, Julien-Charles, François-Michel, Mahdieh, Sophie, Felix-Antoine, Maryam et bien d’autres.

Je n’aurais jamais pu réaliser ce travail doctoral sans l’aide précieuse de mon conjoint Darwin Brochero, et sans le soutien, générosité et patience de mes deux filles Celeste et Violeta. Vous ne pouvez pas imaginer tout ce que l’amour et le soutien affectif sans faille m’ont apporté pour terminer ma thèse. Merci du plus profond de mon cœur.

J’exprime ma gratitude à ma famille élargie, à mes amis colombiens et québécois, pour les bons moments partagés, c’est une chance immense de vous avoir dans ma vie.

Mais surtout, je souhaite aussi exprimer ma gratitude à ma mère. Gracias mamá pour creer en mi, pour el apoyo incondicional, pour la fuerza de mujer guerrera que me ayudó a superar los momentos difíciles.

(16)

Abbreviations

k-NN k-Nearest Neighbors AC Always Charge

ANN Artificial Neural Network

APCA Adaptive Piecewise Constant Approximation BEV Battery Electric Vehicle

CPP Critical-peak pricing CV Cross-Validation DNN Deep Neural Network DP Dynamic Programming DSM Demand Side Management EDR Electric Driving Range EV Electric Vehicle

FGS Forward Greedy Selection G2V Grid-to-Vehicle

GAS Gasoline Vehicle

HOD Hourly Ontario Demand HOEP Hourly Ontario Energy Price IS Information System

MAPE Mean Absolute Percentage Error MDP Markov Decision Process

ML Machine Learning

PCA Principal Component Analysis PHEV Plug-in Hybrid Vehicle RBF Radial Basis Function RD Random Decisions RL Reinforcement Learning RMSE Root Mean Square Error RTP Real-Time Pricing

SCEV Smart Charging of Electric Vehicles SG Smart Grid

SNN Shallow Neural Network SoC State of the Charge SVM Support Vector Machine SVR Support Vector Regression TBR Threshold-Based Rule TM Outdoor Temperature TOU Time-of-Use pricing V2B Vehicle-to-Building V2G Vehicle-to-Grid V2H Vehicle-to-Home

(17)

Introduction

In addition to environmental concerns related to climate impacts, aspects such as governmental incentives, depletion of fossil fuels and improvements in battery capacity have contributed to the widespread adoption of Electric Vehicles (EVs) (including both battery electric and plug-in hybrids) and consequently to the growth of the electric charging infrastructure, in the last few years [23]. Although this provides numerous social and environmental benefits, the charging of a large fleet ofEVscarried out in an uncontrolled manner could have a negative impact on the power grid, threaten the stability, increase the risk of overload and even challenge the green electricity supply [27]. By anticipating a considerable increase in electricity demand and to prevent that common usage patterns may lead to many vehicles being charged simultaneously (e.g., at dinner time, during the night), especially if this charge coincides with traditional demand peaks, there is a particular interest in integrating Demand Side Management (DSM) intoEV charging [35;92;103].

DSMaims at proposing incentive-based measures to influence electricity consumption patterns, encouraging users to shift their electricity consumption away from peak periods in order to better flatten the demand throughout the day, ensuring that the existing infrastructure is efficiently used, minimizing investments in additional generation capacity [77]. Time-of-Use pricing (TOU) is an example of fixed rates charged to low demand customers (less than 50 kW) in Ontario, Canada1_{, which varies based on the time of day, week and season that customers}

use electricity. Many power utility companies such as HydroQuébec, are planning to adopt measures to improve current pricing practices using hourly or minute price signals such as Real-Time Pricing (RTP) which could better reflect the real cost of power electricity. In that context, the issue that an EV must face is, on one hand, to ensure that the vehicle has enough energy for its trips avoiding a depleted battery, especially in the case of Battery Electric Vehicle (BEV) which is not supplemented with a gasoline engine. On the other hand, the charging times should be chosen carefully, since using simple charging strategies such as charge as soon as the vehicle is plugged-in could be costly given that the fleet of EVs users could use the same strategy, which could lead to an appreciable increase of electricity prices [33;75]. However, some users argue that there are no intelligent devices that can respond to a dynamic rate benefiting of low tariffs thereby allowing to reduce the electricity costs while

1

(18)

meeting the user’s particular transportation needs [33;75].

The problem of Smart Charging of Electric Vehicles (SCEV) can be referred from different per-spectives according to the optimization objective. In this thesis, theSCEVis vehicle-oriented, where each EV is equipped with a system able to make their own charging decisions in order to reduce charging costs, considering their own driving patterns. Most of the previous studies of SCEV were focused on the optimal charging schedule, which is related to determining a sequence of actions (charge decisions) in an interval of time limited by the arrival and departure time (connection time). This type of problem usually has been formulated in two steps. The first step includes methods to predict future values in a conventional time-series forecasting problem. The second step considers an optimization problem that seeks to determine the control variables related to the charging rates and times. Although these two steps have been widely applied to solve charging schedule problems, there are disadvantages to their implementation in real time because the performance of these approaches depends on the accuracy of the predictions of future information or temporal distribution estimations of variables, leading to inefficient schedules that are not based on actualEVuser requirements. To overcome this difficulty, instead of determining an optimal charging schedule using traditional optimization models, we are proposing an approach based on the exploitation of historical data to obtain an end-to-end learning model able to make the decision whether or not to charge an EV in real-time during connection times without requiring future information. bi

Motivation

The scenario of EVs has changed. We have more and more EVs circulating, with different characteristics and patterns of use. The electric utility companies have started to use dynamic rates to prevent manyEVsto charge their batteries at the peak times (e.g., at dinner time) or even generate peaks in times where usually the demand is low. Despite the fact that variable tariffs have been used over a number of years in DSM with a high degree of acceptability, the intelligent devices that can respond to these dynamic rates are not common and require significant work to make them effective [69].

Our main practical motivation is to develop an automated system to respond to a dynamic rate that enables the decision of when to recharge the batteries of an EVin such a way that it satisfies the user’s mobility needs while allowing the cost of operation to be minimized. Although this motivation can be financially beneficial toEVusers, this also can unequivocally encourage the voluntary participation of EVs in DSM, which is advantageous for the power grid since it can help to reduce demand peaks and flatten the load curve, in order to circumvent the necessity of building new power plants [115].

Given that the environment is dynamic both for the vehicle that has its own characteristics and their own mobility patterns as well as for the power grid that should adjust the price to

(19)

variable demand, we also want to make an analysis of available data of both vehicles and the power grid in such a way that the outcome can be an Information System (IS) that includes the more relevant variables from which we can make more intelligent EV charging decisions with data-driven models.

We are motivated to develop a system that uses the most advanced data technology to find empirical support for driven-data SCEV. We do so by using Machine Learning (ML) models that allow us to learn from the IS so that the system can be executed in real-time and can work, even without having information about the future.

Problem Definition

The thesis addresses the problem of the SCEV as a decision-making system supporting EV users, to determine the best times to recharge anEVso that the operational cost is minimized. The SCEV is understood within the framework of a dynamic environment, where both the price of electricity and the driving patterns are variable, so the system has to be flexible enough to allow the adaptation to changing circumstances. On the basis of current and historical information, and considering that the future information is unknown, it will be necessary to determine the variables that influence the recharge of an EV in such a way that only useful information is included in the system. In addition, it is necessary that the system considers the temporary resolution of the variables in order to obtain a model able to adjust to time intervals. With regard to the operation cost, it is important to mention that it should be defined as the sum of the charging cost incurred by the vehicle for the prices paid for the actual energy transferred to the battery. This cost function should contain a penalty parameter as a way to punish an EV running out of energy. The problem definition has to consider the particular driving patterns, a realistic model to calculate the energy consumption and the State of the Charge (SoC) of batteries. The model has to be scalable and extensible to a large EVs population facilitating better integration ofEVs with theDSM.

Objective

The principal objective of this thesis is to obtain a decision-making model to SCEV able to determine the best times to charge the vehicle so that the operation cost is minimized. In the following, the specific objectives and the resulting research questions for achieving this objective will be detailed.

Define SCEV as a Decision-making Problem

It has been shown that a planning model to determine the times to recharge anEVwhere the connection times are fixed and the driving patterns of the vehicles are not considered could result in ineffective schedules [74]. The work associated with this objective aims at answering

(20)

the following research question: Is it possible to obtain an optimal sequence of actions of a decision-making problem related to SCEV considering some driving patterns and energy consumption model of the vehicle?

Define an Information System related to SCEV

The objective of defining an information system is to identify the variables suitable forSCEV that can be used to complement the simple variables, allowing a richer state representation of the vehicle. This stage comprises the preparation and statistical analysis of the historical data related to the power grid such as demand and electricity price as well as mobility patterns of EVs. The following question is associated with this objective: how to explore the different variables related toEVcharging that would describe a more complete picture of theEVstate?

Propose Methods to handle Large Time Series Datasets

Given that the information to be processed in a SCEV problem corresponds largely to time series such as the price of electricity and vehicle usage, considerable growth of these series is expected with the passage of time. This must be handled with great attention. For that purpose, we will propose some strategies to obtain a small subset of data that allows the selection of the most useful data instances, leaving aside redundant or non representative instances that contribute less to forecasting and decision-making. The question that arises is: how to determine an unsupervised method for selecting a small subset from a bigger time series dataset?

Propose Machine Learning Models for SCEV

Thinking of the growing volumes and varieties of data related to the EVs and smart grid, the definition of machine learning models for SCEV aims at processing large and complex data to identify patterns and make decisions with minimal human intervention. This should be a profitable opportunity to exploit and to give value to data. The research question we aim at answering here is: how we can design MLmodels that can make effective real-time decisions for SCEV?

Plan

This thesis is organized in five chapters. Chapter 1 presents an overview of the different frameworks and principal subjects underlying to theSCEV. Chapter2presents the formulation and development of the optimal charging schedule as a decision-making model that can be solved with a classical optimization method, such as Dynamic Programming (DP). Chapter3 covers univariate and multivariate time series analysis of the exogenous variables that may directly or indirectly influence the recharging of EVs and show how it can be used in an

(21)

IS. Since the analysis of historical data may be complex due to the requirements to store, prepare, and process huge amounts of varied information, Chapter 4 introduces the concept of the stratification by clustering to select a small set of data avoiding the redundancy in large time series which may be used in situations when big memory is involved in processing forecasting functions. Chapter 5 features the convergence between the recent MLalgorithms and traditional optimization methods such as the DP in a learning process facilitating the SCEVin real time. Finally, Chapter5.4presents some concluding remarks, contributions and future work.

(22)

Chapter 1

Electric Vehicle Charging Schemes

Smart Charging of Electric Vehicles (SCEV) is part of a system where the Electric Vehicle (EV) users interact with the complex electrical infrastructure. This element of the system has been driven by a new paradigm called Smart Grid (SG), which integrates infrastructure, communication technology, huge data collection, control, intelligent decision making and smart sensors/actuators for various smart energy applications enabling real-time interconnection between the supply and demand points.

This technology allows users to update their energy consumption profiles based on real-time power market information and support the interactions among EVs, charging stations and utility providers, considering services such as demand regulation, pricing, locating and reserving charging stations, management of energy flow and data security around the information exchange, ensuring effectiveness, reliability, security, environmental impact, and interoperability [34].

Within theSG, an effective control technology must support automated charging management such that real-time prices are communicated fast enough to allow real-time charging decisions. Requirements for communications between SGandEVusers or fleet controllers can be found in Wang et al. [105].

Demand Side Management (DSM) is a way to allow decentralized consumption control within a SG. DSM aims at proposing incentive-based measures to encourage users to shift their electricity consumption to periods of low demand in order to efficiently use the existing infrastructure, minimizing investments in additional generation capacity [77].

It has been demonstrated that a well-definedDSMstrategy can handle a large number ofEVs and other electric devices, reducing the operating cost of both electric utilities as well as of the end-users [59]. The development of effective DSM results from a trade off between monetary incentives and EV user willingness, which largely depends on the manner in which the EV charging periods are managed in response to dynamic price signals.

(23)

which have been foreseen to promote energy-efficient behavior [22]. Typically, this rate is adjusted using a Block Pricing method which is based on the quantity of the energy consumed. In this method, the consumer pays a fixed electricity rate for each block of energy consumed. Another way is to use a Dynamic Pricing by the time of day, week or season, which aligns the price of energy with the cost of producing that energy according to the demand [33]. Several dynamic prices have been proposed for volunteer participation of the EV users in DSM programs, such as Time-of-Use pricings (TOUs), Critical-peak pricings (CPPs), and Real-Time Pricings (RTPs) [75].

Time-of-Use pricing (TOU) varies depending on when the electricity is consumed, based on the time of the day, the season and day of the week1_{. Critical-peak pricing (CPP) is similar to}

a TOUpricing, with a significantly higher price tier during peak periods, for a quick response to an emergency. Real-Time Pricing (RTP) are tariffs that vary on an hourly or sub-hourly basis according to the current market demands and generation costs.

RTP is known as the price that best reflects the changes in the wholesale price to the end consumers, allowing customers and utilities to move closer towards its optimal operating point helping to prevent anEV-charging peak in times where electricity consumption and price are usually lower and thus allowing EVowners to reduce their electricity cost [17;49].

Another element to define in the SCEV system is the type of interaction: unidirectional or bidirectional: The concept of unidirectional power flow assumes that power flows from the Grid-to-Vehicle (G2V). Some concepts such as Vehicle-to-Grid (V2G), Vehicle-to-Building (V2B) and Vehicle-to-Home (V2H) technology assume a bidirectional flow of energy allowing charging and discharging. V2HandV2Bis when anEVprovides emergency backup power to homes or buildings respectively and V2Gwhen a vehicle provides reverse power flow into an electric grid for regulation purposes.

Taking advantage of the bidirectional interaction possibility, numerous initiatives have been crafted to make up the cost and early wear of the batteries by utilizing the energy stored in batteries as an alternative ancillary service resource, provide storage capacity, reliable emergency backup services, grid support, backup stabilization, and operation cost reductions when electricity prices are unusually high [28;44;93].

The benefits evidently are the distributed storage capacity of the electrical energy of the batteries. This undoubtedly favors the supply of energy to the electricity grid in specific periods reducing grid operation costs allowing EV owners to have the option of earning incentives. However, it is crucial to consider the effect of aging due to the provision of auxiliary service because the battery life can be shortened [21;41]. In addition, related to the adoption ofV2Gtechnologies, other barriers have to do with concerns of efficiency of energy conversion

1_{The TOU pricing has been adopted, for example, by the Pacific Gas and Electric Company in California,}

USA, and by the Ontario Energy Board in Ontario, Canada, offering to the consumer the possibility of selecting the rates plan that best fits their needs, which may have a price tier according to the demand (Off-peak, Mid-peak, and On-peak)

(24)

Figure 1.1 – Conceptual map of SCEV.

[29], the associated environmental cost [96], incurring major modifications of the infrastructure to provide individual metering, fast response, flexibility and extendability [42], not to mention that the contracted capacity in ancillary markets is usually in megawatts which is considerably higher than the capacity of an individual EV, usually expressed in kilowatts. Additionally, as the contract terms require times of connection, it can compromise the primary function of a vehicle that is the transportation [108]. This thesis has an exclusivelyG2Vfocus and therefore does not delve further into the details of V2G. However, as will be seen later it would not be complex to adapt it to the proposed optimization framework.

1.1 Conceptual Map

Fig. 1.1presents a conceptual map showing the aspects in which the scheme ofSCEVcan be framed. In principle, we have the type of architecture, then several types of objectives that have been commonly associated. Subsequently, the strategies and, finally, the requirements of the different charging models. These concepts have been covered by reviews related to the SCEVs inSGprepared by García-Villalobos et al. [39], Mukherjee and Gupta [74], and Wang et al. [105] among others.

According to the architecture concept which refers to the level at which the charging decision is made, there are two principal categories, that is centralized or decentralized [39]. In a centralized architecture, a central controller such as agreggator2 _{or utility has direct control}

of the charging decision of all EVs participants. This concept is normally grid-oriented, that

2_{An aggregator agent for electric vehicles is a commercial middleman between a system operator or smart}

(25)

means that the main objectives will result in a benefit to the power grid. In contrast, in a decentralized architecture, the charging decisions are managed directly by theEVuser. In this type of architecture, the objectives are usually user-oriented, in the sense that the benefit ofEV users is sought, for example, the minimization of theEVcost based on the preferences of each user. It could even consider a third hybrid architecture, where the EVscontrol their charging decisions in a decentralized way and the utility managed the participation of aggregators in a centralized way according to a complex model involving aggregator strategies to maximize their profit depending on the users’ requirements and the strategies for buying, storing and selling electric energy.

Besides architecture, the charging strategies can also be established from the information needed by the models. Thus, the strategies of the models for load regulation need a large amount of information ranging from the projections of the adoption of EVs and the analysis of energy sources to the individual characteristics of each EV among others factors. In the same way, aggregator strategies frequently also include the preferences of EV users, such as the state of charge desired at the end of the period of charge, connection time availability, and the management of energy prices as a source of uncertainty.

Finally, the user-oriented strategies, frequently focus on establishing statistical driving patterns to determine the possible time of connection [54]. Subsequently, such availability and the market behavior (including load and electricity price) are the principal information to consider in a scheduling optimization problem. Usually, mechanisms such as quadratic programming accompanied by energy price forecasts have been used to solve this problem [112].

The cost minimization is one of the objective functions most widely used in the problem of charge scheduling of EVs [74]. However, the definition of the charging cost in the problem of charge scheduling depends on the perspective (centralized or decentralized), the orientation of the perspective (grid or user-oriented), and the type of interaction or power flow direction (unidirectional versus bidirectional). Some of the definitions found from a user-oriented perspective are listed below:

• Yifeng et al. [112] define the charging cost as the payment that has to be issued by the EV owner for charging in the time interval, or the balance of the charging and discharging payment for providing frequency regulation. In such a charging cost model, the electricity price is a linear function of the total instant load.

• Hoang et al. [46] define the charging cost in a time interval as the cost for charging plus the traveling cost which is a function of the distance traveled or in the case of bidirectional power flow, the profit for discharging minus the traveling cost. In this scheme, theEV owners have the possibility to buy cyber insurance to ensure the lowest cost for charging or highest profit for discharging. The immediate cost function considers

(26)

the product of charging decisions, the insurance coverage, and the insurance availability to determine the immediate cost.

• Nguyen and Le [76] define the charging cost in two parts: the first part is the total electricity cost expressed in terms of the total power charged in a time interval and the second part is the discomfort cost, which is related to the preferred indoor temperature for example.

Another aspect of great relevance is the communication infrastructure which is much more simple in a decentralized architecture compared with that needed in a centralized architecture. In a decentralized perspective, the infrastructure needed could be summarized to sensors to register variables associated with the movement of the EV and the online update of energy prices. Instead, a centralized perspective could need an interconnected network that allows the stocking and sharing of the information of all of the participants in theDSM scheme.

1.2 Decision Control and Objectives

1.2.1 Centralized Scheme

In a centralized architecture, a single agent directly manages the charging of all EVs participating in aDSM program. In such an architecture, a global scheduling optimization is formulated to deal with the charging decisions of a fleet of vehicles according to some objective and the decision variables are optimized considering a set of general constraints. In this way, the central agent enables automatic scheduling over a future finite time-horizon discretized into timeslots [2].

This scheme can be adapted to different objectives, such as minimizing the aggregated loads or minimizing the generation cost for example. In addition, it usually needs to know the future information which includes the base load, electricity price curves, regulation up and down curves, the mobility information and the energy required by the driving cycle, etc. The constraints, in general, are related to the power grid and batteries capacity.

The centralized scheme is not included in the scope of this thesis which has an exclusively decentralized focus and therefore does not delve into the details of a centralized managing of the charging decisions.

1.2.2 Decentralized Scheme

The decentralized perspective to the charging control receives more attention because EV’s owners have complete control over their charging decisions, giving them more flexibility to participate inDSMprograms. The global optimization schemes covering the charging decisions of a fleet ofEVsin order to achieve the objectives of the grid or the aggregator are not always

(27)

the most beneficial to handle the particularities of each vehicle and may not lead to the most beneficial result for an EV owner.

A user-oriented charging schedule is a scheme that aims at minimizing the overall charging cost for an EVowner within a time horizon while guaranteeing the electric energy needed for the trips [105]. The cost function should include the state of the vehicle to determine the amount of energy that will be supplied to the battery during each fixed time interval t, to meet the State of the Charge (SoC) expected by the owner, subject to instantaneous energy and time constraints.

Although in a decentralized approach there is a central agent that controls the price-based mechanisms (utility or market dynamics), it does not require all information from all participating EVs alleviating the computation burdens of a centralized perspective. In such a decentralized approach, an EV may adequately respond to dynamic prices satisfying their own needs and avoiding the concerns related to allowing an external agent to remotely start and control the charging of EVs [30].

A decentralized approach may be grid-oriented or user-oriented, according to the objectives to be achieved. In a grid-oriented perspective, the objectives are the same as a centralized approach. In contrast, the interest of anEV is not necessarily to meet the needs of theSGor the aggregator but aims, on the contrary, to minimize the charging cost while meeting their energy needs [79].

Grid-oriented decentralized charging strategies have been proposed by several authors: Ma et al. [65] consider that a common electricity price profile, as a strategy to determine the total energy delivered to the EV in each time step, facilitates the minimization of a cost function that captures the trade-off between total generation cost, the costs associated with battery degradation and costs associated with overloading.

EachEVautonomously determines its optimal charging strategy with respect to this common electricity price profile. This electricity price profile is updated considering theEVsdecisions, repeating these two steps until the change in the price profile is negligible.

Inspired by the water-filling principle in a moving horizon, Mou et al. [73] aim to flatten the load curve improving the load balancing of every low-voltage transformer. For this, an optimal control sequence of the charging power of each EV is calculated at each time step assuming stochastic arrival/departure times, implementing only the first sample. Then, the horizon is shifted one sample and the optimization is restarted with new information concerning the measurements.

Ma et al. [64] have shown how a decentralized charging process can optimally handle the valley-filling problem using concepts from non-cooperative games. These authors, as well as Gan et al. [38], have demonstrated that a real-time electricity price scheme reflecting the demand, drives a unique Nash equilibrium that influences each EV decision to optimally

(28)

determine the charging profiles of a large population of EVfacilitating the minimization of its individual charging cost.

Ahn et al. [4] proposed a decentralized charging algorithm to load shifting and frequency regulation to minimizing both, electricity generation cost and carbon dioxide emissions of power plants. This algorithm requires having access to information of the predicted demand, accurate forecast of baseload, estimation of the number of EVs connected to the grid, the plug-off time and SoC of the vehicle and the AC frequency of the local grid.

As with their centralized counterparts, a decentralized grid-oriented approach also needs to have updated information of all EVsor accurate predictions of future information. Moreover, these algorithms require information about charging strategies adopted by other vehicles. The information that is normally furnished by theEVuser is, for example, the times at which the EV will be available for charging, location information, charging station availability, the SoC of the batteries, energy needs, and preferences. Sometimes private information is needed to calculate the actual energy needed according to the driving patterns, the associated expected energy consumption, and parameters such as the battery size and the maximum charging power, which can affect the welfare ofEV users.

In addition, it is important to consider that the information provided by EV users may not be consistent with the real needs as is shown by Robu et al. [85], where theEVusers follow a strategic behavior misreporting information (e.g. early departure time or higher consumption) to have a higher priority in the recharging of the vehicle relieving range anxiety3_{. This}

author proposes a mechanism based on game-theoretic principles to avoid speculation in the information provided by users.

Most of the works related to theSCEVare grid-oriented, leaving thus a gap in the user-oriented recharging problem which could help the EV user to defend their interests. This thesis is focused on the user-oriented perspective where the aim of the SCEV is to determine the charging schedule in a way that can minimize the operational cost of an EV.

1.3 Strategies and Requirements

1.3.1 Load Curve Regulation

Among the most frequent objectives to handle the load curve can be found [50;95]: • Maximizing the use of renewable energy resources.

• Maximizing the grid economic benefit.

3

Range anxiety is a term primarily used in reference to Battery Electric Vehicle (BEV) related to the fear that a vehicle does not have enough energy or the range is insufficient to reach its destination which is considered one of the main barriers to the broad adoption of fully electric vehicles.

(29)

• Minimizing generation costs. • Improving voltage profile. • Minimizing the power imported. • Reducing the peak load demand.

• Avoiding saturation of transformers and lines. • Maximizing load factor.

• Reducing needs of spinning reserve.

To achieve these objectives, it is necessary to define the technique to be used: peak clipping that aims to reduce peak demand levels, valley filling to increase base demand levels, load shifting with which the peak demand is shifted to periods of low demand, strategic conservation or load reduction, strategic load growth encouraging to limit increases in energy consumption, flexible load shape or redistribution of power requirements to different times [59].

To reach an optimal solution with these strategies, a centralized approach requires prior knowledge of a large amount of data that is difficult to forecast given their stochastic nature [83;112]. In addition, the number of control variables increases dramatically with the number of EVsas well as the decisions that should be distributed simultaneously to all nodes in their network, which requires extensive bi-directional communication and heavy computations that may have drawbacks of impracticality and scalability [56].

In this type of optimization problem, it is difficult to establish a trade-off between the interest of a central agent and the interest of eachEV, given that a global schedule approach determines the coordination of EVs charging from a grid or aggregator oriented perspective [105]. For example, considering the valley filling technique, it is possible that a high weighting to the cost of the particular vehicles has a detrimental effect in the minimization of the energy cost of the central agent [65].

Khodayar et al. [51] propose the Monte Carlo method to minimize the grid operation cost. In this formulation, different probability distribution functions are used to simulate the wind energy, the load, the outages of the power system and the requirement of a fleet ofEVs. They conducted a comparative analysis between the charging control made by the power system operators and the control made by the EV users, from which we can conclude that when the consumers have the control, the grid operation cost increases as consumers introduce additional constraints related to the desired level of SoC before departure time.

The aggregator, for its part, should coordinate EV charging operations by designing pricing policy using, for example, the Nash Equilibrium in order to maximize the profit for all EVs, as proposed by Wu et al. [110].

(30)

Yifeng et al. [112] use a global optimization model which obtains the global minimum cost taking into account the energy constraints related to the battery and grid capacity. For this, a real-time price has been modeled as a linear function of the total instant load (base load plus the load of all of the vehicles). Such an optimization model assumes that the future information of base load and the departures/arrivals of vehicles are known.

Abbatantuono et al. [1] propose a Three-Phase Optimal Power Flow (TOPF) formulation to monitor and control low voltage grid so that dispatched charging power remains close to the power requested by the users using a non-linear optimization technique.

In Hoang et al. [46], a variable energy price for both charging and discharging can be different at each charging station according to market influence and demand. They propose using cyber insurance to provide a price guaranteed service by which the risk of paying a high cost for charging and a low revenue for discharging can be significantly reduced.

However, although these strategies have reached the objectives of the central agent considering network limits, the performance of a centralized approach is dependent on the accuracy of the predictions of future information and most of these centralized strategies are grid oriented or seek a common strategy to have an equilibrium for all the EVs participants that cannot consider the particularities of each vehicle, especially the stochastic driving behavior that has a huge influence on the energy needs. Additionally, the variables increase with the number of EVs, for which a centralized approach is not computationally tractable for large fleets.

1.3.2 Electric Vehicle User Particularities

Even though considerable efforts have been made to optimally schedule the charging of EVs in order to minimize the cost of energy, few have adopted a user-oriented approach. Rotering and Ilic [87] propose a strategy to obtain an economic benefit for the EV owners based on a forecast of future electricity prices using Dynamic Programming (DP).

The authors propose two algorithms that should ensure that the EV be fully charged at the end of the time horizon. The first aims to minimize the charging cost for anEVoptimizing the charging time and charging rate and the second takes into accountV2Gto provide a regulation service as a means of earning revenue. For this, the authors assume that future information of electricity prices, regulation up and down curves, the arrival and departure times, and the energy required is available.

Hoang et al. [46] propose an optimization problem in order to minimize the cost, making decisions (actions) in each coverage period (morning, afternoon, evening and night). There is no battery model involved, so the energy consumption rate was fixed. The action space is defined for all combinations of two groups of actions, the first containing the actions of charging, discharging (V2G), and standby, and the second group the actions to buy or not cyber insurance. The concept of cyber insurance aims to ensure the lowest cost for charging or

(31)

highest profit for discharging. The immediate cost function considers the Cartesian product of charging decisions, the insurance coverage, and the insurance availability to determine the immediate cost. A learning algorithm was proposed to learn an optimal policy to select the actions using an idealized gradient method as a function of a parameterized cost function. Ahmad et al. [3] present methods based on the price, on load and onSoCto control the charging rate in order to reach the minimum charging cost ofEV, considering a day-ahead price model via an evolutionary algorithm. TheSoCrepresents the state variable while charging rate works as a control variable in the objective function. The limitation presented by these methods is that they depend on the price forecast. In addition, the connection times are arbitrarily established, which does not correspond to the real operation of the vehicle.

Valogianni et al. [102] propose an optimization model in order to maximize individual welfare. They propose a characterization of the customer, with a simulation of driving behavior and real mobility data4_{, then used Reinforcement Learning (RL) to learn the individual household}

consumption. Once the household consumption is known and the algorithm determines the energy that must be charged, taking into account the energy constraints related to the battery, grid capacity and the energy needed to cover the daily trips.

Hutson et al. [47] propose a binary particle swarm optimization to determine the appropriate combination of charge and discharge times in a time window delimited by the arrival and departure times, in order to maximize the daily profit. In this algorithm, each particle defines whether the vehicle has to buy, sell or hold in each interval of time t. There is no evidence of a battery model which takes into account the connection to calculate the energy available in the battery.

On the other hand, we found that decentralized models have been slightly investigated for learning from historical information of distances traveled by the EVs. In the few cases where a history of trips is taken into account, it is only used to statistically determine the timing and quantity of electricity and gasoline consumption for a fleet of EVs [9].

Several surveys have been made in order to identify trends in travel behavior. One of them is the Household Travel Survey (NHTS) [70], which has conducted an investigation of the main variables related to travel behavior, among which can be listed the following: miles driven per year, trip purpose, trip distance, numbers of trips per day, etc. This information normally is grouped by age, gender, household income, disability/ability to travel, and rural or urban area location.

This information is useful to determine the profile of the load of the vehicles, for example Ashtari et al. [9] propose a stochastic method to predict EVs load, which can be used even when real vehicle usage data is not available; Grahn et al. [40] propose a model to determine the EV load profile as a function of the relationship between the trip purpose, the energy

4

(32)

consumption, and the charging opportunities.

However, there are two elements related to the driving pattern that needs careful investigation. First, there is the relationship between characteristics of EVs mobility and the electricity stored in battery packs. Secondly, there is also the charging pattern of EV, which is related to when and how to charge the vehicle. These characteristics have been investigated through statistical approaches or selecting distribution models in order to predict the impact of the EVs in the power grid, but the analysis from an individualEVcustomer’s perspective has not been explored [54]. The use of temporal distributions of departure and arrival times become the only mobility issues considered [97;106;112].

To develop a more realistic solution, it is necessary to capture dependence between the time and the states of the vehicle related to mobility aspects of EVs, such as the type of trip, charging opportunities, etc. Otherwise, when the optimization models use some statistics related to the energy consumption and connections times, they are actually losing the time-dependency of states of vehicles [40].

Likewise, the dependence of the charging models on predicted variables such as the electrical load or energy price [87] can be seen as a possible limitation of the user-oriented strategy, since they inherit the limitations of forecasting models, which are worth saying are quite complex in themselves. With reference to time-series analysis, recent efforts are increasingly focused on approaches based on deep learning [82]. Ryu et al. [88] use a Deep Neural Network (DNN) for short-term load forecasting while Mocanu et al. [72] proposed the use of DNN for the prediction of energy consumption. DNNfor time-series classification problems is also documented by Wang et al. [107].

It is worth mentioning that there are few studies that use personalized information of each vehicle, only a few go beyond the use of a day or a week in the establishment of the model. This is largely due to the volume of information that should be managed by the proposed model given that the temporal resolution of these models is to the minute to allow the capturing of the variability in the SoCand the variation in the price of energy. Furthermore, even though several works include the provision of variables to be used in an optimization problem, there is no study of the variables that could influence a decision-making model to make charging decisions, which could result in the basis of the Information System (IS) needed of a SCEV. In the meantime, significant hardware and algorithmic developments involving Machine Learning (ML) have shown astonishing results for complex problems in many fields. In particular, DNN as a disruptive approach in areas as diverse as computer vision and natural language processing [53]. In this regard, there is a convergence between the recent ML algorithms and the strategies proposed so far to produce a methodology that can be applied to theSCEVor in many other similar decision-making problems, e.g.,DSMfor smart appliances.

(33)

Table 1.1 – Summary of the advantages and drawbacks of the strategies of the EV charging schemes.

Objective Advantages Drawbacks

Grid oriented

• Well known architecture • Requires a large amount of data about • Better utilization of network capacity both the power network and ofEV

• Maximizing the use of renewable • Complex communication infrastructure energy resources • Computationally-demanding

• Direct peak load demand regulation • User privacy compromised • Avoid saturation of transformers and lines • Lack of modularity

• Range anxiety

User oriented

• Minimal data requirements • No direct load regulation

• Minimal communication infrastructure • Inheritance of lack of precision in • Privacy is not a concern forecasted variables

• May reduce range anxiety • Limited ancillary services provision • Non complex communications • Mainly statistical inference models

infrastructure required without online measurements • Computationally-demanding atEV • Models based on collective behaviors

scale (e.g. driving patterns)

1.4 Conclusion

In accordance with the objectives that have to be achieved, the framework to the charging schedule of EVs can be centralized or decentralized, which in turn can be grid-oriented or user-oriented. Under a centralized scheme, a central controller (smart grid or aggregator) has direct control over loads of all of theEV participants inDSM. In contrast, in a decentralized scheme, the control of the load is managed directly by the EV user.

In the case of a centralized perspective, its strong point is to allow direct control of the demand curve while the main drawback is on the difficulty to scale with the number of EVs in term of synchronization. In addition, this perspective can hardly to adapt the charging schedule to the real needs of each EV user. As for the decentralized perspective, the advantages lie in the flexibility over the charging policies with the possibility to use different strategies for each vehicle, with many customizations possible if desired and the fact that no complex communications infrastructure are required.

Most of the methods reported in this chapter, as in the literature, are grid-oriented, as they aim to reduce peak demand levels, shifting peak demand to low demand periods, and minimizing generation costs among others. This approach has the disadvantage of needing a large amount of information from all of the vehicles or a sufficiently accurate forecast to reach an optimal solution of an optimization problem. On the other side, a user-oriented approach seeks benefits for the EV user which can be formulated as a local optimization problem relying only on historical information of the vehicle and the price signal from the central agent (grid

(34)

or aggregator) to produce a personalized charging schedule.

Table 1.1summarizes the advantages and drawbacks ofEV charging schemes reported in the literature. From theEVuser perspective the minimization of the cost of use, and as a possible collateral effect, the regulation of the demand curve is the product of monetary incentives of strategies such as DSM.

In general, models conventionally used to charging time scheduling are limited by the accuracy of forecasts of some variables such as the load and the price of energy. Also, the temporary resolution of the scheduling problem, which is of the order of minutes, makes the size of the databases a challenge in the models of forecast and decision making. Likewise, there is not much evidence of the functioning of these models with data measured in the field (real databases), which is why these strategies are questioned by some authors.

(35)

Chapter 2

Simulation and Optimization of

Electrical Vehicle Charging Schedule

As mentioned in Chapter 1 a Smart Charging of Electric Vehicles (SCEV) aims at reducing charging costs for Electric Vehicle (EV) owners to take advantage of Demand Side Management (DSM) programs; selecting charging times away from peak hours. This incentive corresponds to the rate paid by consumers for charging their vehicle which could be adjusted by utilities to reduce demand peaks and flatten the load curve to circumvent the necessity of building new power plants (e.g. based on fossil energy) [115].

In this chapter, we want to determine the optimal charging schedule of an EV in order to minimize the cost assuming that both the electricity price and driving pattern of the vehicle are known including the future values. In addition, we suppose that the purpose of the battery of an EVis to provide power to drive the vehicle and not to supply the electricity grid. We formulate this as a sequential decisions problem, where the system of each vehicle should take an action (charge or standby) at each time step when the vehicle is connected, as a measure to respond to a dynamic electricity price while minimizing charging costs. Furthermore, when the vehicle is not connected, the system should calculate a reward or penalty according to the State of the Charge (SoC) of the battery during a trip.

We address the problem by defining the optimal value function that is computed by Dynamic Programming (DP), following Bellman’s principle of optimality [12], relying upon a database of the real use of EV over several months and an hourly electricity and gasoline price. As seen in the Chapter 1, several authors have proposed to useDP to determine the optimal charging schedule of anEV[57;87;89]. However, these optimization approaches are assuming predefined connection times and energy needs which cannot easily capture the dependence between the charging decisions and the real traveling behaviors. The analysis based only on connection times, where the driving patterns and the actual needs of the EVs are not considered, would produce impractical and inefficient schedules that are not based on actual

(36)

EV user requirements.

The results of ourDP approach are compared with two simple scheduling strategies. The first strategy attempts always to charge the battery when theEVis connected without considering the implication of the price or personal needs. The second strategy proposes to choose charging actions randomly. These approaches are evaluated through simulations, having, on the one hand, a database of measured data collected from a real fleet of cars, and, on the other hand, the data on the price of gasoline as well as the hourly electricity price.

The remainder of this chapter is organized as follows. In Sec. 2.1 the mathematical model of the smart charging schedule is provided. The modeling of this optimization problem as a Markov Decision Process (MDP) is provided in Sec. 2.2. Then, in Sec. 2.3 we describe the classic Dynamic Programming (DP) algorithm to solve that MDP by defining the optimal value function. Experiments with real-life datasets are presented in Sec. 2.4, demonstrating the effectiveness of the proposed approach, before concluding in Sec.2.5.

2.1 Optimization Problem

We define the optimal charging schedule of an EV as the best programming for charging an EV over time, in order to minimize the overall cost. This can be framed into a decision problem where we should determine the sequence of actions to take at each time interval (e.g., 15-minute period) when theEVis plugged in. Without limiting the generality of the foregoing, we consider that while an EV is connected, an intelligent system must decide whether to charge the vehicle (charge) or to leave it on standby mode (standby) at each time interval. The charging decisions should be based on the electricity price information, the SoC of the batteries, and the trips information.

2.1.1 Mathematical Formulation

The mathematical formulation of the cost objective function for determining the optimal actions is divided into two parts corresponding to two states, namely when theEVis plugged into a charging station (Sp) and when it is not (Su). When the vehicle is plugged in, the

cost function depends on the electricity price and the decision to charge or not, and can be described as:

Sp(t) = a(t) · Cel(t) ·

Ech(SoC(t))

η , (2.1)

where:

• a(t) is the decision variable on whether the vehicle is being charged (1) or on standby (0) at time t;

(37)

• Ech(SoC(t)) is the energy [kWh] supplied to the battery when charging is conducted

during a fixed time interval, with battery state of charge SoC(t) at the beginning of the period — as charging rate (in kW) varies considerably depending on the current battery charge level, we use the Algorithm 1 to calculate Ech, where Enom represents

the nominal full pack energy, Qnom the nominal pack capacity, tch the charge time, τ1

is the time of charge of the first phase, and IBi the battery current.

• η ∈ [0, 1] is the charger efficiency;

• SoC(t) is the state of charge (fraction of the nominal battery capacity) at time t. When the vehicle is not plugged in, there is no decision to make, but it is necessary to calculate the new SoC and the possible cost. The cost is null except when the battery is depleted, in which case the vehicle must fall back to the gas engine. So the following equation applies:

Su(t) = Cf uel(t) · max(Fc(SoC(t)), 0), (2.2)

where:

• Cfuel(t)is the gasoline price [$/l] at time t;

• Fc(SoC(t)) is the fuel consumed in liters by the vehicle once the battery is depleted.

Combining Eq. 2.1and Eq.2.2yields the global optimization objective:

min {a(t)}T t=1 T X t=1 [z(t) · Sp(t) + (1 − z(t)) · Su(t)] , (2.3)

where z(t) returns a Boolean value indicating whether the vehicle is plugged in (1) or not (0) to a charging station at the time interval t. This objective is subject to the following constraint:

Ech(SoC(t))

η ≤ Enom· (1 − SoC(t)). (2.4)

It indicates that the power used to charge anEV (Ech(SoC(t))/η) cannot exceed the current

capacity available in the battery (Enom(1 − SoC(t))).

The charging time (tch) that it would take to fully charge a vehicle from a SoC(t) can be

calculated with the Algorithm 2 which also uses the battery model proposed in AppendixA, where Vmaxrepresents maximum cell voltage, Vmin the minimum allowable cell voltage (cut-off

voltage), IBmax and IBmin the maximum and minimum current value respectively, and RB

describes the internal resistance of the battery.

Once the charging time (tch) is known, we can calculate the newSoC (SoC(t + ∆t)) and the

energy supplied to the battery Ech(SoC(t))[kW h] after charging during a time interval ∆t

(38)

Algorithm 1 Energy supplied to the battery E_ch[kW h]when charging during an interval of time ∆t

Require: E_nom, Q_nom, t_ch, τ₁, I_Bi and SoC(t) if τ1 <∆t then SoC(t + ∆t) = SoC(t) + IBi[τ1+ min(∆t− τ1, tch− τ1)) Qnom else SoC(t + ∆t) = SoC(t) + IBi· ∆t Qnom end if

Ech= Enom(SoC(t + ∆t) − SoC(t));

return SoC(t + ∆t), Ech

Algorithm 2 Time of charge of an EV

Require: Qnom, Vmax, Vmin, IBmax, IBmin, RB and SoC(t)

if SoC(t) < 1 − RB· IBmax Vmax− Vmin then IBi = IBmax τ1 = Qnom IBmax 1 − SoC(t) − RB· IBmax Vmax− Vmin else IBi =

(1 − SoC(t)) · (Vmax− Vmin)

RB

τ1 = 0

end if

τ2= −_VR_maxB·Q_−Vnom_min · ln

IBmin IBi tch= τ1+ τ2 return tch and IBi

It should be stressed that the optimization procedure proposed in this thesis is adaptable to any battery charging model. Any model can be used as long as it allows the computation of the newSoCresulting from charging for a given duration. In particular, the amount of energy stored in the batteries can vary according to theSoCat the beginning of the charging period (i.e., for a given charging session, less energy is stored for an initial highSoC compared to an initial low SoC).

In the case of Plug-in Hybrid EVs (PHEVs), the energy consumed may be greater than the energy available in the batteries given that the gas engine starts running when the batteries are depleted. In the case of a Battery Electric Vehicle (BEV), Cf uel(t)can be modified by adding

an arbitrary stronger penalty associated with running off energy. The energy consumed by a Plug-in Hybrid Vehicle (PHEV) during a trip depends on the capacity and the current SoCof the battery and other conditions that are more difficult to model such as the characteristics of the terrain, the driving technique, outdoor temperature, among others. In this thesis, we