HAL Id: tel-00758118
https://tel.archives-ouvertes.fr/tel-00758118
Submitted on 28 Nov 2012HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
multi-agent simulation.
Tomas Navarrete Gutierrez
To cite this version:
Tomas Navarrete Gutierrez. A control architecture for complex systems, based on multi-agent simu-lation.. Multiagent Systems [cs.MA]. Université de Lorraine, 2012. English. �tel-00758118�
D´epartement de formation doctorale en informatique Ecole doctorale IAEM Lorraine´ UFR STMIA
Une architecture de contrˆ
ole de
syst`
emes complexes bas´
ee sur la
simulation multi-agent.
TH`
ESE
pr´esent´ee et soutenue publiquement le
pour l’obtention du
Doctorat de l’Universit´
e de Lorraine
(sp´ecialit´e informatique)
par
Tom´
as Navarrete Guti´
errez
Composition du jury
Pr´esident : Sylvain Contassot-Vivier
Rapporteurs : Ren´e Mandiau Professeur, Universit´e de Valenciennes Michel Occello Professeur, Universit´e Grenoble 2
Examinateurs : Abderrafiˆaa Koukam Professeur, UT de Belfort-Montb´eliard Sylvain Contassot-Vivier Professeur, Universit´e de Lorraine
Laurent Ciarletta Maˆıtre de Conf´erences, Universit´e de Lorraine
Directeur de th`ese : Vincent Chevrier Maˆıtre de Conf´erences, HDR, Universit´e de Lorraine
Remerciements
Agradezco al Consejo Nacional de Ciencia y Tecnología de méxico por la beca de estudios que financió mi doctorado.
Au terme de ce travail, je tiens à exprimer ma profonde gratitude ainsi que mes sincères remer-ciements à Vincent Chevrier, pour l’encadrement de ce travail, ses conseils, confiance, et patience. Mes remerciements s’adressent également à Laurent Ciarletta, qui s’est montré à l’écoute et s’est impliqué profondément dans mon travail de thèse.
Je tiens également remercier sincèrement Messieurs René Mandiau et Michel Occello pour avoir accepté de rapporter mes travaux de thèse et par la même occasion à tous les membres du jury : Monsieur Abderrafiâa Koukam et Monsieur Sylvain Contassot-Vivier.
Je souhaite remercier très fort les précieux collègues doctorants de l’équipe MAIA qui m’ont rendus ces années très riches. Nous avons intégré une très bonne ambiance entre les doctorants ce qui m’a permit de faire ma thèse dans un cadre de travail très agréable. Arnaud, Jano, Sylvain, Olivier Bourre, Olivier Buffet, Boris : sachez que avoir partagé tant d’heures (au resto, à la cafet, tuant des zombies ou des ogres) fut un énorme plaisir pour moi. Je remercie également Alain et Vincent, avec qui j’ai eu le plaisir de partager un bureau et avec qui j’ai pu à plusieurs reprises discuter de tout et de rien.
En fin je voudrais remercier les membres de ma famille pour leur compréhension et leur soutien durant ces années. Une pensée très particulière pour Milo, qui a été mon moteur principal dans la dernière ligne droite de la thèse.
Je dédie cette thèse à Aurore.
Índice general
Introduction
Chapter 1
Complex Systems
1.1. Complex Systems Definition . . . 5
1.2. Relevant Characteristics . . . 7
1.3. Examples . . . 10
1.4. Challenges in the study of complex systems . . . 14
1.4.1. Modeling . . . 14
1.4.2. Engineering . . . 15
1.4.3. Control . . . 15
1.5. Difficulties of control of complex systems . . . 16
1.5.1. Local actions with global effects . . . 16
1.5.2. Entities autonomy . . . 16
1.5.3. Modeling . . . 17
1.5.4. Preexisting systems . . . 17
1.6. Governance and control of complex systems . . . 17
1.7. Synthesis . . . 18
Chapter 2 Related Work 2.1. Introduction . . . 21
2.2. Control Theory . . . 22
2.3. Equation Free approach . . . 24
2.4. Modeling complex systems . . . 26
2.4.1. Multi-agent paradigm . . . 26
2.4.2. Agent-Based Models . . . 27
2.5.1. Organic Computing . . . 31
2.5.2. Control of Self-Organizing systems . . . 33
2.5.3. Prosa . . . 34
2.5.4. PolyAgent . . . 36
2.5.5. Morphology . . . 37
2.5.6. Emergent Engineering . . . 39
2.5.7. Control of reactive multi-agent system with reinforcement learning tools . 40 2.6. Synthesis . . . 40 2.7. Conclusions . . . 41 Chapter 3 A control architecture 3.1. Introduction . . . 43 3.2. Principles . . . 44
3.3. Definition of the architecture . . . 45
3.3.1. Architecture elements . . . 45
3.3.2. Hypothesis of the architecture . . . 46
3.3.3. Concise definition . . . 47
3.3.4. Preliminary synthesis . . . 47
3.4. Detailed view . . . 48
3.4.1. Block Definition Diagram of the architecture . . . 49
3.4.2. Internal Block Diagram of the architecture . . . 51
3.4.3. Execution flow of the architecture . . . 51
3.4.4. Complementary view of the elements of the control architecture . . . 52
3.5. Assessment on the relevance of our proposition . . . 54
3.5.1. Architecture elements and difficulties of the control of complex systems challenge . . . 54
3.5.2. Equation free approach and multi-agent models in the architecture . . . . 55
3.5.3. Governance related aspects . . . 56
3.6. Conclusion . . . 57
Chapter 4 Proof of concept Implementation 4.1. Introduction . . . 60
4.2. Peer-to-Peer networks as complex systems . . . 60
4.3. Target system specification . . . 61
4.3.2. Target system state . . . 61
4.4. Target system implementation . . . 61
4.4.1. Internal behavior of the peers . . . 62
4.4.2. Number of peers in the network . . . 63
4.4.3. Network organization . . . 63
4.4.4. Initial proportion of sharing peers . . . 63
4.4.5. Update Scheme . . . 63
4.4.6. Summary . . . 63
4.5. Preliminary study on the behavior of the target system . . . 64
4.5.1. Analytical point of view . . . 64
4.5.2. Beyond the analytical model . . . 66
4.5.3. Results and conclusions on the preliminary study . . . 66
4.5.4. Retained Initial conditions . . . 66
4.5.5. Focus on the behavior of the target system associated to retained initial conditions . . . 68
4.6. Experimentation platform . . . 69
4.7. Experimental Setup . . . 70
4.7.1. Control Objective . . . 70
4.7.2. Architecture implementation overview . . . 70
4.7.3. Architecture implementation details . . . 71
4.8. Experiments and Results . . . 74
4.8.1. Description of experiments . . . 75
4.8.2. Synthesis of results . . . 76
4.8.3. Discussion on the results . . . 76
4.9. Discussion . . . 78
4.10. Conclusions . . . 79
Chapter 5 Multi-agent simulation questions inside the architecture 5.1. Introduction . . . 81
5.2. Initialization of models . . . 82
5.2.1. Target systems implementation . . . 82
5.2.2. Nominal Behavior . . . 83
5.2.3. Experimental setup . . . 86
5.2.4. Experiments and Results . . . 88
5.2.5. Discussion . . . 91
5.3. Model selection . . . 92
5.3.1. Target system implementation . . . 92
5.3.2. Experimental setup . . . 92
5.3.3. Experiments and Results . . . 93
5.3.4. Discussion . . . 93 5.4. Conclusions . . . 94 Chapter 6 Conclusion Bibliography 97 Annexe A Résumé étendu : Une architecture de contrôle de systèmes complexes basée sur la simulation multi-agent. A.1. Les systèmes complexes . . . 112
A.1.1. Caractéristiques de Systèmes Complexes . . . 112
A.1.2. Exemples de systèmes complexes . . . 113
A.1.3. Les challenges dans l’étude des systèmes complexes . . . 115
A.1.4. Gouvernance et contrôle de systèmes complexes . . . 116
A.1.5. Difficultés du contrôle de systèmes complexes . . . 116
A.2. Travaux connexes . . . 117
A.2.1. Modélisation des Systèmes complexes . . . 117
A.2.2. Le paradigme multi-agent et les systèmes complexes . . . 120
A.2.3. Théorie du contrôle . . . 120
A.2.4. Approche « equation-free » . . . 121
A.2.5. Applications du paradigme multi-agent au contrôle de systèmes complexes 122 A.2.6. Synthèse . . . 127
A.3. Proposition . . . 128
A.3.1. Principes . . . 128
A.3.2. Vue Générale . . . 129
A.3.3. Flux d’exécution de l’architecture . . . 130
A.3.4. Des principes à l’implémentation . . . 130
A.4. Preuve de concept . . . 133
A.4.1. Introduction . . . 133
A.4.2. Les réseaux pair-à-pair comme systèmes complexes . . . 133
A.4.4. Implémentation de l’architecture . . . 135
A.4.5. Expériences . . . 136
A.4.6. Résultats . . . 138
A.4.7. Discussion . . . 139
A.4.8. Conclusions . . . 140
A.5. Questions de la simulation multi-agent dans l’architecture . . . 140
A.5.1. Scénario expérimental . . . 140
A.5.2. Implémentation de l’architecture . . . 141
A.5.3. Expériences . . . 142
A.5.4. Résultats . . . 142
A.5.5. Discussion . . . 143
A.5.6. Conclusions . . . 143
A.5.7. Selection des modéles . . . 143
Introduction
Context
Within the last twenty years, we have observed that there are many systems, both natural and artificial, with a large number of participants that exhibit some characteristics at the group level that cannot be identified at the individual level. These are called “complex systems”. The characteristics of complex systems are: a large number of autonomous entities, sensitivity to initial conditions, different organization levels, dynamic structures, emergent properties and different time and space scales within the system.
Examples of systems called complex usually cited in the literature are: the omnipresent inter-net, groups of insects, the economy, the human brain, electricity distribution networks, transport networks, etc. A new kind of science has emerged to deal with such systems. This new kind of science, known as complexity science is an interdisciplinary framework harnessed by the advances in different scientific fields such as chaos theory, sociology, nonlinear dynamics, biology, ecology, statistical mechanics, graph theory, thermodynamics, probability theory, numerical simulation and others (Bar-Yam, 1997).
One sign of the importance of this newborn science is the creation of research institutions dedicated to the study of complex systems. Some of these institutions, like the Santa Fe Institue and the NECSI in the United States, the ISCV in Chile, the ISCPIF and the IXXI in France, are not directly dependent on research programs of a specific university. Others, are departments within a university like The Department of Complexity Science and Engineering in the university of Tokyo in Japan or C3 in the UNAM in Mexico. Finally, some of them are “virtual” networks of researchers interested in complex systems, as the RNSC or the CSS and the CSIRO. Another sign of the importance of complexity science is the wide spectrum of contexts where complexity has been studied as a main concern (Nature insight, 2001; La Complexité, 2003). This spec-trum ranges from nature, through human activities and up to artificial or man-made systems. Within the natural context (Camazine et al., 2001; Bak and Bak, 1996), there are physical pro-cesses (Nicolis and Prigogine, 1977), genetics (Kauffman, 1993), natural disasters (Ball, 2004), epidemics (Bolker and Grenfell, 1993), and ant foraging (Nicolis and Deneubourg, 1999). In the human activities context (Castellani and Hafferty, 2009) there are economy (Anderson et al., 1988) and management (Van Eijnatten, 2005). In the artificial or man-made context there are cellular automata (Wolfram, 1994; Langton, 1986) and communication networks (Kocarev and Vattay, 2005).
Yet, there is no formal, “one simple statement”, commonly accepted definition of a complex system (Mitchell, 2009; Grobbelaar and Ulieru, 2007; Nicolis and Nicolis, 2007; Boccara, 2004). Defining complexity or complex systems is by itself a challenge, as Horgan (1995) already noticed. Lloyd (2001) has found no less than forty-five different ways to measure (and thus define) complexity. Definitions in literature are as varied as the contexts within which they have been formulated. For Edmonds (1999) for example, complexity is
“that property of a language expression which makes it difficult to formulate its overall behaviour, even when given almost complete information about its atomic components and their interrelations.”
Given our interest in the multi-agent modeling within this thesis work, we could have been compelled to work with the definition given in the preface of (Boccara, 2004):
“Although there is no universally accepted definition of a complex system, most researchers would describe as complex a system of connected agents that exhibits an emergent global behavior not imposed by a central controller, but resulting from the interactions between the agents. These agents may be insects, birds, people, or companies, and their number may range from a hundred to a million.”
We agree with (Ladyman et al., 2012), that the problem of defining complexity science and even complex systems may benefit from the work of the philosophy of science. This is an objective out of the scope of this thesis. We shall focus in this work not on the definition of a complex system but rather on the characteristics of complex systems. In this way, we expect our work to be applicable to systems sharing characteristics with those of complex systems may they be natural or artificial, man-made or man-centered. Our work takes place within the context of complex systems to build up from preexisting work and not be forced entirely to create the context. We shall focus on one particular challenge that arises when systems exhibit characteristics inherent to complex systems: “control”.
Control challenge
We have observed that there are complex systems all around us but also, we have also noticed that given some of their characteristics, their behavior is not always the one sought or desired. Undesired phenomena within them, have been observed : energy outages (Newman et al., 2011; Pourbeik et al., 2006), internet outages (Dainotti et al., 2011; Smith, 2011; Paxson, 2006), recurring financial crises (Fender et al., 2012; Vivier-Lirimont, 2008; Llaudes et al., 2010; Artus and Lecointe, 1991), transport networks traffic bottlenecks (Lee et al., 2011; Xu et al., 2011; Kerner and Klenov, 2009).
In this thesis work, we shall focus on the challenge of driving a complex system to exhibit a certain behavior. Control theory assumes that given a model, an optimal way to control a target system exists. This optimization is based on using analytical models. However, we consider that such models are rarely useful in the context of complex systems. It has been recognized that given the characteristics of complex systems the concept of governance is better suited (Chavalarias et al., 2009) than control. Multi-agent modeling and simulation have been identified as suitable tools for complex systems (Phan and Amblard, 2007a). Based on these elements we will tackle the challenge of control of complex systems.
We propose an architecture to control a complex system based on an equation-free approach with multi-agent model simulation. In our proposition, we will integrate different existing ele-ments such as the equation-free approach, the multi-agent paradigm and the basic concepts of control theory. The objective of this thesis work is to evaluate the implications of integrating all these elements in one coherent architecture. The evaluation takes place at two levels: find out if the proposition is feasible and know how to compare the answers given to the questions implied in using the multi-agent paradigm in our context.
Thesis plan
Chapter 1. Presents our working definition of complex systems and explains characteristics of complex systems relevant to our work. In this chapter we describe the current challenges of the study of complex systems such as engineering and design, modeling and control. We elaborate on the specific difficulties of the control challenge.
Chapter 2. Different approaches existing in the literature to deal with the control of complex systems are presented.
Chapter 3. Our proposition on a control architecture to govern a complex system based on an equation-free approach with multi-agent model simulation is detailed in this chapter. We state the different hypothesis under which our architecture is conceived and the principles used to design it. The architecture is initially presented from a coarse point of view to establish the links between the difficulties of the control challenge previously defined and the elements of the architecture that specifically deal with them. Then, we present a more detailed description that allows to identify different decisions that must be made when implementing our architecture. We pay special attention to the questions implied by using the multi-agent paradigm in our proposition, from an abstract point of view.
Chapter 4. A first implementation of our architecture made within the context of the free-riding phenomenon in peer-to-peer file sharing networks is presented in this chapter. We define an experimental environment under which we drive a complex system to exhibit a desired behavior through the usage of our architecture.
Chapter 5. We show within the same experimental context as the previous chapter, different answers given to questions of validation, calibration and translation of multi-agent models in our architecture.
Chapter 6. We present a synthesis of our findings and identify different future directions and development of our work.
Chapter 1
Complex Systems
Contents
1.1. Complex Systems Definition . . . 5
1.2. Relevant Characteristics . . . 7
1.3. Examples . . . 10
1.4. Challenges in the study of complex systems . . . 14
1.4.1. Modeling . . . 14
1.4.2. Engineering . . . 15
1.4.3. Control . . . 15
1.5. Difficulties of control of complex systems . . . 16
1.5.1. Local actions with global effects . . . 16
1.5.2. Entities autonomy . . . 16
1.5.3. Modeling . . . 17
1.5.4. Preexisting systems . . . 17
1.6. Governance and control of complex systems . . . 17
1.7. Synthesis . . . 18
1.1.
Complex Systems Definition
In this thesis work we have decided to work with the definitions of complexity subjects (complex systems and their challenges) resulting of a collaborative effort, namely the “French Complex Systems Roadmap” of the French National Network for Complex Systems1(Chavalarias et al., 2009). However, this is not the only collective effort: (Rouse, 2007) for example, is the work of a group of thought leaders in the United States. One important difference between the two approaches, despite being collaborative efforts, is in the applications they keep as goals of their research plans. In the work of (Rouse, 2007) health-care, infrastructure, environment, security, and competitiveness are the key objectives of their research goals. Whereas in (Chavalarias et al., 2009) the objective of the collaborative work is to establish directions in research regarding a wide landscape of scientific disciplines (and at the same time, one science of complexity). Both collective works agree in aspects such as the importance of modeling and design of complex
1
As can be seen in the website of the complex systems registry (CSS, 2012a), the French roadmap is also used as the basis for a European roadmap within the context of a European community research program.
systems as well as the phenomena of interest to study such as time and space inter-dependencies and different time and space scales in complex systems. We have selected (Chavalarias et al., 2009) as our reference framework because it directly addresses the challenge of control of complex systems.
Our working definition of a complex system is thus as follows.
Definition 1 A “complex system” is in general any system comprised of a great number of het-erogeneous entities, among which local interactions create multiple levels of collective structure and organization. (Chavalarias et al., 2009)
This definition of a complex system focuses on three “core concepts”, all of them at different levels of the system. We define the different description levels of a complex system as follows.
The core concepts included in the definition are:
The characteristics of the entities belonging to the system: great number and heteroge-neous.
The mechanisms responsible for the global behavior of the system: interactions among entities.
The structure of the global behavior of the system: multiple levels of organization and structure.
The core concepts and the different description levels defined above are related in the following ways. The first concept regards the microscopic or local level. It tells us what to look for when looking at the entities of a complex system. The second concept regards both the macroscopic or global level and the microscopic or local level. It tells us where to look for when looking at mechanisms producing the behavior of a complex system. At the global level, we have the behavior of the complex system. The mechanisms responsible for the global behavior are at the local level because they are rooted in the interactions of the entities. The third concept is completely at the global level. It tells us what to look for when looking at the structure and organization of the system.
We have also chosen this definition because it is the result of the joint work of a large group of researchers of major French research institutions (of different scientific fields), and not only authors of a book or members of a research institute or university department. Since this defini-tion is the result of a collective effort, it has the advantage of being domain independent. From this point of view, we consider it useful for different scientific fields, like sociology, informatics or physics because it is stated at a relatively abstract level. It does not say for example that the constituent parts of a complex system are molecules, robots, customers of a cell-phone company or anything specific to a particular scientific field. Also, there is no specification of how many are “a great number” of entities, or how is the heterogeneity of the entities to be measured. This level of abstraction is necessary to study systems that share some characteristics but that do not belong to the same scientific field.
Although there is not one single definition of complex systems, many authors agree on com-mon characteristics shared by complex systems. A quick look at the literature on complex systems shows that the characteristics of complex systems are not limited to the three core con-cepts from our definition, see for example Sitte (2009). We shall focus on some the characteristics of complex systems relevant to the difficulty of their study (Amaral and Ottino, 2004).
1.2. Relevant Characteristics
1.2.
Relevant Characteristics
The following is a set of characteristics that we consider relevant to the difficulties of studying complex systems. The set of characteristics is not explicitly given within the cited framework.
Emergence
Its definition is usually directly linked to Aristotelian principle of “the whole is more than the sum of the parts”. This means that the behavior exhibited by the whole system cannot be solely inferred from the behavior of its constituent elements. Emergence is one of the most controversial characteristics of complex systems (Fromm, 2004). The center of the controversy is the almost “magical” sudden appearance of a property of a system. The controversy sometimes takes the form of defining if emergence is only in the eye of the observer. Despite that such considerations are close to the study objects of philosophy, we agree with (Brückner, 2000) who points out that a general theory of this concept has not yet been found.
In our context, emergence is the implicit idea behind the second core concept of our definition of a complex system. An emergent property of a complex system is one that is observed at a level different from the local (global or any intermediary one) but that is not directly given at the local level.
Definition 2 A phenomenon is emergent if and only if we have:
a system of entities in interaction whose expression of the states and dynamics is made in an ontology or theory D;
the production of a phenomenon, which could be a process, a stable state, or an invariant, which is necessarily global regarding the system of entities;
the interpretation of this global phenomenon either by an observer or by the entities them-selves via an inscription mechanism in another ontology or theory D0.
(Müller, 2003)
Another concept related to emergence present in the French roadmap of complex systems (Chavalarias et al., 2009), but not explicitly given in the definition of complex system is immer-gence. Emergence can be seen as the influence of local interactions on the global outcome, but it is not the only influence originating at one level and having consequences at another level. There also is influence from the global outcome of a complex system executed at local level. This influence is known as immergence. Immergence is the process by which the global behavior of the system influences the way interactions are held among entities in a feedback loop.
The feedback loop means that the output of a system will be the input of the same system or process. In the complex systems context this means that the interactions among entities will eventually take as input the global behavior of the system, which at its turn is the output of the interactions among entities.
Different levels of description
At least two different levels to describe a complex system can be considered: the local or microscopic and the global or macroscopic. The description level of a complex system is said to be local or microscopic when it exclusively employs characteristics inherent to the entities of the
system. The description level of a complex system is said to be global or macroscopic when it employs characteristics of groups of entities of the system.
At the local level, we can define each of the elements of the system, as well as the way they interact with other elements. At the global level, we can characterize the system as a whole.
Additionally, if the elements of the system can be organized in hierarchies or groups, other intermediate description levels are possible.
Scale
Complex systems are made of entities that can be described at their local level. The entities interact with each other, at their own (local) time and space scale. However, when the system is observed at a “macroscopic” or global level, the view changes. The time and space scales at which the whole system evolves (that is, at global level) are orders of magnitude away from the local scales.
Sensitivity to initial conditions
Very small differences in the initial conditions of a complex system produce significantly different outputs at quantitative and qualitative level. A usual example of a system highly sensitive to initial is the chaotic Lorenz system. It is made of the ordinary differential equations:
˙ x(t) = σ(y − x) ˙ y(t) = x(ρ − z) − y ˙ z(t) = xy − βz
When ρ = 28, σ = 10, β = 83 the system has chaotic solutions. The evolution of the x variable with different initial conditions is illustrated in figures 1.1 and 1.2.
-20 -15 -10 -5 0 5 10 15 20 25 0 10 20 30 40 50 x t
Evolution of x in the Lorenz equations with σ=10, β=8/3 and ρ=28 initial conditions
x=0,y=1,z=1.05 x=0.00001,y=1,z=1.05
Figure 1.1: The evolution of x in the Lorenz equations, with initial conditions differing only in the x variable by 1 × 10−05.
Nonlinear dynamics
The principle of superposition is the basis to differentiate between linear and nonlinear dy-namics of a system. The principle states that, in a linear system, the net response at a given
1.2. Relevant Characteristics -15 -10 -5 0 5 10 15 20 25 -20 -15 -10 -5 0 5 10 15 20 25 30 0 10 20 30 40 50 60 z initial conditions x=0,y=1,z=1.05 x=0.00001,y=1,z=1.05 x y z (a) -15 -10 -5 0 5 10 15 20 25 -20 -15 -10 -5 0 5 10 15 20 25 30 0 10 20 30 40 50 60 z initial conditions x=0,y=1,z=1.05 x=0.00001,y=1,z=1.05 x y z (b) -20 -15 -10 -5 0 5 10 15 20 25 -30 -20 -10 0 10 20 30 0 10 20 30 40 50 60 z initial conditions x=0,y=1,z=1.05 x=0.00001,y=1,z=1.05 x y z (c)
Figure 1.2: The evolution of Lorenz system of equations with two initial conditions. In 1.2a the system follows the same trajectory, after some time (400 more iterations of the system) we see in 1.2b that the trajectories begin to take different directions and, in 1.2c, we see that after 2400 iterations the trajectories are completely different.
place and time caused by two or more stimuli is the sum of the responses which would have been caused by each stimulus individually. In other words, the sum of the parts equals the whole. A nonlinear system is one whose output is not directly proportional to its input. In other words, the dynamics cannot be expressed as a sum of the behaviors of its parts.
In complex systems the sensitivity to initial conditions plus the nonlinear dynamics means that a small perturbation may cause a big effect, or even no effect at all. In linear systems, effect is always directly proportional to cause.
Dynamic structures
The structures in a complex system are the product of interactions between entities of the system. Like the interactions, the structures in the complex system evolve over time.
The object of study of this work are complex systems. Earlier we have presented our working definition of complex systems as well as some characteristics relevant to the study of complex systems. However the characteristics are not necessarily part of the original context of our definition. Furthermore, the set of characteristics we presented is voluntarily not exhaustive. We take a look now at the relationship between the core concepts of our complex system definition and the relevant characteristics we are interested in as well as other characteristics found in the literature.
Definition core ideas and characteristics
Our work is based on a definition of a complex system which is given in terms of three core concepts. Some characteristics of complex systems can be directly linked to such core concepts, like the different levels of description and emergence. However, other characteristics like dynamic structures and organizations at the different time and space scales are only indirectly linked to the three core concepts. The importance of the characteristics we focused on, regarding the control challenge, is what guided us in selecting the list of characteristics. This importance shall be highlighted through this chapter. In figure 1.3 we illustrate the previous links between the three core concepts of the definition of a complex system with the characteristics of complex systems.
Figure 1.3: Relationships between core concepts of complex systems definition used in this thesis and the characteristics of complex systems
So far we have given characteristics of complex systems, from an abstract point of view. In the following section, we provide examples of complex systems in different contexts to identify what the abstract definitions refer to, in concrete cases. Two of the examples are present in nature and the other two are artificial, or man-made.
1.3.
Examples
Natural
Ant foraging
Social insects like ants, bees or termites are known for accomplishing collective tasks such as nest building, defense and foraging. If we consider an ant colony as a complex system, the entities
1.3. Examples
composing it are the ants. Ants in a colony organize to find food and bring it to the colony. At the local level, ants are rather simple organisms and at least their communication means are limited. However, at the global level, an ant colony is capable of collectively producing emergent phenomena. An example of an emergent behavior in ant colonies is the foraging phenomenon (Deneubourg et al., 1990). Experiments have allowed to observe how by the usage of pheromones, ants communicate the position of a food source. The initial experimental protocol consists in establishing two different “bridges” from the nest to a food source. The results of the experiments show that eventually, the ant colony collectively chooses only one of the two paths to the food source.
The Brain
One way to conceive the brain as a complex system is by considering that its entities are the neurons. There is a large number of neurons (and other kinds of cells) in a human brain. At the local level of description, neurons are the basic elements of the brain. Additionally, they can be subdivided into compartments, synapses, channels, molecules and so forth, with relevant physiological and dynamical properties on all levels (Olbrich et al., 2011a). At the global level, the brain is capable of producing emergent behavior. Different time scales exist in the brain spanning several orders of magnitude: synaptic processes may occur within sub-milliseconds to milliseconds whereas cognitive processes occur in fractions of seconds to perhaps hours, and memory phenomena last for seconds to years. It can be assumed that processes inside the brain are not isolated acts but instead they are embedded into a network of intricate mutual dependencies across time (Olbrich et al., 2011a). However, it is still not understood what neural signals mean or how they give rise to global cognitive behaviors. And all this despite the vast knowledge on the structure of neurons and their interactions (at the chemical level) with other neurons. Most neural dynamics models are basically nonlinear because of voltage-dependent ion channels, firing thresholds, nonlinearities in synaptic transmission and other mechanisms. Some models such as rate-based ones, describe neural behavior in terms of average rates of action only. Within these models, the firing rate of a neuron is given by a sigmoidal function of the membrane potential. Successful usage of these models depends on overcoming a serious obstacle. Because of the large number of neurons in the brain, neural dynamics happens in a very high-dimensional state space, while the tools of nonlinear dynamics are easily applicable only to low-dimensional or moderately high-dimensional systems (Olbrich et al., 2011b).
Artificial
Internet
Internet is one of the biggest man-made techno-social system. “Modern techno-social systems consist of large-scale physical infrastructures (such as transportation systems and power distri-bution grids) embedded in a dense web of communication and computing infrastructures whose dynamics and evolution are defined and driven by human behavior” (Vespignani, 2009).
Internet is made of a very large number of heterogeneous entities: computers, servers, mobile phones, internet things (home appliances, cars) . . . (Willinger et al., 2002; Park, 2005). As a complex system, we could consider that at the local level we have every device which is capable of communicating with any other device in the internet. However, a more accurate description would involve several levels of organization among the different entities: routers, autonomous systems, local area networks, wide area networks, tier “x” level networks, etc.
Internet as a complex system has different space levels. Internet allows communication from one place on the world, to virtually everywhere else (provided internet connection exists). But, also, communication is possible between entities connected to the network that are just next to each other. Communication can take place at hundreds of thousands of kilometers of distance, or at mere some meters. The current speeds of communication in internet allows to communi-cate almost instantaneously. Posting a comment on a website can take milliseconds or seconds before it is publicly available all over the world, however the time necessary before other entities acknowledge this novelty can take minutes or even days.
Cellular Automata
A cellular automaton is a collection of “colored” cells on a grid of specified shape that evolves through a number of discrete time steps according to a set of rules based on the states of neighboring cells. The rules are then applied iteratively for as many time steps as desired (Weisstein, 2012).
If we consider that a cell is an entity, we can associate as follows the characteristics of a cellular automaton to those of complex systems. The set of rules can be seen as the interactions between entities. Cells change their color based on the interactions with other neighboring cells. No specific cell dictates how all the other cells will behave, which means that the entities are autonomous. The rules are given in terms of the entities, and thus the global behavior of the whole automaton cannot be deduced directly from the entities. A rule for a cell can be stated as: “for current time step, if all neighbors have color black, at next time step, the current cell will have color black”. Elemental cellular automata are “lines” of cells, where each cell only has two neighbors. The boundary conditions of the line are “toric”, meaning that the first cell has as neighbors the second cell and the last cell. A Wolfram rule specifies the next color in a cell, based on its color and the color of its immediate neighbors. Given the boundary conditions of the line of cells, each cell only has two neighbors. A neighborhood is made of a cell and its two neighbors. There are thus 23 = 8 possible configurations for a cell. We can consider each of the possible states as a three bit word, where a bit in 1 means the cell is full (colored in blue for example) and a 0 bit means that the cell is empty (or colored in white). State 010 corresponds to the following neighborhood of cells: .
The outcomes of the rule are encoded in binary. For example, for rule 30 the outcomes are 00011110. An outcome of 1 means that at the next time step the cell will be colored, and an outcome of 0 means that at the next time step the cell will be white. Each binary digit of the outcome corresponds to a neighborhood configuration. We can label the bits in the rule from right to left (from least significant to most significant) as outcome 0, 1, 2, 3, 4, 5, 6, 7 and then associate each label with a neighborhood. The decimal representation of the neighborhood configuration corresponds to a label of outcome. For example, with rule 30 = 00011110, neighborhood with binary representation 010 and decimal representation 2 will be associated to outcome with label 2 which is 1 (meaning a colored cell). The whole list of neighborhood configurations and outcomes for rule 30 are:
Neighborhood configuration 7 6 5 4 3 2 1 0
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
Next color of cell
Outcome 0 0 0 1 1 1 1 0
1.3. Examples
Figure 1.4 illustrates the evolution of cellular automaton with Wolfram rule 30 after 15 time steps.
Figure 1.4: Evolution after 15 steps of Wolfram rule 30 with a single colored cell (Wolfram, 2002, page 55). The initial state is at the top, the final state is at the bottom.
In cellular automata we observe the sensitivity to initial conditions by simply changing the amount of initially colored cells in the same automaton or by changing the spatial distribution of the initially colored cells. Figure 1.5 illustrates this with the cellular automaton of Wolfram rule 30.
(a) Same number of initially colored cells as figure 1.4, but different spatial distribution.
(b) Only two colored cells in initial condition.
(c) Same number of initially colored cells as figure 1.5b but with different spatial distribution.
Figure 1.5: Illustration of the sensitivity to initial conditions of Wolfram rule 30. The initial state is at the top, the final state is at the bottom.
There is no clear way to identify the time and space scales of a cellular automaton. Under a computer simulation, it takes from just a few time steps up to hundreds to observe patterns that can be interesting at the global level.
Figure 1.6: The elemental cellular automaton with Wolfram rule 55 after 29 time steps. The initial state is at the bottom, the final at the top. Obtained with FiatLux (Fates, 2012)
Interactions usually consist of observing the color of neighboring cells and then change its own color. If a cellular automaton is simulated with a computer we can assume that the interaction takes very little calculation time. A generation in a cellular automaton is when all cells in the automaton have changed their color.
(Hanson, 2009) portrays the presence of emergence in cellular automata as follows:
In cellular automata, the system’s state consists of an N -dimensional array of discrete cells that take on discrete values and the dynamics is given by a discrete time update rule. The “phenomena” that emerge in CA therefore necessarily consist of spatio-temporal patterns and/or statistical regularities in the cell values. [. . . ] Possibly the simplest type of emergent phenomenon in CA is synchronization, which is the growth of spatial regions in which all cells have the same value. A synchronized region remains synchronized over time (except possibly at its borders) and it may either temporally invariant (i.e., the cell values to not change in time) or periodic (the cells all cycle together through the same temporal sequence of values).
If we simulate the elemental cellular automaton with Wolfram rule 55, we can see an example of synchronization, and thus emergence as previously stated. This is illustrated in figure 1.6, where synchronized regions with temporal period 2 emerge from random initial conditions.
1.4.
Challenges in the study of complex systems
Based on our reference framework (Chavalarias et al., 2009), we present below some of the current challenges of the study of complex systems.
1.4.1. Modeling
A model is any physical, mathematical, or logical representation of a system, entity, phe-nomenon or process. In the scientific context, a model serves a specific purpose: a model exists as long as we can answer to certain questions regarding the modeled object by observation and manipulation of the model (Minsky, 1965). Models are the scientific tools used when it is ei-ther impossible or impractical to create experimental conditions where outcomes can be directly measured.
1.4. Challenges in the study of complex systems
There are three elements to the modeling challenge of complex systems: the nonlinear nature of their dynamics, the different levels of description of complex systems and the availability of big amounts of information observed from complex systems.
Analytical models of complex systems have a very limited usefulness. That is, although it is possible to create an analytical model of a complex system, it will most often not be solvable either because of the nonlinearity or because of the high number of variables.
Complex systems have multiple levels of description, and models should take into considera-tion these levels. This is not feasible with analytical models because they only describe systems at the global aggregated level dynamics.
Nowadays, technological advance allows to have abundant information at very fine levels of complex systems. It is a challenge to find how to use this information to create models that can accurately represent the system.
1.4.2. Engineering
Human-made complex systems are designed to serve a purpose, to exhibit certain behavior. In other words, they are meant to work in a certain specific way. Classical engineering reaches its limits when dealing with complex systems (Gershenson, 2007).
A first difficulty of the engineering complex systems has to do with conciliating two seemingly contradicting aspects. One might want to build a complex system whose global behavior should exclusively emerge from local interactions, without directly inscribing in the entities what the global behavior should be. The question here is to find out what local interactions will produce the global output. That is, the global behavior is identified (we know what we want it to be) but the local interactions leading to it are not. Put in other words, the question is obtaining the desired output, despite the fact that the way the system arrived to that output that can not be known, is too complex or not computationally reproducible (Buchli and Santini, 2005). Another aspect of this difficulty is whether it is an acceptable assumption for all application domains to not know the mechanisms that lead a system to exhibit a certain behavior or produce a certain output.
Another difficulty is in building complex systems on top of other complex systems. Techno-logical development has allowed to build systems that we consider as complex today, like internet. More and more systems are built on top of already deployed complex systems.
Finally, when engineering any kind of system, engineers are confronted to designing control mechanisms to make sure the system behaves as desired and as we shall see in the following section, control of complex systems is already a challenge by itself.
1.4.3. Control
To control an object means to influence its behavior so as to achieve a desired goal (Sontag, 1998). All along this thesis we shall call a “target system” a system that we wish to control. Traditional control theory assumes that a model of the target system is available and that it is analytically (or numerically) possible to use it. Control implies having a model and, as stated before, modeling complex systems is already a challenge by itself. If we assume, for example, that one way to “modify” the global behavior of a complex system is by influencing at the local level (because local interactions produce the global outcome), we need a model including those two different description levels in order to evaluate the effects of control actions as well as the evolution of the target system.
Moreover, given that complex systems are made of autonomous entities with different levels of description, the question of how to observe the system becomes a supplementary complication to achieve control.
1.5.
Difficulties of control of complex systems
Certain characteristics of complex systems make their study particularly challenging (Amaral and Ottino, 2004; Helbing, 2007; Rouse, 2003; Bar-Yam, 2003). The control of complex systems is one of the current challenges of complex systems, as we stated in 1.4.3. We present in this section the characteristics of complex systems that we consider to be related to the challenge of control of complex systems. They are: the global effects of local actions, the autonomy of the entities of a complex system, modeling and, finally, dealing with preexisting systems.
1.5.1. Local actions with global effects
One of the three core ideas of our definition of complex system is that the structures and organization arises from the interactions between the system elements. These interactions happen at the local level, because they are held between the system entities. However, as emergent phenomena arise in complex systems, we can see that the repercussions of local actions (possibly control actions) will be seen at a global level. To clearly identify the nature of the influence is complicated because of thec sensitivity to initial conditions, and because of the nonlinear dynamics of complex systems. Thus any control action should be carefully designed with this in mind. However, due to the difficulty of modeling complex systems by using models considering at the same time the local and global level of description of the system, this is not a trivial task.
The difficulty in the context of complex systems control is that if we assume that control actions shall be applied at a given level (local for example), the repercussions will be observed at another level (global for example).
1.5.2. Entities autonomy
In some complex systems, it is not possible to tamper with the entities autonomy. Because of legal, ethical and technical reasons, it is impossible to directly modify the behavior of entities. For instance, it is not possible to consider a control action in a target system where changes to the internal working of the entities would mean to substantially alter them. We think for example of mechanical or electronic entities. And even if we could overcome the technical reasons, we would still need to have the “blueprints” or a good model of the entity, in order to efficiently modify its inner workings.
On the other hand, we can consider that it is eventually possible to change the inner workings of “inanimate” entities in a target system.
Additionally, autonomy of participants means that as time goes by, because of immergence and emergence, and also because no central entity in the system dictates how other entities should behave, the individual behavior may change, leading to changes at all levels. This is also a problem, if the new behavior is not considered in the “model” used in the control mechanisms of the system.
The difficulty posed by the participants autonomy is that it is not always possible to consider direct modification of the inner behavior of the entities and that the autonomy may lead the system to evolve to not previously contemplated conditions.
1.6. Governance and control of complex systems
1.5.3. Modeling
We have acknowledged that complex systems include different description levels, at least two: local and global. Also we said that they exhibit nonlinear dynamics and usually are sensitive to initial conditions. Analytical models taking into consideration these two aspects, rapidly grow too difficult to be solved. Mathematical models rely on the identification of the key system components, often representing them in a discrete manner. This limits mathematical models because emergence present in complex systems arises as a consequence of local interactions, and cannot be previously identified as a key system component. Mathematical models are analogues, but cannot provide significant insight into the continuous internal process of a complex system (Polack et al., 2008). Moreover, they only take into consideration the global point of view, usually not explaining the reasons locally leading to the global behavior of the modeled system (Edmonds and Bryson, 2004).
The difficulty is to create a model that takes into account the multiple levels of a complex system. In the control context, a model of a complex system must take into account multiple levels because control actions taken at a level will have an influence at other levels. On the one hand, if we take control actions at the local level, by emergence, the global behavior of the system will be affected. On the other hand, if we take control actions at the global level, by immergence, the local level interactions will be affected, closing a feedback loop.
1.5.4. Preexisting systems
Systems may exhibit an undesired behavior because their internal control mechanisms have become useless. This can occur when their control mechanisms were not built to consider the conditions that provoked the change in the behavior. These conditions include: open system with new entities or environmental changes, the evolution of the behavior of the autonomous entities in the system, and the addition of whole new systems running on top of the preexisting system.
If the internal control mechanisms are no longer useful, they must be modified. This modifi-cation may imply for some systems, stopping the target system. Take for example the internet. It is technically possible to modify the behavior of a router in an autonomous system, but it would definitely be disastrous to modify all the routers at the same time (not to speak to the technical limits given by the fact that not all routers in a network belong to the same stake holders).
The difficulty of preexisting systems in the context of control of complex systems is that when the control mechanisms built with the system are no longer useful, it is not always possible, on the one hand, to stop the system to change the control mechanisms and on the other hand, it is not always possible to change the entities inner behavior.
Control theory assumes that given a model, an optimal way to control a target system exists. It has been recognized that given the characteristics of complex systems, the concept of governance is better suited (Chavalarias et al., 2009) than control. In the following section we take a closer look at the notion of governance and how it applies to complex systems.
1.6.
Governance and control of complex systems
‘governance’ is now often used to indicate a new mode of governing that is distinct from the hierarchical control model, a more co-operative mode where state and non-state actors participate in mixed public/private networks. (Mayntz, 2003)
Within the social sciences context, governance theory seeks to improve the social order in a new way, by considering societies as systems with hierarchical governments to steer them.
In our reference framework, governance is preferred over control because some of the assump-tions of control theory do not hold within the complex systems context. Complex systems have several different scales, they usually have multiple dimensions and they imply the presence of many heterogeneous points of view. One first assumption of control theory that does not hold within the context of complex systems is that it is not a trivial task to obtain an analytical model that can deal with ease with all these aspects.
A second assumption is optimality. Within the complex systems context it should be consid-ered possible to drive the behavior of the system but not always in the optimal conditions (not always maximizing a function). We cannot obtain an optimal value as a reference if we cannot have a model that produces it. In more mathematical terms, if we cannot have a model yielding a function to maximize, we cannot have optimal control.
1.7.
Synthesis
The challenge of control of complex systems, as we stated it in 1.5 is directly related to some characteristics of complex systems.
The local interactions produce the global outcomes of the system
They are made of decentralized systems made of autonomous entities
They are not easily (or at least usefully) modeled by analytical models
Preexisting complex systems may not be legally, or technically stopped or tampered with in order to control them
We consider that the control of complex systems is defined by overcoming a series of difficulties given by the characteristics of complex systems.
The modeling challenge is of capital importance by itself and also because it is directly related to the engineering challenge. In the control challenge, modeling is one difficulty but the local actions with global effects difficulty is closely related to the difficulty of modeling.
From the intuitive definition of control of section 1.4.3, we deduce that it is necessary to have a model to identify the behavior that we wish to observe or to avoid. One major difficulty that any control mechanisms of a complex system faces is modeling the evolution of the system behavior. Overcoming this difficulty means to characterize the evolution of the behavior of the target system, taking into consideration:
The different levels of a complex system (local and global for example).
The emergence of global outcome from local interactions.
A second difficulty that control mechanisms for complex systems face are preexisting systems. On one hand, overcoming this difficulty means to identify ways to control a system where the autonomy of the entities must be respected (because of legal, ethical or technical reasons). On
1.7. Synthesis
the other hand, it means to identify effective control actions in contrast to those already built in the target system that may be failing.
A third difficulty is to identify control actions that take into consideration emergence. Over-coming this difficulty means to identify a model where control actions are included. Modeling the behavior is not trivial and modeling the behavior when control actions are applied, is not trivial either.
New ways to find solutions to the control problem that do not intend to be optimal must be found. Specially, a better suited term to denote the “lead of a complex system to a desired state” that does not include optimality, is necessary. The term “governance” allows to relax the notion of optimality implicit in control.
Additionally, it is an hypothesis also taken as basis for work in research projects such as the French thematic network on the governance of complex systems (rncsgouv2012, 2012). Our selection of governance over control is beyond the semantic context. We adhere to the ideas behind the concept of governance: find a new way to control a complex system, without assuming that optimal control is possible.
Although we shall continue to refer to the “control” challenge, our work takes place under the same hypothesis of governance: optimal control for a complex system is not possible. The main reason behind this hypothesis is that optimal control implies the presence of an analytical model that allows to obtain a function to be maximized (to obtain an optimal value). And as we have already stated, this kind of models are hardly useful, because they do not take into consideration the different levels of organization of complex systems.
In the following chapter, we will take a look at works that have been confronted to the difficulties of the control challenge.
Chapter 2
Related Work
Contents2.1. Introduction . . . 21 2.2. Control Theory . . . 22 2.3. Equation Free approach . . . 24 2.4. Modeling complex systems . . . 26 2.4.1. Multi-agent paradigm . . . 26 2.4.2. Agent-Based Models . . . 27 2.5. Applications of multi-agent paradigm in the control of complex
systems . . . 31 2.5.1. Organic Computing . . . 31 2.5.2. Control of Self-Organizing systems . . . 33 2.5.3. Prosa . . . 34 2.5.4. PolyAgent . . . 36 2.5.5. Morphology . . . 37 2.5.6. Emergent Engineering . . . 39 2.5.7. Control of reactive multi-agent system with reinforcement learning tools 40 2.6. Synthesis . . . 40 2.7. Conclusions . . . 41
2.1.
Introduction
We present different works that have been confronted to the control of complex systems. That is why we present at first place the framework of control theory. From this framework we shall focus on the basic elements of a control loop and on the place of models in it. Secondly, we present the basic ideas of the “equation-free” approach. We shall focus on how it allows to analyze a system at macroscopic level, without explicit solution to macroscopic equations. Given that the context of modeling and simulation is at the heart of control and our proposition, we thirdly present, the concept of modeling within the context of our work, the multi-agent paradigm as well as some elements pertinent to the modeling and simulation framework. Finally, we present different applications of the multi-agent paradigm in the context of the control of complex systems.
r u y
C T
Figure 2.1: Closed feedback loop. The output u of system C is used as the input for system T while the output y of system T is used as the input of system C.
2.2.
Control Theory
Definition
The purpose of control theory is to determine which control actions will make a system reach or maintain a certain state (Sontag, 1998; Ogata, 2001). Within control theory, dynamical systems are described in terms of ordinary differential equations (ODE) like:
( ˙
x(t) = f (x(t), α(t)) (t > 0) x(0) = x0
where x is a function representing the evolution of the state of the system and f depends also on some “control” function α. In control theory, a feedback controller measures the effect of inputs on the process outputs and the results yielded by the controller are used again as input to the process, closing the loop (Astrom and Murray, 2008).
Feedback in control
Feedback exists when two entities mutually influence each other. In the context of control theory and control engineering, feedback is defined as being a loop where the output of one system is the output of another one. Figure 2.1 shows a graphical representation of a feedback loop. If the link between the output of T and the input of C disappears, we say that the loop is open, otherwise the loop is said to be closed.
In control engineering, feedback loops are exploited to gain control of a system. The basic setup of a control feedback loop is as follows: One of the systems in the loop is the system to be controlled (the target system T ) and the other one, the controlling system (C). The input of the controller system C is r, a reference value indicating the desired output of system T . The output of system C is u which in turn, is the input of system T . The output of system T is y. The difference between the observed output of the target system y and the reference value r is called the error e.
The goal of control theory is to find a control law that will be used to establish the outputs of the controller system, so that the output of the target system is as close as possible to the reference (which in turn is the input of the controller system). The control law of a controller system is its core. Since most of the tools used by control theory to find such laws are issued from operations research, ODE and game theory, it is natural to find that such laws have an analytical form.
The simplest way to implement a control law in a feedback loop is by using an "on-off" approach. It consists in applying the maximal control action possible (umax) whenever the
difference e between the reference value r and the measured output of the target system y is positive, and a minimal control action when the difference e is negative. The control law of an on-off controller only considers as input the current output of the target system. When there
2.2. Control Theory
is no difference, no control action is applied. A limit of this approach is that every time there is a difference (e 6= 0) one control action at one of the limits of the spectrum is applied. This controller is known to yield the target system oscillating and not very useful when the target system requires a certain input value in order to maintain the output at a particular level.
u = (
umax e > 0
umin e < 0
PID controllers overcome the limit of on-off controllers by taking into consideration a bigger set of information when creating the control law. PID stands for Proportional Integral and Derivative controller. In the family of PID controllers we first have the Proportional controllers. The control laws of these controllers define a range emin < emax called the proportional band.
When the error e is within the domain, the control action will be proportionally adjusted to error. Otherwise, the control action will be maximal for a maximum error and minimal for a minimum error. u = umax e ≥ emax kpe emin< e < emax umin e ≤ emin
Despite this new control law with a proportional approach is advantageous over the one of the on-off approach, it is known to have a limit: e should have a nonzero value in order to keep the target system at a steady desired value of y.
Proportional Integrative controllers overcome this limit by making the control action propor-tional to the integral of the error over the lapse of time u(t) =R0tkie(τ )dτ . In other words, the
history of error e is considered.
A final refinement consists in letting the control law consider a prediction of the error Tdtime
units ahead. This new element, the prediction, could be simply obtained for example by using a derivative over the error.
The final control action will be thus given in a PID controller as the sum of three terms: the one issue of the proportional control law, the one of the integrative control law and finally the one of the derivative control law.
A control law of the PID family can be described as the sum of three elements. 1. one that is meant to adjust the control action to be proportional to the error
2. one that is meant to adjust the control action by taking into consideration the past history of the error
3. one that is meant to adjust the control action by taking into consideration a prediction of the error
Elements two and three are called respectively integrative and derivative because they have an exact match to such operations in calculus. That is, the second element will have the form of an integral and the third one of a derivation. These elements are mathematical models of the error behavior. When developing a control feedback loop within the context of control theory, we seek to find the mathematical models that best fit the system. When we say best, this could mean either that the dynamics of the system are accurate or that the desired results are obtained within some ranges (optimized).
Definition 3 Feedback control loop: measure (regularly) the state of the system, and based on these measurements, some controllable parameter(s) is adjusted to drive the system to a certain state.
Modeling
The basic working hypothesis of control theory is that there is a model of the target system that identifies the evolution of the target system in terms of its inputs and outputs. Based on this knowledge, the modifications necessary to be executed to the inputs to produce a certain output or behavior can be directly obtained.
Models used in classical control theory, such as ordinary differential equations, are analytical and thus directly represent the global functioning of the system. In this kind of representation, the phenomena leading to this global functioning are rarely made explicit in terms of local actions. One first problem regarding the usage of control theory for complex systems is that, it considers analytical modeling of the system possible. Even if this is possible, the resulting model needs to account for the nonlinear dynamics of complex systems. This means, that the resulting model will be hardly solvable.
Synthesis
Models are necessary in control theory to foresee the effects of control actions and to estimate the future state of a target system. This work is supported on the hypothesis that to model complex systems, it is necessary to take into consideration the multiple levels of description present in them. From this point of view we consider traditional analytical models to be of little use in this case, because they only take into consideration one level. Moreover, analytical models are usually of little usefulness because they are not always solvable.
The equation-free approach directly deals with the case when explicit solutions to global analytical models are not available but local models are available. In the following section we present the approach and summarize the key ideas that we consider relevant to our work.
2.3.
Equation Free approach
Equation-free refers to a paradigm for multiscale computation and computer-aided analysis (Samaey, 2010). It is designed to be used in problems when the evolution of a system is observed at a global or coarse level while accurate models are only given at a local or finer scale level of description. The paradigm proposes to bypass the derivation of explicit macroscopic evolution equations when these exist but are not solvable because they are not in a closed form. The central idea is to avoid the explicit definition of coarse equations by using short bursts of appropriately initialized fine-scale simulation (Kevrekidis et al., 2003, 2004; Siettos et al., 2006; Siettos, 2011). This is not directly possible because of computational cost and because finer scale models are not always easy to be analyzed. Performing coarse-scale computational tasks with fine-scale models is often unfeasible: direct simulation over the full spatiotemporal domain of interest can be computationally prohibitive. Additional modeling tasks, such as numerical bifurcation analysis, are often impossible to perform on the fine-scale model directly: a coarse steady state may not imply a steady state for the fine-scale system.
The solution to overcome this limits of computational approaches given in the equation-free paradigm is as follows. Short bursts of fine scale simulation (short computational experiments) are designed, executed, and their results processed and fed back to the process, in integrated protocols aimed at performing the particular coarse-grained task (the detection of a macroscopic instability, for example). Two models, a fine-scale (f ) and a coarse scale (F ), are assumed to exist, each with an associated time-stepper. The fine-scale model is given by: ∂tu = f (u). The
2.3. Equation Free approach
given by ∂tU = F (U ). The coarse model time-stepper is given by: U (t + δt) = S(U, δt). The
fine scale model involves fine-scale variables: u(t). The coarse model is given in terms of coarse variables which are assumed to exist, but are unavailable in closed form: U (t).
The key building block of the approach is the coarse time-stepper. It can be seen as an algorithm that will use the output of a fine-scale simulator (given certain initial conditions) to set boundaries to the coarse model and thus be able to express it in a closed form. Formally speaking, the coarse time-stepper can be described as follows. Given an initial condition for the coarse variables U (t∗) at t∗ the coarse time-stepper involves:
Lifting. Create fine-scale initial conditions u(t∗), consistent with U (t∗)
Simulation. Use the fine-scale simulator to compute the fine-scale state u(t) at t ∈ [t∗, t∗+ δt]
Obtain the coarse state U (t∗+ δt) from the fine-scale state u(t).
Once a closed form of the coarse model is found, the approach suggests to use numerical methods to obtain solutions to the equations describing it. The suggested methods include: coarse projective integration (Gear et al., 2002) and patch dynamics (Samaey et al., 2009).
Examples of the equation-free approach include its usage in chemotaxis (Erban et al., 2006) and data clustering (Samaey et al., 2008). In (Samaey et al., 2008), data clustering is accom-plished by a multi-agent system. Agents move data items by picking them up and dropping them next to similar data items. The items are picked up and dropped with respect to some probability which is determined by the presence of similar data items in the local neighborhood. The clusters are therefore formed in an emergent way. The objective is to understand the perfor-mance of such a system. Instead of exhaustively exploring (experimentally) the spatiotemporal domain, they use the equation-free approach. They conclude that applying the approach is not a trivial task, and propose an iterative bottom-up technique to identify the variables to use in the coarse level.
The important ideas that we retain from this approach are:
It allows to obtain information about the state of a system given in a global description level when only a model given in the local description level exists.
This is done by simulating the model given at local description level but this simulation may take too much calculation time or resources.
To overcome this difficulty, a coarse time-stepper decides when to use the simulation of the local level of description model and specially how to translate the results that are given by at a local description level to the global description level.
This work is supported on the hypothesis that the multi-agent paradigm in particularly useful in the specific context of modeling complex systems. In the following section we elaborate on this hypothesis and present the basic concepts of the multi-agent paradigm. We shall first present the general concepts of the multi-agent paradigm and then the specific concepts related to multi-agent models.
Multi-agent models are by nature experimental: they have to be simulated in order to provide answer to the questions they were built for. Therefore, further in the following section, we shall present the general framework of modeling and simulation of (Zeigler et al., 2000), where our work is placed.
