HAL Id: hal-02067031
https://hal.archives-ouvertes.fr/hal-02067031
Submitted on 13 Mar 2019
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of
sci-entific research documents, whether they are
pub-lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Counterfactual causal analysis in the Kappa language
Mickael Laurent
To cite this version:
Mickael Laurent. Counterfactual causal analysis in the Kappa language. [Internship report] Carnegie
Mellon University. 2017. �hal-02067031�
Counterfactual causal analysis in the Kappa language
Mickaël LAURENT, advised by Jean YANG
Computer Science department of Carnegie Mellon University
General context
Because of the increasing volume of available experimental data, biologists are facing the chal-lenge of building larger and larger models of the network of chemical reactions that inhabits cells. Kappa is a modeling language that is used to model such networks, chemical reactions being represented by local graph rewriting operations, that are equipped with stochastic firing rates. It comes with a stochastic simulator along with some analysis tools.
The question that we study here is the following: given a Kappa model and an event (the triggering of a specific rule or the appearance of a particular species), what combination of other events can lead to it and how are they causally related ? We can try to answer this question by using static analysis. However, there are often a lot of ways to trigger an event and many of them are very unlikely to happen according to the kinetic of the model. It is hard to generate relevant solutions using static analysis, that’s why an other method is used.
Instead of directly analyzing the model, a simulation is run. It generates a trace where ev-ery event that happened is logged (i.e. evev-ery rules that have been applied, when and on which agents). Then, we look into this trace for an event that interests us, and a story of this event is constructed in order to explain how this event has been triggered. A story is a graph of causality : each node represents an event that happened and each edge a relation of precedence. It describes a way to make an event E happen, by showing some events that are sufficient to trigger E and by highlighting relations of precedence between them. The advantage of this method is that the sto-ries generated are based on sequences of events that have really happened during the simulation, and so they are generally more relevant than solutions obtained with static analysis.
The problem
However, some events of the trace could have had an important role without being shown in these stories. For instance, such a story can’t express something like this event is important because it inhibits an other event that would have prevented our event of interest to be triggered. My internship consisted in formalizing and generating more complete (and more relevant) stories that are able to show such important events.
First of all, we have to find a way to determine whether or not an event X is important for the happening of an event Y. For that, we would like to be able to answer counterfactual statements, that is questions of this kind : if X had not happened, would have Y happened anyway ?. These statements have been formalized and a tool has been implemented to allows us to recompute a trace with some interventions on it (which we call resimulator). So, we are able to answer counterfactual statements.
Then, stories have to be extended in order to show every important events and to be able to explain why they are important, and this is what I have worked on.
Contributions
First, I have studied examples on which the regular stories generated were not relevant enough, in order to determine a way to make them more complete. By reasoning on these examples, we can note that a thing that is missing with regular stories is inhibition relations. Indeed, they are only describing positive relations (also called relation of activation, i.e. an event directly allows another event to apply), but not inhibition relations (i.e. an event that prevent another potential event from happening).
Given that, I have defined and formalized counterfactual stories, that is stories with both activa-tion and inhibiactiva-tion relaactiva-tions, and with some nodes that correspond to factual events (that is events that happened in the original trace) and some other nodes that correspond to counterfactual events (that is events that have not happened in the original trace but that could have happened if some-thing had changed). I have designed an algorithm that use the resimulator in order to generate some counterfactual stories and to synthesize them into a more complete story (an extended story). Finally, I gave a 50-minutes talk in Carnegie Mellon University where I presented Kappa and my work.
Proofs and hypothesis
I have proved a completeness theorem to show that the combination of activation and inhibition relations is enough to explain any change that happened during a resimulation. I have also done some case studies in order to test my algorithm, and I have proved its correctness. These theorems and their proofs are detailed later.
The theorem of completeness use a locality property : the applicability of a rule on some agents and the modifications it provokes only depends on these reagents (it depends not on any other element of the mixture nor a global property). This locality property is verified in Kappa. However, it does not take any kinetic consideration, nor our definitions of activation and inhibition do. Indeed, we can’t totally preserve the kinetic of the original trace in our stories, however they would be very big (almost every events of the original trace would be in the story). But even so we can try to take some more kinetic aspects into account, and for that we would have to rethink our definitions of activation and inhibition.
Review and prospects
The problem of causality is here applied to the Kappa language, but actually it is a more general problem. The concepts and algorithms that are presented in this report could be applied in an other context as soon as we are able to define formally relations of activation and inhibition and the idea of resimulation.
Concerning Kappa, I have contributed to generate more relevant stories by adding counter-factual events and inhibition relations, but there still is work that can be done in order to improve it. Finally, we should not forget that the main application for Kappa is biology, and so we should discuss it with biologists in order to know what is the most important thing to do next.
Contents
1 Context and problem 1
1.1 Context of Kappa, biological applications . . . 1
1.2 Overview of Kappa . . . 1
1.3 Simulations and problem of causality . . . 2
1.4 Kappa stories . . . 3
1.4.1 Definitions . . . 3
1.4.2 Compression . . . 4
1.4.3 Example . . . 5
1.5 Limits of stories . . . 5
2 Counterfactual causal analysis 7 2.1 Counterfactual statements and resimulation . . . 7
2.1.1 The Monte Carlo semantics . . . 7
2.1.2 Formalization . . . 7
2.2 Counterfactual experiments . . . 8
2.3 Inhibition relations & counterfactual stories . . . 8
2.4 Extended Kappa stories . . . 9
3 My contributions 10 3.1 The algorithm . . . 10
3.2 Heuristics . . . 11
3.3 Results and proofs . . . 11
3.3.1 Completeness theorem . . . 11
3.4 Ideas for improvement . . . 12
4 Conclusion 14 5 Appendixes 14 5.1 Another example of Kappa model . . . 15
5.2 Simplified version of the algorithm . . . 16
5.2.1 Pseudo-code . . . 16
5.2.2 Example of an inhibition arrows search . . . 17
5.2.3 Termination . . . 18
5.2.4 Soundness . . . 18
1
Context and problem
1.1
Context of Kappa, biological applications
We are interested in the problem of intracellular signaling modeling. A cell is a chemical sys-tem composed of organic molecules that interact with each other. In order to understand cells behavior, it is important to model these networks of molecules. However, these models can be huge and grow every time that new interactions are discovered, that’s why we need computers to analyze them.
Traditionally, researchers use systems of ordinary differential equations (ODEs) to model intracellular interactions. It permits to model interactions between proteins, as well as post-translational modifications that change the state of sites on the proteins (for example, phospho-rylation).
However, there are two problems with ODE models. First, a precise model requires a different ODE to model each distinct chemical species, causing ODE models to scale poorly. Secondly, ODE models have little structure that we can exploit for scale mitigating analysis.
As an alternative, rule-based languages allow the representation of models as programs de-scribing rewrites over graphs, where nodes correspond to proteins and edges describe links be-tween them. Each rule can apply on patterns instead of full specified species, and so models do not need to enumerate each possible interaction. We lose no precision by using rule-based pro-grams, as they have a corresponding ODE semantics. Not only are these programs more concise than the corresponding ODE systems, but their structure also supports various analyses that are otherwise not possible. Kappa is such a language.
By transposing a biological model into a the Kappa language, we can easily and efficiently simulate the interactions between agents, see their evolution and get a lot of information thanks to static analysis and causal analysis. For instance, it is possible by using static analysis to draw a contact map (a map that synthesize the possible links between agents), to find some rules that can’t happen (dead rules) or more generally some invariants, etc. This report is about causal analysis of Kappa models.
1.2
Overview of Kappa
Kappa is a modeling language that defines some local graph rewriting operations. More pre-cisely, a Kappa model define the followings objects :
• A list of agent signatures. An agent typically corresponds to a molecule or a protein. It has a name and a list of sites. A site is a part of the agent that can have an internal state and on which an other agent can bind (a site can link to another site). Each site has a name and eventually a list of possible values for its internal state (the first specified is considered as the default state).
• A list of rules. A rule has a name, a left-hand side that describe the tests (i.e. a pattern that the reagents must match), and a right-hand side that describe the actions (i.e. the modifications to apply on the reagents tested, like a link creation/removal, an internal state modification, or the creation of a new agent). It also has a rate. Note that a rule can’t define an action for an element that is not tested (i.e. that is not specified in the pattern), except for the creation of a new agent.
• A list of initial agents (i.e. the quantity of agents of each type initially present in the mix-ture).
• A list of variables, modifiers and observers. Modifiers are events that will happen at a certain time, for example ’Add 1000 agents of type A after 5s of simulation’. Observers
allow us to track some quantities (for example, the concentration of a particular species in the mixture).
Below is an example of a Kappa code that model a biological phenomenon that is well-known by the biologist as the prozone effect. We will see later other examples.
1 # Agent d e c l a r a t i o n s 2 %agent: A( b , c ) 3 %agent: B ( a ) 4 %agent: C( a ) 5 6 # Rules 7 ’A. B ’ A( b ) , B ( a ) −> A( b ! 1 ) , B ( a ! 1 ) @ ’ o n _ r a t e ’ 8 ’A/B ’ A( b ! 1 ) , B ( a ! 1 ) −>A( b ) , B ( a ) @ ’ o f f _ r a t e ’ 9 ’A. C ’ A( c ) , C( a ) −> A( c ! 1 ) , C( a ! 1 ) @ ’ o n _ r a t e ’ 10 ’A/C ’ A( c ! 1 ) , C( a ! 1 ) −>A( c ) , C( a ) @ ’ o f f _ r a t e ’ 11 ’ intro_A ’ −> A( ) @ ’ i n t r o _ r a t e ’ 12 13 # R a t e s and c o n s t a n t s
14 %var: ’AV ’ 1E4 # M^(−1 ) 15 %var: ’ o n _ r a t e ’ 1 / ’AV ’ 16 %var: ’ o f f _ r a t e ’ 0 . 5 17 %var: ’ i n t r o _ r a t e ’ 0 . 4 ∗ ’AV ’ 18 %var: ’ C0 ’ 1 ∗ ’AV ’ 19 20 # I n i t i a l c o n d i t i o n s 21 % i n i t : ’ C0 ’ B ( ) , C ( ) 22 23 # Observable ( recommended s i m u l a t i o n t i m e : 30 s )
24 %obs: ’BAC ’ |A( b ! _ , c ! _ ) | / ’AV ’
Another example of Kappa model is available in appendix.
1.3
Simulations and problem of causality
A stochastic Kappa Simulator (KaSim) is available. It allows us to see how our system evolve with time. It proceeds like this : it weights each rule depending on their rate and the number of present agents on which it could apply, and it randomly chooses a rule to apply according to these weights. Then, it determines the time that the rule take before applying, by following an exponential distribution. The simulator repeats this procedure until no rule can apply or until the time limit set by the user is elapsed. As output, it can produce a trace that logs every event (see below) that happened during the simulation. Note that every agent present in the mixture is discernible (each agent has an unique id). The Kappa simulator is coded in OCaml, like most of other Kappa tools. It can be tested online at http://dev.executableknowledge.org/.
Definition 1(Event). An event is a rule application : it is characterized by the rule that is applied and the agents on which it is applied. When we are in the context of a simulation, an event designates one rule application in particular, and so it is also characterized by the time at which it happens.
The graph above has been obtained by running the simulator on the prozone effect model.
1.4
Kappa stories
Simulating is not the only thing that we want to do with a model. Another thing that we are interested in is to determine which are the ways to trigger a specific event. We can try to answer this question by using static analysis. However, there are often a lot of ways to trigger an event and many of them are very unlikely to happen according to the kinetic of the model. It is hard to generate relevant solutions using static analysis, that’s why an other method is used.
Instead of directly analyzing the model, a simulation is run and the resulting trace is analyzed. We look into it for an event that interests us, and then a story (defined later) of this event is con-structed in order to explain how this event has been triggered. The advantage of this method is that the stories generated are based on sequences of events that have really happened during the simulation, and so they are generally more relevant than solutions obtained with static analysis.
1.4.1 Definitions
First, let’s define formally relations of precedence and relations of activation between events.
Definition 2(Logical site). A logical site is a variable associated with an agent that can correspond to : • The internal state of one of its sites
• The binding state of one of its sites
• The presence or not of the agent in the mixture
So the state of an agent that has n sites can be fully characterized with 1 + 2n logical sites. In practice, rather than its sites, we prefer reasoning on the logical sites of an agent.
Definition 3(Concurrency). We say that two events A and B are concurrent iff :
• There exists a configuration (a state of the mixture) in which they can both be applied.
• In every configuration where they can both be applied, we can apply the two consecutively in any order and we get the same result.
Property 1. Two events are concurrent iff there is no logical site that is tested by one of them to a certain
value and that is tested or modified by the other to a different value. So, concurrence between two events does not depend on their context.
Definition 4(Precedence). An event A must precede an event B iff A happens before B in the trace (or subtrace) and A is not concurrent with B.
Definition 5(Activation). An event A activates an event B iff there exists a logical site such as A is the last event before B (in the trace/subtrace) to modify this logical site from a value not accepted by B to a value accepted by B. Activation is a particular case of precedence: A activates B ⇒ A precedes B.
Definition 6(Story). A story is an oriented graph without cycle : each node represents an event of a given trace, and each edge represents a relation of precedence. A story is valid if every precedence relations are correct and if we can successfully apply all its events one after the other in any order that respects the precedence relations (the initial agents in the mixture are created by the special event init).
A story tries to explain how a particular event E has happened in a trace. Generally, we do not show a relation of precedence if it can be deducted by transitivity. We can also highlight relations of activation (in this case, activation relations shown must be valid too). We want the story to be as small as possible, but without loosing any important event. In particular, we don’t want to show events that do not participate to the happening of E. Note that we can also compute stories to explain a set of events instead of only one event.
1.4.2 Compression
We want to keep only events that have participated to the happening of the event E. For that, we can take the event E and all its activators, all the activators of these activators, etc. until the beginning of the trace (the event init has no activator) : the set of events obtained is called causal past or causal cone.
However, actions are often reversible in a Kappa model, and so a trace can contains a lot of repetitions and useless sequences of events (for example, a binding is created, then it is broken, then it is created again...). The causal past can contains a lot of such useless events and so it can be huge, this is why we have to compress the trace. The compression of a trace consists in computing a subtrace (a subset of events of the trace) that is still valid and that contains the event of interest E.
We can find a minimal subtrace by transposing this problem to an instance of minSAT, but there is not an unique minimal solution. We can impose some additional constraints if we want an unique solution and/or a quasilinear algorithm.
In summary, given a trace and an event of interest E, the computation of a story for E is done in two steps :
1. The compression of the trace : We find in the trace a (minimal ?) subset of events that is sufficient to make the event E happen, i.e. a subtrace that contains E and that is valid. 2. The representation of this subtrace : we compute every relation of precedence between the
events of the subtrace, and we construct a graph : every event of the subtrace is a node, and every edge a relation of precedence.
Note that in practice, relations of precedence are only computed at the end when we show the story to the user : in the code, a story is just a subtrace because it is easier to manipulate.
1.4.3 Example
For instance, consider the Kappa model below :
1 %agent: A( b , c ) 2 %agent: B ( a~u~p ) 3 %agent: C( a~u~p ) 4 5 ’ Bp ’ B ( a~u ) −> B ( a~p ) @ 3 6 ’ Bu ’ B ( a~p ) −> B ( a~u ) @ 3 7 ’Cp ’ C( a~u ) −> C( a~p ) @ 3
8 ’Cu ’ C( a~p ) −> C( a~u ) @ 3
9 ’A. Bp ’ A( b ) , B ( a~p ) −> A( b ! 1 ) , B ( a~p ! 1 ) @ 1 10 ’A. Cp ’ A( c ) , C( a~p ) −> A( c ! 1 ) , C( a~p ! 1 ) @ 1 11
12 % i n i t : 1 A( ) , B ( ) , C ( )
13 %obs: ’BAC ’ |A( b ! 1 , c ! 2 ) , B ( a ! 1 ) ,C ( a ! 2 ) |
In this model, there is 3 agents : • A, with 2 sites b and c.
• B, with 1 site that can be phosphorylated (internal state p) or by default unphosphorylated (internal state u). There are rules to phosphorylate or unphosphorylate B, and if B is phos-phorylated, its site can bind to the site b of A.
• C is similar to B, but binds on the site c of A.
We want to know how BAC species can be created. To do that, we can add an observer for BAC species in the model, and say to the simulator to apply a special rule BAC each time a species that match this observer appears. In this way, we can then search an application of this rule in the trace and compute its story.
Here is an example of trace for this model (there is only one instance of each agent so we only put the name of the rule to represent an event, but normally we should indicate also on which agents it applies) :
1 i n i t Cp Bp Bu Bp Cu Cp Bu A. Cp Bp Bu Bp Bu Bp A. Bp BAC
After compression, a possible subtrace is :
1 i n i t Cp A. Cp Bp A. Bp BAC
Finally, the story corresponding to this subtrace is :
1.5
Limits of stories
Sometimes, some events that happened during the simulation have a very important role but are not shown in these stories, and so they are not always relevant. By slightly modifying the
previous example, we can construct a new example where the story generated is not complete enough : now, B and C can bind to A even if they are unphosphorylated, but in this case the binding created is weaker and is very likely to be broken.
1 %agent: A( b , c ) 2 %agent: B ( a~u~p ) 3 %agent: C( a~u~p ) 4 5 ’ Bp ’ B ( a~u ) −> B ( a~p ) @ 3 6 ’ Bu ’ B ( a~p ) −> B ( a~u ) @ 3 7 ’Cp ’ C( a~u ) −> C( a~p ) @ 3
8 ’Cu ’ C( a~p ) −> C( a~u ) @ 3
9 ’A. B ’ A( b ) , B ( a ) −> A( b ! 1 ) , B ( a ! 1 ) @ 1 10 ’A. C ’ A( c ) , C( a ) −> A( c ! 1 ) , C( a ! 1 ) @ 1 11 ’A/Bu ’ A( b ! 1 ) , B ( a~u ! 1 ) −> A( b ) , B ( a~u ) @ 5 12 ’A/Cu ’ A( c ! 1 ) , C( a~u ! 1 ) −> A( c ) , C( a~u ) @ 5 13
14 % i n i t : 1 A( ) , B ( ) , C ( )
15 %obs: ’BAC ’ |A( b ! 1 , c ! 2 ) , B ( a ! 1 ) ,C ( a ! 2 ) |
1 i n i t Cp Bp Bu A. B Cu A/Bu Bp A. C A/Cu Bu Cp A. C Bp A. B BAC
A minimal subtrace and its corresponding story would be :
1 i n i t A. C A. B BAC
This story is not suitable because some important events are missing ! Indeed, the rules Bp and Cp are important for the creation of the BAC species : if they had not happened, A.B and A.C would have been very unlikely to both happen before one of them breaks.
A more suitable story would be :
Here, dotted arrows are relations of precedence, plain arrows are relations of activation, and red crossed arrows are relations of inhibition (defined later). Note that nodes in red are events that have not happened in the original trace! Such stories are called counterfactual stories.
2
Counterfactual causal analysis
We will see in this part how to formalize such counterfactual stories. We first have to find a way to know whether or not an event is important. In the previous example, we said that Bp and Cp were important because if they had not happened, the event of interest would not have happened. More generally, the question that we would like to answer is : What would have happened if ... had not happened ? Such statements are called counterfactual statements.
2.1
Counterfactual statements and resimulation
We would like to have a way to resimulate a trace by imposing some interventions. Such a resimulator should :
• Apply the interventions and give us a probable resulting trace.
• Conserve every event of the factual trace that are not impacted by these interventions. In other words, an event should not be blocked or added without a reason.
We can’t just rerun a simulation by preventing some events to happen, because simulations are stochastic and so the resulting trace would be completely different from the original.
To formalize counterfactual statements and resimulation, we must introduce the Monte Carlo semantics of models.
2.1.1 The Monte Carlo semantics
For a given model, we define a potential event as a pair (r, φ) where r is a rule of the model and φ an injective mapping that associates to each local pattern tested by the rule a global agent in the mixture. It corresponds to a potential rule application : r is the rule and φ the reagents.
For each of these potential events, we associate a bell that rings according to a Poisson pro-cess of parameter the rate of the rule (so the time between two ringings follows an exponential distribution). These ringings correspond to the times at which the rule could be triggered.
Then, a simulation trace can be seen as a random variable T that is determined by such a set of bell ringings as follows: starting from the initial mixture and every time a bell rings, we apply the corresponding potential event if it is applicable in the current mixture. Otherwise, we do nothing.
Let ι be a predicate over events that indicates which of them we want to block. This cor-responds to the interventions (note that we could also define other types of interventions like adding some new bell ringings to trigger a new event, but we don’t need it here). We can define a random variable Tι that is similar to T except that bell ringings that correspond to an event
blocked by ι are ignored.
2.1.2 Formalization
Now, for a predicate φ over traces, a trace τ and some interventions ι, we can define the following counterfactual statement:
If the events that satisfy ι had not happened in the trace τ , φ would have been verified with probabilty p ⇐⇒ τ |=p[ι]φ ⇐⇒ P (φ(Tι)|T = τ ) ≥ p
In this formalism, for a factual trace τ and some interventions ι, a counterfactual trace can be seen as the random variable Tιconditionned by T = τ .
2.2
Counterfactual experiments
Resimulation allows us to evaluate a counterfactual statement and so to determine whether or not an event is important, but it also gives us an explanation of why the event is important. We want to add this explanation to our stories, that’s why we introduce counterfactual experiments and counterfactual stories.
Definition 7(Counterfactual experiment). A counterfactual experiment is a couple (Tf, ι, Tc)where :
• Tf is a trace, that we will call factual trace.
• ι is a predicate over events that represents the interventions on the factual trace (the predicate is true when the event is blocked).
• Tcis another trace, that we call counterfactual trace.
A counterfactual experiment (Tf, ι, Tc)is valid iff the following property is verified : If an event E that is
not directly blocked by ι happens in one of the trace and if E is also applicable in the other trace at the same moment and with the corresponding reagents (if they exist), then E also happens in the other trace.
In other words, when an event happens only in one trace of a valid counterfactual experiment, there is a reason for that. Thereafter, counterfactual experiments used are supposed valid.
There is 3 types of events in a counterfactual experiment : events that happen in both Tfand
Tc (common events), events that happen only in Tf (factual-only events) and events that happen
only in Tc(counterfactual-only events).
Property 2 (Validity of resimulation). Let Tf be a trace, ι some interventions on it and Tc a trace
obtained by resimulating Tfwith the interventions ι. Then (Tf, ι, Tc)is a valid counterfactual experiment.
2.3
Inhibition relations & counterfactual stories
For every event that happen only in one trace of a counterfactual experiment, we want to find a causal explanation for it. We do that by analyzing inhibition relations between the events of the two traces and computing a counterfactual story.
Definition 8(Inhibition relation). In a counterfactual experiment (Tf, ι, Tc), an event E1of the trace
T1(T1= Tf or T1= Tc) inhibits an event E2of the trace T2(T2= Tfor T2= Tc; T26= T1) iff :
• E2does not happen in T1and E1does not happen in T2.
• There exists a logical site such as E1is the last event before E2in T1(in term of time of happening)
to modify this logical site from a value accepted by E2to a value not accepted by E2.
• There is no event in T1or in T2between E1and E2 that modify this logical site from a value not
accepted by E2to a value accepted by E2.
Definition 9(Counterfactual story). A counterfactual story is a tuple (F, ι, C, p, a, i) where (F, ι, C) is a counterfactual experiment, p is a list of precedence arrows, a is a list of activation arrows and i is a list of inhibition arrows.
It can be seen as a graph. Each node represents an event that can be factual-only (usually represented in blue), counterfactual-only (usually represented in red) or common if the event is in both F and C (usually represented in white or grey). Events of ι can be highlighted. There is an edge for each relation of precedence, activation or inhibition.
A counterfactual story (F, ι, C, p, a, i) is valid iff :
• Every inhibiton relation in i is a valid inhibition relation in the experiment (F, ι, C).
• For every event E of F or C that is not common, there exists a path composed of inhibition relations (in i) and activation relations (in a) that starts from an event directly blocked by ι and ends to E. A completeness theorem that is detailed later ensures that for any event E that happens only in one of the two traces, an explanation can always be found by constructing a path of activation and inhibition arrows from an event directly blocked by ι to E. So for every counterfactual experiment we can find a counterfactual story that explain the differences between the two traces. A counterfactual experiment directly obtained from a resimulation can be huge and so, before using it in a counterfactual story, we need to compress it. A valid compressed counterfactual experiment is a subset of the initial counterfactual experiment (that is a subset of the factual trace and a subset of the counterfactual trace) such as it is still a valid counterfactual experiment.
In OCaml, I represent a counterfactual story only with a valid compressed counterfactual ex-periment and a list of inhibition arrows. We can then easily retrieve the precedence and activation relations by computing them all independently in the two subtraces.
2.4
Extended Kappa stories
Given a trace and an event of interest E, we can compute many counterfactual stories in order to explain the importance of different events. We would like to enrich a regular story with all these counterfactual stories in order to have a more complete story that shows every (or at least more) important events. I call this kind of story an extended story .
An extended story is a pair of a regular story (the main story) and a collection of counterfactual stories. The main story contains, in addition to the event of interest and its compressed causal past, every event that has been determined as important. The collection of counterfactual stories is here to explain why these events are important.
Such stories are more difficult to render than regular stories, because we can’t show the main story and every counterfactual stories in the same graph ! Although they share some factual events, some other factual events can happen in a compressed counterfactual experiments and not in an other, and so we can’t merge all these stories together. That’s why an interactive visu-alization tool is needed : for example, the main story is shown, and when the user clicks on an event that is explained by a counterfactual story, the graph is smoothly transformed in order to show this counterfactual story.
3
My contributions
My goal was to find a way to generate more complete stories by using resimulation. First, I have formalized the notions of inhibition, counterfactual experiment and counterfactual story, and I have proved a theorem of completeness. Then, I have designed an algorithm to generate extended stories and I have implemented it in OCaml. I have also proved its correctness. Next, I worked on an interactive way to represent these extended stories (see examples in appendix) that I have implemented in Javascript. Finally, I gave a 50-minutes talk to some other people of the computer science department of Carnegie Mellon University. I spoke of Kappa in general in a first time, and then in a second time I presented more specifically my work.
3.1
The algorithm
I designed an algorithm that computes some valid compressed counterfactual stories for a given trace and an event of interest. Basically, it does the following :
1. An initial factual story is computed from the factual trace.
2. An heuristic determines some events that might have an important role but that are not in the story.
3. The factual trace is resimulated by blocking these events. If the event of interest happens anyway, we stop here.
4. The counterfactual story associated with this counterfactual experiment is computed (see below).
5. The initial factual story is updated to contains the new important factual events involved in the counterfactual story, and we loop to the step 2.
To compute counterfactual experiments :
1. Let F be a story of the factual trace and C a story of the counterfactual trace. Together, they represent a compressed counterfactual experiment. Initially, F is the initial factual story and C an empty story.
2. For each event of F or C that appears only in one trace, we find a path composed of activa-tion and inhibiactiva-tion arrows that explains why this event happens only in one trace.
3. We update F to contains the new factual events involved in these paths. We update C to contains the new counterfactual events involved in these paths.
4. If some events in F also happen in the counterfactual trace, we update C to contains these events, and vice versa.
5. We loop to step 2 until a fix point is reached (that is until each event that appears only in F or C is explained). The result can be seen as a counterfactual story :
(a) Common events are events that appear in both F and C, factual-only events are those that appear only in F and counterfactual-only events those that appear only in C. (b) Inhibition relations are those that have been computed in step 2.
(c) Activation relations can be computed independently in F and C.
You can find in the appendixes a simplified version of the algorithm in pseudocode and a detailed example of an inhibition arrows searching. The entire algorithm coded in OCaml can also be found on my github : https://github.com/E-Sh4rk/ExtendedStory
3.2
Heuristics
Some heuristics are needed in order to generate extended stories, in particular to determine which factual events to block before resimulating. Indeed, we can’t block every factual event one after the other because resiumlation is expensive and so it would not scale ! Nor we can block too many factual events in the same time because the resimulated trace would be quite different from the original trace and it would be more difficult to determine which of the blocked events were important and why.
For instance, one possible heuristic for this is to do some statistical analysis on some rules of the model beforehand in order to find some properties of this kind : when the rule R is applied, the logical site X of the local reagent A has most of time the value Y, although it is not really needed in order to apply the rule. Then, we can assume that if there is such an application of R in the current factual story and if beforehand an event of the factual trace put the logical site X of the corresponding reagent A to the value Y, it is probably an important event.
Even once we have determined which factual events we should block, we have to determine the predicate to use in order to block them. On one hand, if we only block these factual events (and no other counterfactual event that could happen during the resimulation), there is a high probability that a similar event happen just after in the counterfactual trace : in the Monte Carlo semantics, if we block only one bell ringing for a potential event, this potential event is likely to happen anyway at the next bell ringing (if no other potential event that could prevent it happen meanwhile). On the other hand, blocking all bell ringings of all the potential events involved could have some undesired effects (maybe some of these potential events have another role to play after !).
The current heuristics that I have implemented are based on observations on some toy models. However, if we want more efficient heuristics in practice, we have to do some case studies on real biological models.
3.3
Results and proofs
You can find proofs of the termination and the correctness of the algorithm in the appendixes. Here we will focus on a completeness theorem.
3.3.1 Completeness theorem
We will prove through a completeness theorem that counterfactual stories (with both activation and inhibition arrows) are expressive enough to explain the result of any resimulation. The proof is constructive and so it can easily be adapted to prove the completeness of our implementation of the algorithm.
Property 3(Locality). The applicability of a rule on some reagents and the modifications it provokes only depends on these reagents, more precisely their type and the value of their logical sites.
Proof. This property is trivial :
• A rule is applicable on any agent that match the pattern tested (left-hand side of the rule), so the applicability of a rule only depends on the type and the state of the potential reagents on which it is applied.
Theorem 1(Completeness Theorem). Let (Tf, ι, Tc)be a valid counterfactual experiment. If an event
E happens only in one of the two traces Tf or Tc, then there exists a path composed of activation and
inhibition arrows from an event directly blocked or added by ι to E.
Proof. For a counterfactual experiment R = (Tf, ι, Tc), we define Nf(R, E)the number of events
that happened in Tf before E included (if E has not happened in Tf we can compare the time of
happening of the events). We defineNc(R, E)the same way but for events in Tc.
Let N (R, E) = Nf(R, E) + Nc(R, E). We prove the property by induction on N (R, E) :
For N (R, E) = 1 : E is the first event : it is the init event. The result is immediate : if init has not happen in a trace, that’s because it has been directly blocked or added by ι.
For N (R, E) = N > 1: We suppose the property true for every n < N .
Let R = (Tf, ι, Tc)a valid counterfactual experiment and E an event that appears only in one
of the two traces, such as N (R, E) = N .
If E has been directly blocked or added by ι, we are done. So we can suppose that E is not directly blocked or added by ι.
Since R is a valid counterfactual experiment, we deduct with the locality property that there is a logical site tested by E that was not satisfied in the other trace. We call Tethe trace in which
Ehappened, and Tnthe other.
With our definitions of activation and inhibition, we immediately deduct that one of these propositions is verified :
• There exists an event Eethat was an activator of E in Teand that has not happened in Tn.
• All activators of E have happened in Tn, but there exists an event En that happened in
Tn between one of the activator of E and E, and that set the logical site corresponding to
this activator from a value accepted by E to a value not accepted by E. If there is many events like that, we take the last one. We can easily verify that this event Enis necessarily
an inhibitor of E.
In the first case, we can apply the induction hypothesis to the event Ee and complete the path
with an activation arrow between Eeand E. In the second case, we can apply the induction
hypothesis to the event Enand complete the path with an inhibition arrow between Enand E.
3.4
Ideas for improvement
There is still work to do in order to improve the generation of extended stories :
• Optimizing the code in order to scale better : for instance, we could avoid to load the entire trace into the memory and only access it through a stream.
• Testing on real biological models and improving heuristics depending on the observations we make.
• Doing more advanced statistical tests to determine whether or not an event is important. It could also be very convenient to be have an interactive way to generate stories. For instance, an initial story could be computed, and then the user could propose some modifications (like "I don’t want this event here"). He could also choose to block some events and then, if at least one of them was important, a counterfactual story would be computed and shown to him. In other words, the user could play the role of the heuristics (or work in conjunction with them).
Concerning the kinetic, although extended stories are expected to better preserve the kinetic of the original trace than regular stories, there is still improvements that could be done in this field (note that the notion of kinetic of a story is hard to define formally...). For example, the property of locality that we used in the previous section doesn’t take any kinetic consideration into account. Indeed, this property is not verified if we add kinetic considerations :
• The probability for a rule to be applied depends on the composition of the mixture (more precisely, it depends on the number of species that match a pattern of the rule and the total number of species).
• Actually, a rule can have two different rates depending on if its reagents are on the same connected component or not. This distinction can involve many other agents that those which are tested in the left-hand side of the rule.
If we want stories to fully preserve the kinetic of the original trace, we would have to take into account these two constraints. Indeed, it is not desirable to take into account the first point, because it would need to keep in the story almost every event of the original trace in order to preserve the concentration of each species. However, we could try to take into account the second point : when a rule has very different rates depending on the connectivity of its reagents, we could modify our definitions of causality (activation and inhibition) in order to preserve the fact that the rule is applied on a connected component or not.
Finally, we should not forget that the main application of Kappa is biology. Personally, I only had a logical/mathematical approach during this internship, but in order to have more efficient tools in practice we have to discuss it with biologists and adapt our approach.
4
Conclusion
This internship has been very interesting. I have studied causality and counterfactual reasoning in the context of the Kappa language, for biological purpose. However it is actually a more general problem that could be applied to many other domains. In robotics for example, causal reasoning could be used for decision making : we could define some relations of activation and inhibition between the actions taken by the robot, the events that happened in the environment, etc.
It was my first experience within a research team, and it was really interesting to collaborate and discuss with the other researchers. It was also a great opportunity to practice my english and meet a lot of researchers in various fields. In particular, I have discussed with some researchers in security, a field that interests me a lot.
I look forward to renewing such experience next year in order to discover other fields of research !
References
[1] V. Danos, J. Feret, W. Fontana, R. Harmer, and J. Krivine. Rule-Based Modelling of Cellular Signalling, invited paper. In L. Caires and V.T. Vasconcelos, editors, Proceedings of the Eigh-teenth International Conference on Concurrency Theory, CONCUR 2007, Lisbon, Portugal, volume 4703 of Lecture Notes in Computer Science, pages 17–41, Lisbon, Portugal, 2007. Springer, Berlin, Germany.
[2] V. Danos, J. Feret, W. Fontana, and J. Krivine. Scalable Simulation of Cellular Signaling Net-works, invited paper. In Z. Shao, editor, Proceedings of the Fifth Asian Symposium on Program-ming Systems, APLAS 2007, Singapore, volume 4807 of Lecture Notes in Computer Science, pages 139–157, Singapore, 2007. Springer, Berlin, Germany.
[3] Vincent Danos, Jérôme Feret, Walter Fontana, Russell Harmer, Jonathan Hayman, Jean Kriv-ine, Christopher D Thompson-Walsh, and Glynn Winskel. Graphs, Rewriting and Pathway Reconstruction for Rule-Based Models. In Deepak D’Souza, Telikepalli Kavitha, and Jaiku-mar Radhakrishnan, editors, IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science, FSTTCS 2012, volume 18 of LIPIcs, pages 276–288. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2012.
[4] JY Halpern and J Pearl. Causes and explanations: A structural-model approach. Part I: Causes. The British journal for the philosophy of, 2005.
[5] Florian Leitner-Fischer and Stefan Leue. Causality Checking for Complex System Models. In VMCAI, volume 7737, pages 248–267, 2013.
[6] David Lewis. Causation. The journal of philosophy, 70(17):556–567, 1974.
[7] David Lewis. Causation as influence. The Journal of Philosophy, 97(4):182–197, 2000.
5.1
Another example of Kappa model
1 # ### S i g n a t u r e s
2 %agent: A( x , c ) # D e c l a r a t i o n o f agent A 3 %agent: B ( x ) # D e c l a r a t i o n o f B
4 %agent: C( x1~u~ p , x 2 ~u~p ) # D e c l a r a t i o n o f C with 2 m o d i f i a b l e s i t e s 5
6 # ### Rules
7 ’ a . b ’ A( x ) , B ( x ) −> A( x ! 1 ) , B ( x ! 1 ) @ ’ o n _ r a t e ’ #A binds B
8 ’ a . . b ’ A( x ! 1 ) , B ( x ! 1 ) −>A( x ) , B ( x ) @ ’ o f f _ r a t e ’ #AB d i s s o c i a t i o n 9
10 ’ ab . c ’ A( x ! _ , c ) ,C ( x1~u ) −> A( x ! _ , c ! 2 ) ,C ( x1~u ! 2 ) @ ’ o n _ r a t e ’ #AB binds C 11 ’mod x1 ’ C( x1~u ! 1 ) ,A ( c ! 1 ) −> C( x1~p ) ,A ( c ) @ ’ mod_rate ’ #AB m o d i f i e s x1 12 ’ a . c ’ A( x , c ) ,C ( x1~ p , x 2 ~u ) −> A( x , c ! 1 ) ,C ( x1~ p , x 2 ~u ! 1 ) @ ’ o n _ r a t e ’ #A binds C 13 ’mod x2 ’ A( x , c ! 1 ) ,C ( x1~ p , x 2 ~u ! 1 ) −> A( x , c ) ,C ( x1~ p , x 2 ~p ) @ ’ mod_rate ’ #A mod x2 14
15 # ### V a r i a b l e s
16 %var: ’ o n _ r a t e ’ 1 . 0 E−3 # per molecule per second 17 %var: ’ o f f _ r a t e ’ 0 . 1 # per second
18 %var: ’ mod_rate ’ 1 # per second 19 %obs: ’AB ’ |A( x ! x . B ) |
20 %obs: ’Cuu ’ |C( x1~u? , x 2 ~u ? ) | 21 %obs: ’Cpu ’ |C( x1~p? , x 2 ~u ? ) | 22 %obs: ’Cpp ’ |C( x1~p? , x 2 ~p ? ) | 23 24 %var: ’ n_ab ’ 1000 25 %obs: ’ n_c ’ 0 26 %var: ’C ’ |C ( ) | 27 28 # ### I n i t i a l c o n d i t i o n s 29 % i n i t : ’ n_ab ’ A( ) , B ( ) 30 % i n i t : ’ n_c ’ C ( ) 31 32 %mod: [ T]=10 || [ f a l s e ] do $ADD 10000 C ( )
5.2
Simplified version of the algorithm
5.2.1 Pseudo-code
functionEXTENDED_STORY(trace Tf, event of interest eoi)
involved_events ← [eoi] S ←Story(T , involved_events) CSs ← []
interventions ←Heuristic_interventions(Tf, blacklist, S, eoi)
blacklist ←events in interventions Tc← Resimulate(Tf, interventions)
while eoidoes not happen in Tcdo
CS ←Counterfactual_story(Tf, Tc, intervention, eoi)
involved_events ← involved_events∪ events in the factual story of CS S ←Story(T , involved_events)
CSs ← CSs ∪ CS
interventions ←Heuristic_interventions(Tf, blacklist, S, eoi)
blacklist ← blacklist∪events in interventions Tc← Resimulate(Tf, interventions)
end while return(S, CSs)
end function
functionCOUNTERFACTUAL_STORY(Tf, Tc, interventions, eoi)
important_events ← least fixpoint of
(events → Add_explanations(Tf, Tc, events)) containing ([eoi]∪ events in interventions)
f_story ← Story(factual events of important_events)
cf_story ← Story(counterfactual events of important_events)
return (f_story, cf _story, arrows, interventions)
end function
functionADD_EXPLANATIONS(Tf, Tc, important_events)
f_story ← Story(factual events of important_events)
cf_story ← Story(counterfactual events of important_events)
(inhibitions, involved) ←Inhibition_arrows(Tf, Tc, f _story, cf _story)
important_events ← important_events∪ common events in f _story ∪ common events in cf _story ∪ involved
return (important_events, inhibitions)
end function
functionINHIBITION_ARROWS(Tf, Tc, f _events, cf _events)
.Arrows are tuples (source event, logical site involved, destination event) (arrows_cf, involved_cf ) ←S
e∈f_eventsInhibition_arrows_of_event(Tf, Tc, e)
new_cf _events ← counterfactual events of involved_cf (arrows_f c, involved_f c) ←S
e∈cf_eventsInhibition_arrows_of_event(Tc, Tf, e)
new_f _events ← factual events of involved_f c
(new_arrows, new_involved) ← Inhibition_arrows(Tf, Tc, new_f _events, new_cf _events)
return(arrows_f c ∪ arrows_cf ∪ new_arrows, new_involved ∪ involved_f c ∪ involved_cf )
end function
functionINHIBITION_ARROWS_OF_EVENT(T1, T2, event) . eventmust happen only in T1
if eventhappened in T2then return []
end if
arrows ← []
for alllogical site s tested by event do
ifthe last event before event in T2that modify s set it to a refused value then
inh ←the last event in T2before event that modified s from an accepted value to a
refused value OR the init event if such event does not exists
ifthere is an event in T1between inh and event that modify s from a refused to an
accepted value then
new_destination ← last event that verify this condition
(new_arrows, new_involved) ← Inhibition_arrows_of_event(T1, T2, new_destination)
arrows ← arrows ∪ new_arrows
events_involved ← events_involved ∪ new_involved
else
arrows ← arrows ∪ [(inh, s, event)]
events_involved ← events_involved ∪ [inh]
end if end if
inhibitor ←last event in T2happened before event that set s from a
end for
return (arrows, events_involved)
end function
Note that :
• The function Story can correspond to any compression algorithm : it takes a trace and a list of events of interest, and returns a subtrace that contains the given events of interest and that is still valid.
• The function Heuristic_interventions is a function that returns a list of factual events to block during the resimulation. These events must not be in the blacklist.
5.2.3 Termination
• Inhibition_arrows_of_event : when it calls itself, the two first parameters T1 and T2 are
unchanged, and the third parameter event is strictly decreasing (by using the happening order in T1). There is a finite number of events and so this function always terminates.
• Inhibition_arrows : same thing as above. The new set of events on which this function calls itself is strictly decreasing by considering the happening order of the latest event (in term of happening order) of the set.
• Add_explanations : no recursion, no loop.
• Counterfactual_story : We can find a fixpoint p by starting from a =([eoi]∪ events in interventions) and iterating the function f =(events → Add_explanations(Tf, Tc, events)) until we have
a fixed point. It always terminates because there is a finite number of set of events and because f is such as f (x) ⊇ x. Moreover f is monotonic and so p is the least fixpoint : Let p0 be a fixpoint containing a. a ⊆ p0 so f (a) ⊆ f (p0) = p0 ; f (f (a)) ⊆ f (p0) = p0 ... Recursively : p ⊆ f (p0) = p0.
• Extended_story : a factual event can be blocked only once thanks to the blacklist, so Heuris-tic_interventions can be called at most length(T1) times before accepting every factual event.
Then the condition of the while can’t be true because the resimulator generates a valid trace. So this function always terminates.
5.2.4 Soundness
This algorithm produces valid counterfactual stories (F, ι, C, p, a, i) .
• F and its corresponding precedence/activation relations is a valid story. Same thing for C. This is verified by construction : F and C are constructed by computing a story with the function Story, that is supposed to be valid. p and a are not given by the algorithm but can easily be computed later in F and C independently.
• Every inhibiton relation in i is a valid inhibition relation in the experiment (F, ι, C). We can easily verify by construction that Inhibition_arrows_of_event return inhibition ar-rows that are valid in the experiment (Tf, ι, Tc).
Moreover, the final F is a valid subtrace of Tfthat contains all the factual events involved
in i, and the final C is a valid subtrace of Tc that contains all the counterfactual events
involved in i. So inhibition arrows returned by Inhibition_arrows_of_event are still valid in the experiment (F, ι, C) (trivial).
• For every event E of F or C that is not common, there exists a path composed of inhibition relations (in i) and activation relations (in a) that starts from an event directly blocked by ι and ends to E.
For that property, we must ensure that Inhibition_arrows find such paths for every non-common event given as argument. This will not be detailed here but you can refer to the proof of the completeness theorem. Also note that this function does not return any activation arrow (they will be computed at the end independently in F and C, just before rendering the story). However, every event involved with the paths found are added to F and C and so it ensures that all necessary activation arrows will be found at the end.
5.3
Examples of stories generated
For the model used in the section Example of an inhibition arrows search :
For an extended version of the previous model with a new agent D that is similar to B and C :
An example of competition between species : we want that two agents A and B bind together. However, they are in a solution with many D agents that also want to bind to A on the same site as B would do ! Hopefully, there is also some C agents that can bind to A on an other site and that prevent D agents from binding.
1 %agent: A( b , c ) 2 %agent: B ( a ) 3 %agent: C( a ) 4 %agent: D( a 1 , a 2 ) 5 ’A. B ’ A( b ) , B ( a ) −> A( b ! 1 ) , B ( a ! 1 ) @ 1 6 ’A. C ’ A( c ) , C( a ) −> A( c ! 1 ) , C( a ! 1 ) @ 1 7 ’A.D ’ A( b , c ) , D( a 1 , a 2 ) −> A( b ! 1 , c ! 2 ) , D( a1 ! 1 , a 2 ! 2 ) @ 1 8 % i n i t : 1 A( ) , B ( ) 9 % i n i t : 10 C ( ) , D( )
