Contributions

In this section, we present the main contributions of our work. As the topic of our work has been the adaptation of existing methods to the causal study of telecommunication networks, our contributions are mainly focused on the methodology.

Methodology

In the different studies presented in the previous chapters we could show how to avoid the main pitfalls inherent to a causal study by keeping in mind a series of considerations when designing a causal study:

Scenarios Not all scenarios are eligible to a causal study. One should some-how foresee the model that he could obtain and the use that he would make of this model. An common mistake is to expect that by giving messy data to a machine learning algorithm a clean model will be inferred. Un-fortunately, this is not the case and time and efforts will be spared if a special care is taken in the design of the causal study.

Level of details of a study Assuming that we decide to perform the causal study of a system that, we believe, would benefit from a causal approach.

The level at which the system will be represented is an important deci-sion. At a too high level, the model would not highlight any interesting dependencies, at too low level we take the risk of modeling mechanisms requiring an important number of parameters to be modeled, and could suffer from the presence of latent variables. Additionally, if we try to model mechanisms that rely on hard-coded protocols we do not want to try to model deterministic mechanisms that would violate the faithfulness as-sumption necessary for our algorithm to infer a system causal model.

Level of abstraction A subtle difference with the previous point is the choice of the parameters that will form our system model. Each parameter cap-tures, in its variations, the influence that all the mechanisms and external parameters have on this parameter. The choice of the parameters of our model should be dictated by the level of abstraction that the system model is aiming at. This choice should also consider the possible latent

7.1. OBJECTIVES AND ACHIEVEMENTS 151 variables present in the intended model. Coming back on the example of thewindshield wipersandcar accidents, if we want to study the probabil-ity ofcar accidents and we do not see beforehand that therainis a latent variable that influences both, the usage ofwindshield wipersand the prob-ability ofcar accidents, introducing only thewindshield wipersvariable will introduce the presence of a latent variable that will make the detection of the dependence between the two variables windshield wipersandcar accidentmore difficult. The problematic of feature selection is a very com-plex problem, which we briefly addressed in Section 1.3.4.

Data preprocessing As highlighted by the study of the impact of the DNS ser-vice on CDN user throughput in Chapter 6, understanding the input will help to understand the causal dependencies inferred by the causal model inference algorithm to detect errors in the measurement process or pa-rameter estimations. Outliers are important for the prediction of interven-tions but the opposite is often true for the inference of causal dependen-cies. In our work, we often restrict our input data to the samples falling in the[P5,P95]during the stage of the causal model inference. WherePX

represents theX^th percentile of a given parameter observed values.

Domain knowledge Domain knowledge is necessary at every step of a causal study: in the choice of the parameters, in the interpretation of the causal dependencies, and in the interpretation of the predictions.

Our work underlined the limitations of the application of a causal study to real world applications. Once the assumptions of normality, linearity and discrete-ness removed, most of the existing independence tests become inaccurate and the inference of a system causal model becomes more complex. Identically, a causal model allows to infer the equations that will define the distributions of the parameters post intervention from their distributions pre intervention. How-ever, the estimation of multi dimensional distributions where no assumption can be made, is a complex problem that prevents the use of the causal theory in many scenarios. Finally, from the do-calculus one can read that the absence of certain observations in the input dataset prevents the prediction of many interventions and little importance has been given to this consideration in the literature that formed the basis of our work. Working with incomplete data is an important and complex issue [DKCDS12, GM04].

However, in our work, we showed that such challenges could be overcome by adopting and adapting the correct methods to the constraints of our problem.

With the use of the KCI+bootstrap method for testing independences and the use of copulae to model conditional probability density functions, we manage to show that the causal theory can be applied to system more complex than the ones where we usually see it applied.

A causal approach to the study of Telecommunication networks

152 7. CONCLUSIONS

Results

Our work shows, through examples, that causal studies can, as claimed, be applied to a range of problems and, while special care has to be taken when the systems are more complex, the tools we present offer a good accuracy and exploitable results. The models inferred in our work uncover not only the de-pendencies suggested by the literature but also new dede-pendencies that, while totally supported by the mechanisms behind telecommunication networks per-formance, would have been difficult to detect otherwise.

In our study of the FTP server performance, in Section 5.2, the causal model presents the receiver window, the loss frequency and the RTT as direct causes of the throughput, the parameters known to influence the throughput. But the model also detects a dependence between loss events and RTT, that was not present in previous models of TCP performance. It also presents a dependence between the RTT and the receiver window, that turned out to be a hidden auto tuning mechanism used by clients to adapt the resources dedicated to a given TCP connection to the delay between them and the server. The presence of the empirical parameters (the loss, the delay and receiver window) as direct causes of the throughput comforts us on the pertinence of the model, and the presence of dependencies that were not suggested before highlights the poten-tial of a causal approach and its benefits when compared to previous empirical approaches such as the one of the PFTK model.

In the study of the DNS service impact on CDN user throughput, Chapter 6, our model shows that the DNS service impacts the CDN servers that are contacted by the clients. Again, this dependence had been noticed before and the fact that our model exhibits this dependence supports the validity of our model. In addition, the model detects a dependence between the DNS service and the speed at which the data is sent from the servers contacted by DNS service users. This dependence is somehow unexpected and allows us to propose a direction to improve the performance of clients using the DNS service of their ISP.

Finally, through the different studies that have been presented we could see that the causal approach is very valuable to understand complex systems. Once adopting a formal approach that takes into account the different constraints of the systems being studied, we are able to predict complex scenarios that help us understand the role of the different parameters of a system in its global performance. While other systems, such as WISE, have been designed with the same objective, it is our formal approach that allows us to perform complex predictions while using limited resources. To our knowledge, resorting to and estimating counterfactuals to understand and estimate the causal impact of a parameter (DNS service) on a system performance (throughput) was never done in telecommunication networks before.

7.2. LESSONS LEARNED 153

7.2 Lessons learned

The previous section presented the different steps that constitute our work.

During each of these steps we could draw conclusions and take away important lessons that helped us to design or improve our approach to the causal study of a given system. In this section, we present the different lessons that have been learned during this thesis.

This section briefly summarizes the different lessons that have been learned during the different causal studies of Chapters 4, 5 and 6, presented in Sec-tion 4.5 - 6.6.

One notion that might seem obvious and appears at every step of our work is the important difference between theory and practice. The causal theory is not limited to any system, even the study of direct effects through the single-door criterion, that requires linear dependencies, can be extended to non linear de-pendencies as long as the correct precautions are taken and the correct meth-ods are used. However, to put this in practice requires an important amount of work both, to find the correct methods to apply the theory and to implement these methods for real data.

Towards this end, the different studies and the design of our solution to study telecommunication networks taught us a series of necessary steps that we see as requirement to perform a correct causal study.

The first one being to always know the data: the underlying assumptions or constraints, the way such data was obtained, what is the meaning of each parameter and what scenario they are modeling.

The second one is to always validate the methods of the solution that we adopt.

Once we have found or have designed the methods that meet the constraints exhibited by the data, it is important to perform studies in a controlled environ-ment where we are able to test, validate and calibrate the different methods that will be used in the different studies they were designed for.

Finally, the last point that is inherent to any data mining related problem is the data itself. It is the data that will in the end dictate the accuracy, pertinence, and quality of a study. The amount of information captured by the data is what matters and will condition all the steps of a causal study. Therefore, one should always try to obtain as much data as possible to observe as many situations as possible and be able to capture more mechanisms.

Summarizing the two previous steps, one should collect its data with the objec-tive of maximizing the chances to predict several interventions. This choice re-quires taking into account the different implementation choices that were taken and the different limitations that such choices have, to optimize their utilization and the accuracy of a given study.

A causal approach to the study of Telecommunication networks

Chapter 8

Future work

In this chapter, we present some directions for continuing our work and extend it to further usages or applications. In a first section, we try to provide some insights on the improvement of the method presented and used in the previous chapters. In a second section, we present some domains where the approach presented in this thesis could also be beneficially used.

8.1 Method