HAL Id: hal-02435232
https://hal.archives-ouvertes.fr/hal-02435232
Submitted on 10 Jan 2020
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
A training in visualizing statistical data with a unit square
Andreas Eichler, Carolin Gehrke, Katharina Böcherer-Linder, Markus Vogel
To cite this version:
Andreas Eichler, Carolin Gehrke, Katharina Böcherer-Linder, Markus Vogel. A training in visualiz- ing statistical data with a unit square. Eleventh Congress of the European Society for Research in Mathematics Education, Utrecht University, Feb 2019, Utrecht, Netherlands. �hal-02435232�
A training in visualizing statistical data with a unit square
Andreas Eichler1, Carolin Gehrke1, Katharina Böcherer-Linder3 and Markus Vogel4
1University of Kassel, Germany; eichler@mathematik.uni-kassel.de, carolin-gehrke@web.de
3University of Freiburg, Germany; boecherer-linder@math.uni-freiburg.de
4University of Education Heidelberg, Germany; vogel@ph-heidelberg.de
Recent research yielded empirical evidence for a unit square being a useful visualization of Bayesi- an situations. However, most of the studies in the research field of visualizing Bayesian problem situations were conducted in well-controlled experimental settings with university students as parti- cipants. Therefore, we focus in this paper on a training study with 38 eleventh graders in school about visualizing statistical data with a unit square for coping with Bayesian problem situations.
Firstly, we outline some theoretical and empirical basics concerning research about Bayesian situ- ations and a unit square as a facilitating visualization tool. Afterwards, we present a short training sequence in using the unit square effectively. We report on methods, implementation and results of a pilot study in school. The promising results were discussed at the end.
Keywords: Bayesian reasoning, visualization of statistical data, unit square, school training study.
Introduction
According to Arcavi (2003) it is an important task of mathematics education research to investigate visualizations as specific graphical representations of mathematical objects in a theoretical and em- pirical way. Thus, we focus in our ongoing research about the unit square as a powerful visualiza- tion of Bayes’ formula on theoretical and empirical insights. The unit square is shown in Figure 1 visualizing a Bayesian situation which are situations where the Bayes’ formula could be applied.
After travelling to a far country, you learn that on average 10% of the travelers contracted a new kind of disease during their trip. The disease proceeds initially without any clear symptoms, therefore you don’t know whether you had been infected or not.
You learn that a medical test was developed which has the following characteristics:
80% of infected people (I) get a positive test result (sensi- tivity of the test; +).
15% of not infected people ( ) get also a positive test re- sult (specificity of the test, +).
Finally, you decide to carry out the test and get a positive test result. What is the probability that you had been infected actually?
Figure 1: Bayesian situation and visualization with the unit square
Given for example a Bayesian situation as described in Figure 1 the unit square would look like as displayed in Figure 1 in the right side. All essential information is represented in a numerical and a geometrical way. Thus, a unit square facilitates via different strategies the computation of the essen- tial probability, that is, the probability of being infected given a positive test result
.
Focusing on facilitating strategies in Bayesian situations in detail is the main goal of this paper: We refer to certain visualization features of a unit square which makes it an effective facilitator for cop- ing with Bayesian situations and for adequately applying Bayes’ rule which is a fundamental model for dealing with risk in situations of uncertainty. For this we provide first a brief literature review to discuss the advantages of the unit square in comparison to further visualizations of Bayesian situa- tions. Afterwards we refer to the design and the results of a quasi-experimental training study with 38 students in grade 11 which compares the performance of a treatment group vs. a control group.
Our question for this step in an ongoing research project was whether it is possible to improve stu- dents’ dealing with Bayesian situations by a brief training that is based on representing Bayesian situations by natural frequencies and by a unit square. Implications and conclusions are discussed at the end of the paper.
The unit square as facilitator of Bayesian situations
Despite the Bayes’ rule’s relevance for real life situations, psychological and educational research gained evidence that people often fail when applying Bayes’ rule (Diaz, Batanero, & Contreras, 2010; Gigerenzer & Hoffrage, 1995; Kahneman, Slovic, & Tversky, 1982). In their meta-analysis McDowell and Jacobs (2017) concluded that only about 5% of participants in various studies were able to solve a problem as exemplarily given in Figure 1 without any facilitating strategy.
A widely accepted strategy to increase people’s performance in Bayesian situations is representing the statistical information via natural frequencies instead of using percentages (e.g. Gigerenzer
& Hoffrage, 1995). In Figure 2, the Bayesian situation given in Figure 1 is displayed by using natu- ral frequencies. The meta-analysis of McDowell and Jacobs (2017) provides empirical evidence that by using natural frequencies as the only information format people’s performance in coping with Bayesian situations increases from about 5% to about 25%.
After travelling to a far country, you learn that on average 100 out of 1000 travelers contracted a new kind of disease during their trip. The disease proceeds initially without any clear symptoms, therefore you don’t know whether you had been infected or not.
You learn that a medical test was developed which has the following characteristics:
80 out of 100 infected people (I) get a positive test result (sensitivity of the test; +).
135 out of 900 not infected people ( ) get a positive test result (specifici- ty of the test, +).
Finally, you decide to carry out the test and get a positive test result. What is the pro- portion that you actually have the disease?
Figure 2: Bayesian situation with natural frequencies
A further facilitating strategy is to provide additional visualizations (e.g. Brase, 2009; McDowell
& Jacobs, 2017). However, there are ambiguous research findings about which visualization proper- ties could effectively facilitate dealing with Bayesian situations.
From a theoretical point of view, three properties could be suggested to have a facilitating effect.
The first is to use icons as “real, discrete and countable” objects (Cosmides & Tooby, 1996, p. 33).
However, from a practical point of view (in terms of time costs) this strategy turns out to be hardly realizable in regular classroom instruction where students have to create visualizations. Further-
more, results of our own research suggest that icons actually have an additional, but small facilitat- ing effect compared to a unit square as shown in Figure 2 (Böcherer-Linder & Eichler, 2019).
The second facilitating strategy is to make the nested-sets structure transparent which constitutes fundamentally every Bayesian situation (Oldford, 2003). Sloman, Over, Slovak, and Stibel (2003, p. 302) stated that “any manipulation that increases the transparency of the nested-sets relation should increase correct responding”. Actually, our own research revealed empirical facts that the unit square allows exactly for such a manipulation (Böcherer-Linder & Eichler, 2017, 2019). The subsets that have to be identified for successfully solving a task with the Bayes’ formula are directly neighboured in the unit square (cf. Figure 1) and, by this means, they are transparently visualized.
In particular, in different experimental studies, we yielded empirical evidence that the unit square makes the nested sets-structure of a Bayesian situation more transparent than the commonly used tree diagram and showed further that the unit square is more effective to facilitate dealing with Bayesian situations (Böcherer-Linder & Eichler, 2017). Actually, compared to only using natural frequencies the additional use of a unit square could further increase people’s performance in cop- ing with Bayesian situations from about 25% (cf. above) to about 65%-70%.
A third facilitating strategy is to use area-proportional representations of statistical information (e.g.
Talboy & Schneider, 2017). Going beyond pure numerical information a unit square provides addi- tionally the statistical information by displaying exact graphical proportions of the rectangular sub- plots. When comparing a unit square with a 2x2-table which differs from a unit square regarding the missing area-proportionality we found evidence for the effectiveness of the area-proportionality (Böcherer-Linder & Eichler, 2019). Nevertheless, understanding Bayesian situations is not fully reached by only solving tasks as shown in Figure 1 (Borovcnik, 2012). Moreover, understanding also includes the adequate influence-judging of parameters in these Bayesian situations, that is, the base rate or the conditional probabilities that are called sensitivity and specificity in a medical diag- nosis situation. For this purpose, we also investigated the facilitating effect of a unit square com- pared to a tree diagram by regarding the influence of changing parameter values in Bayesian situa- tions (Böcherer-Linder, Eichler, & Vogel, 2017). In this research, we found a significant supremacy of a unit square compared to a tree diagram. Although we have not yet compared a 2x2-table and a unit square concerning these kinds of changing parameter tasks in Bayesian situations, we hypothe- size based on theory (Eichler & Vogel, 2010) that a unit square is of advantage by means of addi- tional visualizing changing proportions.
The research discussed so far is mostly conducted in well controlled studies without a systematic training. However, training should be understood as a fourth strategy which provides facilitating effects additional to the effective strategies mentioned before. However, these very few training studies reported so far were not focused on facilitating properties of visualizations and, they were mostly carried out with university students as participants (e.g. Sedlmeier & Gigerenzer, 2001;
Talboy & Schneider, 2017). For this reason, we conducted a training study that addressed learning how to adequately deal with Bayesian situations based on using natural frequencies and, in addition, on using a unit square. Referring to a sample of university students, the results of a pilot study sug- gest a positive effect of the training with a unit square (Böcherer-Linder, Eichler, & Vogel, 2018).
In this paper, we present a training study with students of grade 11. The main question in this study
was whether students’ performance in Bayesian situations as well as students’ understanding of Bayesian situations in terms of judging the influence of changed parameters could be improved in a brief training based on representing Bayesian situations by natural frequencies and a unit square.
The training
Step 1: Choice of the sample size
For solving the problem, we first consider the question of what the probabilities mentioned in the text imply for a concrete group of travelling people. We choose a sample size of 1000 people.
We represent the group of 1000 people by drawing a unit square.
Step 2: Construction of the frequency representation
Since 10% of the travelling people contracted the disease during their trip, 100 out of the 1000 people are expected to be infected. 900 out of the 1000 people are expected to be uninfected.
Thus, we divide the unit square in vertical direction for “infected” and “uninfected”
at the ratio of 100 to 900. At the bottom of the narrow rectangle we write “100”
and at the bottom of the broader rectangle we write “900”.
Since 80% of the infected people get a positive test result, 80 out of the 100 infect- ed people are positively tested. Accordingly, 20 out of the 100 infected people are tested and result negative.
Therefore, we subdivide the narrow rectangle horizontally into two parts and write
“80” and “20” into the resulting areas.
Since 15% of the uninfected people get a positive test result, 135 out of 900 unin- fected people get a positive result after testing (because 15% of 900 is 135). Ac- cordingly, the other 765 uninfected people get a negative test result.
Therefore, we subdivide the broader rectangle horizontally into two parts for “posi- tive” and “negative” and write the numbers into the resulting areas.
What needs to be found is the probability that a person with a positive test result is actually infected. Thus, we have to calculate which proportion of the people result- ing positive is actually infected. For this aim, we surround all positive people with a dashed line in the unit square and emphasize with grey color all of them, that are infected.
We read out the following numbers:
Number of infected and positive to the test: 80 Number of all positive to the tested: 80+135=215 We calculate:
This proportion corresponds to the requested probability of 37%.
Figure 3: First intervention in the training study (“picture-formula”, cf. Eichler & Vogel, 2010) Due to an ecological validity, we presented the problems in the probability-text-version (see Figure 1) and trained the students to translate the probabilities into natural frequencies and, secondly, to graphically represent the statistical information in the process of problem solving. The training had
0,37
“picture formula”
two phases: phase 1 (10 minutes) contained three steps. It was a worked example showing how to solve the problem of Figure 1 with the help of the unit square (Figure 3; translated from German).
Phase 2 (10 minutes) included an exercise that was structurally identical to the worked out example (Figure 2 and Figure 3) but had another context (Trisomy). An oral explanation and a presentation of the correct solution were given. Even though it was possible to draw a unit square true to scale in this case, we only showed a rough drawing of a unit square, since we wanted to enable the partici- pants to work with the visualization as a thinking tool for problem solving which did not necessarily imply a precise drawing.
Finally, we added a brief exercise (5 minutes) on the change of parameters. For this, we only show the instruction referring to the change of the base rate (Figure 4).
3. Change of the base rate
The base rate indicates the proportion of an individual having trisomy (5% of the children of women of age 45). An increase of the base rate results in an in- crease of the area of the thin rectangle on the upper left side of the unit square and, accordingly, a decrease of the area of the rectangle on the upper right side.
If the base rate increases to 10%, there would be 90 true-positive people and 90 false-positive people. The change of the base rate changes the proportion as shown below:
Actually, the result shows a considerable change of the probability.
Figure 4: Part of the training to changed parameters
Method
Our sample included 38 students in grade 11, from two classes. By choosing two different classes of students which were not grouped randomly we conducted a quasi-experiment. The class to which we administered the training included 22 students. The class that represented the control group con- sisted of 16 students.
Figure 5: Design of the training study
The design of the quasi-experiment is shown in Figure 5. Since there were not all of the students present in the three phases shown in Figure 5, our analysis is based on 16 students (treatment group) and 13 students (control group).
Both the pre-test and post-test consisted of two Bayesian situations. One of the situations in both tests was the same, the other situation differed to prove whether repeating a task has an effect on performance in Bayesian situations. In the pre-test, one situation included only a performance-task and the second situation included both a performance task and a task in which the influence of
changed parameters was to be estimated. In Figure 6 we show the task that was identical in both tests and include both forms of tasks (translated from German).
There are tests for diagnosing people if they have infectious diseases like measles or scarlet. Concerning such an infec- tious disease and the corresponding test the following information is given:
The probability of having the infectious disease is 2%. Given there is a patient having the disease, the test yields in 90% of all cases a “positive” result, which means it indicates correctly the infectious disease. Given there is a non- infected patient, the test shows in 5% of all cases also a “positive” result, which means it indicates the infectious dis- ease by mistake.
a) What is the probability of a patient having actually the disease given a “positive” test result?
b) How changes the probability of section a) if the probability of having the disease is higher?
Figure 6: A test item (translated from German)
Results
Regarding the few items the values of Cronbach’s alpha of 0.63 (pre-test) and 0.85 (post-test) ap- pear to be good. Thus, we summarized the scores in each test referring to the performance test-items a).
Mean Std-dev n
Control Pre .08 .277 13
Treatment Pre .19 .544 16
Sum .14 .441 29
Control post .38 .768 13
Treatment Post 1.75 .577 16
Sum 1.14 .953 29
Figure 7: Descriptive results of the training concerning item a)
We applied a mixed ANOVA (within-factor: performance in the pre-test and post-test; between factor: group) to investigate the training effect in the treatment group compared to the control group. According to the robustness of an ANOVA (Schmider, Ziegler, Danay, Beyer and Bühner, 2010), we did not regard non-normality and differences in variances. In a descriptive way, Figure 7 (left) reports the results referring to the pre-test and the post-test. Figure 7 (right) visualizes a clear effect of the treatment referring the means.
Mean Std-dev n
Control Pre .38 .506 13
Treatment Pre .44 .512 16
Sum .41 .501 29
Control post .77 .388 13
Treatment Post .94 .171 16
Sum .86 .296 29
Figure 8: Descriptive results of the training concerning item b)
As expected, the ANOVA yields a significant effect ( =20,733, =0,000) of the treatment with a strong effect ( 2=0,434). Regarding item b) the results were not as expected (Figure 8). Actually, also the control group has an increased average of correct solutions and the differences between the treatment group and the control group are not significant concerning the ANOVA ( =0,424, =0,520, 2=0,015). By contrast, there is a significant main effect for both groups regarding the difference between the pre-test and the post-test ( =24,938, =0,000, 2=0,480).
Discussion
Regarding the results of the performance tasks it could be stated that a training with the unit square turns out to be helpful for students in school. This finding is in line with the results of our recent study in school investigating the effect of the visualization (cf. Vogel & Böcherer-Linder, 2018).
Furthermore, this is particularly emphasized within a practical point of view, a unit square’s effec- tive use can be trained within a very short time (in Germany 45 minutes corresponds to one lesson).
Regarding the tasks of estimating the effect of changing parameters it should be mentioned that these kinds of tasks were estimated to be more difficult by going beyond the demand of only build- ing a ratio as the Bayes problem’s solution. However, most of the students in both groups were able to find the correct solution, even in the control group and, thus, the significant difference between pre- and posttest indicates a learning progress. Particularly the missing difference between the treatment group and the control group could firstly be explained by the high guessing probability of one third, or even one half, if the question “How changes” is interpreted as either increasing or de- creasing. Moreover, maybe the students intuitively guessed that the conditional probability would be higher if the base rate increased without analyzing the parameter dependence. However, we ex- pect tasks which necessitate an analysis of the parameter dependency to be trainable as successfully as the performance tasks. This effect is focused on in subsequent studies being conducted currently.
References
Arcavi, A. (2003). The role of visual representations in the learning of mathematics. Educational Studies in Mathematics, 52(3), 215–241. doi:10.1023/A:1024312321077
Böcherer-Linder, K., & Eichler, A. (2019). How to Improve Performance in Bayesian Inference Tasks: A Comparison of Five Visualizations. Frontiers in Psychology, 10, 297.
doi:10.3389/fpsyg.2019.00267
Böcherer-Linder, K., Eichler, A., & Vogel, M. (2018). Visualizing statistical information with unit squares. In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceed- ings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan. Voorburg, The Netherlands: International Statistical Institute. iase-web.org
Böcherer-Linder, K., Eichler, A., & Vogel, M. (2017). The impact of visualization on flexible Bayesian reasoning. Avances De Investicación En Educación Matemática - AEIM, 11, 25–46.
Böcherer-Linder, K., & Eichler, A. (2017). The Impact of Visualizing Nested Sets. An Empirical Study on Tree Diagrams and Unit Squares. Frontiers in Psychology, 7(1186), 241.
doi:10.3389/fpsyg.2016.02026
Borovcnik, M. (2012). Multiple perspectives on the concept of conditional probability. Avances de Investigación en Didactica de la Matemática, 2, 5–27. doi: https://doi.org/10.35763/aiem.v1i2 Brase, G. L. (2009). Pictorial representations in statistical reasoning. Applied Cognitive Psychology,
23(3), 369–381. doi:10.1002/acp.1460
Cosmides, L., & Tooby, J. (1996). Are humans good intuitive statisticians after all?: Rethinking some conclusions from the literature on judgment under uncertainty. Cognition, 58(1), 1–73.
doi:10.1016/0010-0277(95)00664-8
Diaz, C., Batanero, C., & Contreras, J. M. (2010). Teaching independence and conditional probabil- ity. Boletin De Estadistica E Investigacion Operativa, 26(2), 149–162.
Eichler, A., & Vogel, M. (2010). Die (Bild-)Formel von Bayes [The (picture-)formula of Bayes].
PM - Praxis der Mathematik in der Schule, 52(32), 25–30.
Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction:
Frequency formats. Psychological Review, 102(4), 684–704. doi:10.1037/0033-295X.102.4.684 Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics
and biases. Cambridge: Cambridge University Press.
McDowell, M., & Jacobs, P. (2017). Meta-Analysis of the Effect of Natural Frequencies on Bayesi- an Reasoning. Psychological Bulletin. Advance online publication. doi:10.1037/bul0000126 Oldford, R. W. (2003). Probability, problems, and paradoxes pictured by eikosograms. Retrieved
from http://www.stats.uwaterloo.ca/~rwoldfor/papers/venn/eikosograms/examples/paper.pdf Schmider, E., Ziegler, M., Danay, E., Beyer, L., & Bühner, M. (2010). Is It Really Robust? Meth-
odology, 6(4), 147–151. https://doi.org/10.1027/1614-2241/a000016
Sedlmeier, P., & Gigerenzer, G. (2001). Teaching Bayesian reasoning in less than two hours. Jour- nal of Experimental Psychology: General, 130(3), 380–400. doi:10.1037//0096-3445.130.3.380 Sloman, S. A., Over, D., Slovak, L., & Stibel, J. M. (2003). Frequency illusions and other fallacies.
Organizational Behavior and Human Decision Processes, 91(2), 296–309. doi:10.1016/S0749- 5978(03)00021-9
Talboy, A. N., & Schneider, S. L. (2017). Improving Accuracy on Bayesian Inference Problems Using a Brief Tutorial. Journal of Behavioral Decision Making, 30(2), 373–388.
doi:10.1002/bdm.1949
Vogel, M., & Böcherer-Linder, K. (2018). The effect of visualizing statistical information in Bayes- ian reasoning problems. In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan. Voorburg, The Netherlands: International Statistical Institute. iase- web.org
