Several universities and institutions for higher education in the Netherlands like the University of Twente, the Radboud University in Nijmegen, the University of Utrecht, and Hogeschool Zuyd in Maastricht have used (and still are using) the KM Quest learning environment in courses on knowledge management. Furthermore, the Yeditepe University in Istanbul (Turkey) has used it.
Four staff members of the Universities of Twente and Amsterdam have used the learning environment for research purposes and have published results of several experiments that have been done in the past years (Shostak & de Hoog, 2004, Christoph, Sandberg and Wielinga, 2005, and Purbojo, 2005).
Data show that players are able to perform their task of knowledge manager in the game successfully. Most of them succeed in improving the knowledge household and indicators like market share, customer satisfaction of the fictitious company.
Furthermore, use of the environment leads to significant knowledge gains (Christoph, 2005). However, before players actually start playing they should be prepared and have some introduction and training in using the environment (Purbojo, 2005).
4.8 Summary
In this chapter an extensive description was given of a game (KM Quest) in the domain of knowledge management. Most problems in this domain are ill structured. It is stated that games could be effective settings for training in solving such ill-structured problems because they contain contextualised problems and constraints, offer opportunities for exploration and direct manipulation, and give direct feedback to the learners.
Is it all in the game?
KM Quest is an appealing environment to use in research on elements that should be included in a game to support a reflective selective mode of information processing because it is a rich and complex game. In such games players tend to use an experiential mode of information processing. The game however, contains some elements that are essential for the use of a reflective strategy like monitoring tools, product and process feedback and essential background information.
Furthermore, the game already contains elements that could support the use a reflective strategy like advice. Additional supports can be introduced relatively easy in the game without a large programming effort. The game also has a log functionality which makes it possible to track the behaviour of players and to check whether supports actually are used.
A simulation game to learn knowledge management
5 Focus questions and lessons learned
5.1 Introduction
In Chapter 3 it was hypothesised that in rich dynamic, low transparent environments in which the key variables and their interrelationships are not salient (like in many games) students often use a (low cost) unselective, experiential mode of information processing that is perceptual and event driven. In this mode players acquire new concepts that are applicable in the game, and instances and examples of successful and unsuccessful sequences of actions. When new abstractions, rules or insights are learned this is mostly implicit and context specific knowledge. This type of knowledge is difficult to articulate and to transfer to other situations. While using this strategy, players can be very successful in a game, but in many cases they can not explain how they did it or why they did what they did. To be able to do this and to be able to transfer knowledge gained in the game to other contexts it is necessary that students at certain points in time use a reflective selective mode of information processing.
Students use the experiential mode of processing as long as it is successful in reaching certain goals in a game. If this mode is not successful anymore and they get into an impasse students might change to a (more costly) reflective mode of processing in which they, based on their experiences (in this game and in other games or similar situations) and on resources that are available (in the game environment), try to induce new knowledge. Because the game is complex and contains many cues, one runs the risk that such a reflective strategy does not lead to new explicit knowledge because players do not know which information is essential, can not access the essential information or are not able to formulate and validate hypotheses about relationships between concepts, events and/or actions. In such cases the use of tools (inside or outside the game environment) or the help of other people (other players or a tutor/supervisor) can support a reflective strategy.
Furthermore, tools could also support the use of a reflective strategy in cases where players would not consider this based on their actions so far. For instance when they are still getting closer to the games goals and do not experience an impasse.
In this chapter results are reported from an experiment in which it was investigated whether the availability of certain support tools led to different learning results. This was done by using two versions of the simulation game to learn about knowledge management (KM Quest) that was described in the previous chapter.
In Chapter 3 several instructional elements were described that could support a reflective strategy. Several of these supports are already implemented in the KM Quest environment (see previous chapter). In the game support is given by means of feedback, monitoring tools, just-in-time information, and an advisor functionality.
Furthermore, players have to collaborate in teams. They have to discuss ideas, make choices and have to reach agreement. This can also support a reflective mode of information processing.
The current study focused on two additional elements that could support the use of a reflective strategy also in game situations where players would not consider this strategy based on the cues and information available and on the effect of their actions so far. The reason is that data from evaluations of a KM Quest prototype that was not fully operational (Christoph et al., 2003) showed that most players/teams
Focus questions and lessons learned
were successful in the game in the sense that they managed to improve the knowledge household and general business indicators (market share, customer satisfaction and profit). However, there were no significant knowledge gains. This could imply that students gained implicit or explicit knowledge that was not measured. But it could also indicate that students did not use a reflective mode of information processing and the supports available because they were successful in the game anyhow.
It is hypothesised that the use of a reflective mode could be supported by giving the players additional tasks. There are indications that such tasks are effective in learning with simulations. De Jong and Van Joolingen (1998) concluded that three instructional measures can be seen as holding the promise of positively influencing learning outcomes in scientific discovery learning with computer simulations:
providing direct access to domain information, providing assignments, questions or exercises, and model progression (when the model is sufficiently complex). “for other individual measures (e.g. hypothesis support, experimentation hints, monitoring tools, prediction support) the evidence is not substantial enough to warrant conclusions”.
De Jong, Härtel, Swaak, and Van Joolingen (1996) found that students who were free to choose, used assignments very frequently in a simulation, and that using assignments had a positive effect on the gaining of intuitive knowledge. This seems to indicate that the type of assignments that were used mainly supported an experiential mode and not a reflective mode.
The assignments used in these simulation environments are mostly investigation assignments that prompt the learner to start an inquiry about the relationship between specified variables (Swaak, 1998). Van Joolingen and De Jong (2003), apart from investigation assignments, also distinguish specification assignments that ask students to predict a value of a certain variable, explicitation assignments that ask the student to explain a certain phenomenon in the simulation environment, optimisation assignments that ask students to reach a specified optimal situation, and operation assignments in which a certain procedure is applied.
These kinds of additional assignments are effective in simulations with a limited number of variables, but probably are less effective in relatively complex game environments, with a lot of variables and where the relationships between variables and between variables and user actions are less self evident.
In these kinds of environments assignments that focus attention on aspects that might have been overlooked or taken for granted without mindful processing of information (Reiser, 2002), might be more effective, as well as assignments that ask the learner to make generalisations based on their experiences so far.
In the current experiment a version of the KM Quest game with two additional assignments was compared with a version of the game (see instruments section below) in which these were not implemented. These assignments are:
Focus questions that have to do with the event that occurred in the previous quarter in the game. The goal of these questions is to make the task more problematic by focusing attention on aspects that might have been overlooked or taken for granted without mindful processing of information. For instance: a “solution” is presented and students have to state whether they think it to be good solution.
Formulating lessons learned after a quarter in the game is finished. This lesson has to reflect upon knowledge management problems, strategies and/or processes: “Based on your experiences so far, formulate a lesson learned concerning knowledge management problems, strategies and/or processes that could be of help to others or yourself in the future.”
The assignments were available by means of the Coltec News icon that is on the desk of the virtual office. When players click on the icon a window is opened that
Is it all in the game?
describes the event of the current quarter and displays links to feedback and the additional assignments (see Figure 5-1)
The hypothesis is that students using the version with additional assignments will gain more explicit knowledge while playing the game than the ones who did not have these assignments, because the additional tasks support players to reflect on their own behaviour and that of “virtual” others and furthermore ask them to make implicit knowledge that was gained while using an experiential mode more explicit.
To test this hypothesis the knowledge gain was measured by administering a pre-test before players entered the game environment and a post-pre-test after completing the game. These tests contained items that intended to measure different types of explicit knowledge but also some items that were meant to test implicit knowledge.
To test whether students could apply the knowledge they learned in the game also in different circumstances, a transfer task was given after the post-test was completed.
The transfer task is a case description with events that is situated in a different context.
To investigate whether the additional tasks had an influence on the way students played the game their behaviour in the game was logged in a file together with the data that are generated by the business simulation model. These data are also used to investigate whether the use of tools, like feedback, handbooks or visualisations, was related to learning gains or game performance.
Figure 5-1. Screen dump showing the content of Coltec News with a link to the additional assignments.
Link to additional assignments
Focus questions and lessons learned