Decision making in dynamic work situations: building a framework for medical alerts in computerized drug prescription

(1)

Thesis

Reference

Decision making in dynamic work situations: building a framework for medical alerts in computerized drug prescription

WIPFLI, Rolf

Abstract

As part of electronic prescription systems medical alerts are a powerful decision support for physicians. They can warn physicians in case of drug interactions or other adverse events.

However, a large part of medical alerts is not taken into account. The aim of the present thesis is to investigate the impact of medical alerts on physicians. We address the question with three different studies carried out at the University Hospitals of Geneva. The first study evaluates physicians' satisfaction with the deployed system. The second study evaluates the work context to learn about the use of medical alerts in a real work situation. Finally, we evaluate in an experimental study physicians' interaction with an adapted prototype of a prescription system. Among other results we found that physicians only consider alerts in unfamiliar medical situations. The prototype where alerts are centralised in a table seem to be better adapted to this behaviour.

WIPFLI, Rolf. Decision making in dynamic work situations: building a framework for medical alerts in computerized drug prescription. Thèse de doctorat : Univ. Genève, 2012, no. FPSE 516

URN : urn:nbn:ch:unige-239570

DOI : 10.13097/archive-ouverte/unige:23957

Available at:

http://archive-ouverte.unige.ch/unige:23957

Disclaimer: layout of this document may differ from the published version.

(2)

Sous la codirection de Mireille Bétrancourt et Christian Lovis

Decision making in dynamic work situations:

Building a framework for medical alerts in computerized drug prescription

THÈSE Présentée à la

Faculté de psychologie et des sciences de l’éducation de l’Université de Genève

pour obtenir le grade de Docteur en Psychologie par

Rolf WIPFLI de Erstfeld UR

Thèse N° 516

Section de Psychologie

Genève Octobre 2012

(3)

(4)

Acknowledgements

The three and a half years while working on the thesis consisted in a large part of reading papers and putting ideas on other papers. At the same time I am aware that it would not have been possible without the support of several persons. My thanks go to

Mireille who had always the right advice in times when I was stuck. I would neither have started nor finished my thesis without her.

Christian who shared his visions with me and let me take part of his experience in the field of medical informatics. He helped me to write the present thesis in perfect work conditions.

Jürgen Sauer and David Sander who were kindly willing to be in my thesis commission and Sylvia Pelayo who agreed to be a member of the jury.

Douglas, Philipp, Alberto, Georges, and Louiselle who helped me during the three years with advice from their fields of knowledge.

The members of Tecfa and SIMED for their pleasant aquaintance and collaboration.

My family for knowing that they are there when I need them.

(5)

Abstract

As part of electronic prescription systems medical alerts are a powerful decision support for physicians. They can warn physicians in case of drug interactions or other adverse events.

However, a large part of medical alerts is not taken into account. The aim of the present thesis is to investigate the impact of medical alerts on physicians. We address the question with three different studies carried out at the University Hospitals of Geneva. The first study evaluates physicians’ satisfaction with the deployed system. The second study evaluates the work context to learn about the use of medical alerts in a real work situation. Finally, we evaluate in an experimental study physicians’ interaction with an adapted prototype of a prescription system. Among other results we found that physicians only consider alerts in unfamiliar medical situations. The prototype where alerts are centralised in a table seem to be better adapted to this behaviour.

(6)

Résumé

Faisant partie des systèmes de prescription informatisée, les alertes médicales sont un outil d’aide à la décision puissant pour les médecins. Elles les interpellent en cas d’interaction médicamenteuse ou d’autres contrindications. Cependant, la plupart des alertes ne sont pas prises en compte pour la prescription effective. Le but de cette thèse est de comprendre l’impact des alertes sur le médecin. Nous abordons la question avec trois études effectuées aux Hôpitaux Universitaires de Genève. Nous évaluons la satisfaction avec le système déployé, nous effectuons une analyse de l’activité afin de connaître l’utilisation des alertes dans un vrai contexte de travail, et finalement nous évaluons expérimentalement l’utilisation d’un prototype de système de prescription adapté. Entre autres résultats, nous avons trouvé que les médecins considèrent les alertes seulement dans des cas médicaux non familiers. Le prototype où les alertes étaient affichées centralisée dans un tableau semble être mieux adapté à ce comportement.

Un résumé plus complet en français est disponible en annexe (chapître A.1).

(7)

Zusammenfassung

Als Teil elektronischer Verschreibungssysteme bieten medizinische Warnungen den Ärzten eine leistungsfähige Unterstützung bei der Entscheidungsfindung. Sie lenken ihre Aufmerk- samkeit auf Arzneimittelinteraktionen oder andere Nebenwirkungen einer Verschreibung.

Der Grossteil der Warnungen wird jedoch nicht für die Verschreibung berücksichtigt. Die vorliegende Dissertation will die Wirkung von medizinischen Warnungen auf Ärzte untersuchen. Wir untersuchen diese Frage mit drei Studien, welche an den Genfer Universitäts- spitälern durchgeführt wurden. Eine Studie erhebt die Zufriedenheit der Ärzte mittels einer Umfrage. In einer zweiten Studie wurde eine Arbeitsanalyse durchgeführt, um die Nutzung von medizinischen Warnungen in einem realen Arbeitsumfeld zu untersuchen. Schliesslich untersuchen wir mit einer experimentellen Studie die Interaktion der Ärzte mit einem ange- passten Prototypen eines elektronischen Verschreibungssystems. Neben weiteren Resultaten konnte unsere Studien zeigen, dass Ärzte Warnungen nur in ungewohnten medizinischen Fällen beachten. Der Prototyp, in dem die Warnungen zentralisiert in einer Tabelle angezeigt wurden, scheint dieses Verhalten besser zu unterstützen.

(8)

1 Introduction

In many working environments jobs are getting increasingly more complex and ask for spe- cialisation. The health care sector is a prime example for this development. Some decades ago the involved personnel in a hospital per patient consisted of a physician, a nurse, and a secretary. Today, a high number of specialists are implicated in the treatment of a patient.

In order to ensure that communication and division of work between the different actors is possible in an organisation there is an increasing need for information and communication technology.

Medical informatics is concerned with the development of solutions to improve the division of work and communication. But there are more benefits that informatics can offer to health care. It can provide the physician information that was unavailable before. We could name powerful imagery that facilitates diagnosis, or advanced analysis of laboratory results that were technically not feasible before.

In all these cases, the system offers different views on the medical issue in order to support the diagnosis. How physicians are using the data and to what conclusion they come is usually not part of the system design. Although, even for specialised health care professionals, the high number of information and the multitude of possible treatments render decision making more and more difficult. Even a highly experienced physician cannot know all diseases, all treatments, all drugs, their risks and interactions by heart. In order to diminish the charge on physicians, more advanced systems do not only communicate factual information, but interactively support physicians’ decision making.

So, when we look at a hospital, we see a complex work environment whose objective is to treat a large number of diseases and injuries, by application of a large number of possible treatments, in a team of highly specialised collaborating health care professionals, and with the support of a multitude of computer systems.

The exigence on a single worker is high. Even if computer systems aim at facilitating work, they sometimes on the contrary add complexity. We all know from personal experience that some systems are easier to use than others. The field of Human-Computer Interaction (HCI) evaluates systems for their user friendliness and proposes methods to improve it. User friendliness, or the more technical term usability, is defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO EN 9241: Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11, 1998). Even if HCI is interested in how information is presented to the user, the impact of changes to the interface goes deeper than the graphical design. The way the interaction with users is designed decides whether the system does support them as intended. Only a well designed system helps the user to accomplish the tasks effectively, efficiently and in a satisfying way. As Raskin is often cited (2000): “As far as the customer is concerned, the interface is the product.”

There often is a discrepancy between how the system designers expect the system is used

(15)

and how users actually use the software once it is deployed. In order to keep the expanse of the gap as small as possible, a system should be tested as early as possible and all along the development phase with intermediary versions. The goal is that a system is developed from the beginning by integrating the user perspective along the project with a User-Centred Design (UCD) method. Some initiatives have tried to combine software engineering methods with methods from the HCI field in a health care context (Bernonville, Kolski, Leroy, &

Beuscart-Zéphir, 2010; Beuscart-Zéphir, Elkin, Pelayo, & Beuscart, 2007).

The medical domain shares with other working environments its dynamic characteristic.

Even if there are guidelines for diagnosis and treatments physicians are confronted with a degree of uncertainty in their work. Physicians may state an initial hypothesis regarding the symptoms of a patient. However, diagnosis and treatments have to be adapted when new information about the patient is available and it must be adapted to the development of the symptoms. A system that support physicians in their daily work must account for this fact.

A system that is designed with a static concept of knowledge may proof dangerous, for it will fail in a situation where there is insufficient data or when the decision model is not apt for an unforeseen situation. HCI methods may fall short in the use for dynamic systems as they usually test with predefined tasks. As it has already been done in the field of air traffic controlling (Bisseret, 1995), controlling of nuclear plants (Alengry, 1988), and air plane piloting (Amalberti & Deblon, 1992), the field of medicine needs research that covers the whole activity in order to improve the utility, usability and as a consequence the safety of these systems.

In the following sections, we present the different parts of an information system in a hospital as it is used for example at the University Hospitals of Geneva (HUG). HUG is the institution where the research for this thesis has been conducted. It is a teaching hospital with 2000 beds and 15.000 electronic prescriptions a day.

1.1 Clinical information systems

Since the beginning of medicine, physicians keep note about their patients’ diagnosis and treatments. It is an essential part of medicine to track the development of a patient’s health status over time and to track the impact of treatments. Medical records contain the physicians’ reasoning and actions and are usually addressed to themselves for later consultation.

When several health care providers are involved in patient’s care the use of paper based medical records has three major inconveniences:

• The medical record and its contained information are only available at one place at a time.

• The handwritten notes of a physician risk to be illegible.

• The notes tend to be ambiguous as physicians writes only facts down that are useful for themselves (context dependant).

These issues can be addressed by information technology. In health care there are several initiatives to digitise the large amount of medical information and knowledge. By transform-

(16)

ing the medical records in electronic records the data would be ubiquitous and available in a standardised and easy to read format.

Clinical Information System (CIS) were introduced in order to replace paper based medical records. Their main advantage compared to paper based solutions lies in the availability of medical data independent of place and time as well as the traceability of patient centred medical data. Also, in an economic perspective, CIS are regarded as one potential remedy to control the unbroken increase of health care costs (Hillestad et al., 2005). Moreover, they might shorten the length of stay, decrease medical errors and improve compliance with guidelines (Kuperman & Gibson, 2003).

1.2 Computerised physician order entry systems

As part of CIS, CPOE provides electronic prescription of treatments like drugs or other therapies. The benefits of CPOE are numerous. First, the system improves the legibil- ity and completeness of orders which simplifies the work of nurses. Also, nurses do not need to transcribe the orders which has shown to be a major source of errors (Bates et al., 1995). Further, the CPOE system can be accessed on any workstation in the hospital and prescriptions are traceable which has become a legal requirement. Finally, there are shorter turnaround times. The most frequent disadvantages are the time-consuming and problematic user-system interactions (Khajouei & Jaspers, 2008) and the enforcement of predefined but too strict relationships between different clinical tasks and providers (Niazkhani, Pirnejad, Berg, & Aarts, 2009).

Indeed, the introduction of CPOE changes work processes considerably (Weir et al., 2007;

Miller, Waitman, Chen, & Rosenbloom, 2005). For instance, the conjoint medical round of nurses and physicians where ambiguity of orders could be sorted out is often replaced by a communication mediated by the system. As a consequence, the physician-nurse collaboration and communication may suffer (Beuscart-Zéphir et al., 2004).

1.3 Clinical decision support systems

Clinical Decision Support System (CDSS) are considered to be a potent tool to avoid potentially harmful medical prescriptions. According to Robert Hayward CDSS are “systems that link health observations with health knowledge to influence health choices by clinicians for improved health care”. An even more specific definition is that a CDSS is “any electronic system designed to aid directly in clinical decision making, in which characteristics of in- dividual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration” (Kawamoto, Houlihan, Balas, & Lobach, 2005).

Shortliffe identified five dimensions to assess novel CDSS (1987):

• System’s intended function: does the system produce a diagnosis (what is true) or does the system recommend propose steps for treatment (what to do).

• Mode for giving advice: does the system is giving advice on request (passive system) or is it giving advice as soon as the data is available (active system).

(17)

• Consultation style: Either the system follows a consulting model, where the system is supporting the physician by giving advice based on available data. In contrast, a system following a critiquing model interacts as soon as the physician has completed the task. The system will check if the given order is in the range of proposed actions.

• Underlying decision-making process: Having a base of clinical information, several methods can be used to provide decision support: Bayesian statistical models, case- based decisions, knowledge-based expert systems, etc.

• Human-computer interaction: How is the system embedded in the daily work process.

Is the system efficient, effective and satisfying in use?

One major goal of CDSS integrated in CPOE is to reduce Adverse Drug Event (ADE), a major problem of drug therapy (Kaushal, Shojania, & Bates, 2003; Kohn, Corrigan, &

Donaldson, 2010). ADE are undesired effects of drug application, may it be preventable or not. There are several use cases how CDSS could prevent ADE (Rochon et al., 2006).

As we will discuss more in the upcoming section, CDSS support is not always used at its potential. In a white paper, Teich et al. (2005) propose different strategies how to promote the use of CDSS in CPOE. Among others, standardisation, financial and legal incentives, advancements of system capabilities are proposed. Further, the authors propose a classification for capabilities that have to be included in basic and advanced decision support. For example, on a basic level the system should provide among other elements alerts that inform about potential interactions of drugs. On an advanced level, they system should include in addition alerts that warn against a drug prescription when laboratory results contraindicate the use of them. Similar initiatives to classify decision support systems are conducted by Wright et al. (2011), Berlin et al. (2006) and Kuperman et al. (2007).

1.4 Outline of the thesis

Following you will find an outline of the thesis:

Chapter 2 System-centred approach: The system centred approach is concerned with improving existing systems by changing their characteristics and evaluate their impact on users. The D&M Information Success Model and the Technology Acceptance Model (TAM) are presented. Then, a selection of research on the impact of CPOE on physician’s behaviour and patient outcome is discussed. Also some existing proposals to improve the impact of CPOE and medical alerts are presented. Further, classifications of CDSS and a framework for medical alerts is presented that will facilitate the design and implementation part in the present thesis. Finally, we present log file analysis, the most common method to evaluate the system’s impact in a system-centred approach.

Chapter 3 Human-centred approach: Whereas the system-centred approach is characterised by a top-down approach, where the system is the primary object of research, the human-centred approach is a bottom-up approach where the implication of human factors is the starting point. We introduce human vision, perception, and attention.

(18)

A section is dedicated to eye tracking, a method to indirectly measure attention and perception. Later theoretical models of decision making are introduced where we also make reference to medical decision making. Some judgement heuristics and biases are introduced that make the need for effective decision support evident. A major part of the thesis is the presentation of the human-computer interaction models that will serve us for our own studies. Mainly, we present models of the human action cycle and its interruption by external events. Moreover, a multitude of HCI methods are discussed that will help us to evaluate users interaction with systems and find ways to improve it.

Chapter 4 Activity-centred approach: The activity-centred approach is in some way a development of system- and human-centred approaches. It integrates the fact that systems are used in a work context with other collaborators and technical systems. The term activity is introduced. Also, theories from Naturalistic Decision Making (NDM) which integrate the notion of activity are presented. As part of the theoretical part, the Skill-Rule-Knowledge (SRK) taxonomy and the abstraction hierarchy model are introduced which represent a major contribution to our system design. Finally, methods derived from the activity-centred approach are introduced.

Chapter 5 Research questions: Based on our review of research on CPOE and CIS and based on the contributions of system-, human-, and activity-centred approaches, we outline the scope of our own research. We aim to investigate the use of medical alerts in real work contexts and develop and test an improved alert system. Several research questions are asked in this chapter.

Chapter 6 Evaluation of present clinical information system: In this chapter two studies are presented. One evaluates the physicians’ satisfaction with the implemented CIS at the HUG. The second study uses ethnographic methods at the same hospital to evaluate how CPOE and medical alerts are integrated in work flow during patient visits. Our research is based on human-centred and activity-centred methods.

Chapter 7 Design and evaluation of novel interface: Based on findings provided by prior research presented in the theoretical part of the paper and based on our own studies in the requirements analysis phase, we propose and explain a novel approach to present medical alerts. The design leads to a prototype of an alerting system that implements the novel approach.

Further, the technical aspects of the prototype development are discussed. We distinguish between the software design where the data models are defined and the prototype development where we describe the technologies that were used to implement the software design.

Finally, we present the major contribution of our thesis. We evaluate the prototype that we have developed. On one hand we test using a cognitive experimental approach how the cognitive models may contribute to an improved alerting system. On the other hand we use a heuristic usability approach to identify usability issues in order to improve further the alerting system.

(19)

Chapter 8 General discussion: The general discussion will link our research questions with the results of the three studies. The discussions of the two studies in the requirements analysis part and the studies in the experimental part are reviewed and put into a larger context. We discuss how our research can be positioned in relation to previous research.

Chapter 9 Conclusion: The main contributions of the thesis are revisited. It discusses future research that is necessary to find better solutions to evaluate and improve medical alerting systems.

(20)

Part I

State of the art

(21)

(22)

In the partstate of the artwe distinguish between system-centered, human-centered, and activity-centered approaches. This distinction proofs to be helpful for keeping apart different concepts that are used within these approaches. We classify theories and methods to system- centered approaches where the system is the given factor and its impact on users is measured.

We classify theories and methods to human-centered approches where the study of cognitive factors influences the system design. Human-centered approaches examine users doing predefined tasks with a system. Finally, we classify theories and methods to activity-centered approaches where cognitive and social factors influence system design. In activity-centered approaches the user is confronted with uncertain conditions and the tasks are not defined in advance.

(23)

(24)

2 System-centred approach

In a system-centred view, a system is composed of the user, the task and the artefact. The goal is to enhance the users’ performance by supporting their work with an artefact (Norman, 1991). The success of such an intervention is measured by the difference of performance that has been achieved with the artefact compared to the previous solution.

I present the system-centred view in comparison with the human-centred and the activity- centred view in the following chapters. In each of them, I present theories that emerged from these approaches followed by methods how to evaluate the system according to the theories. The distinction between the three approaches has also been discussed in an earlier publication (Wipfli & Lovis, 2010). Some text passages might be taken from this publication.

2.1 Alert cycle

Regarding the system perspective level Calvitti and Lenert (2006) propose a model of alert management which defines three stages: (1) a detection state where medical data is captured and classified; (2) the dissemination stage where the alert is transmitted to the concerned medical personnel and finally (3) the presentation stage where the alert is presented to the user taking into account human factors. Hsieh et al. (2004) propose an integrated feed-back system to evaluate the reasons of overriding behaviour and providing an interface to monitor the number and type of overridden alerts (Zimmerman, Jackson, Chaffee, & O’Reilly, 2007).

Subsequently reviewers would evaluate the feedback in order to ameliorate the system. In Fig. 2.1 we propose our own alert cycle which takes into account the two aspects of an alerting system: the different stages in alert handling and the feedback loop for improving utility and usability of the alert system. According to this model, a complete alerting system is providing means for the definition of rules, the generation of rules, the management of rules, the alerting of the user, the evaluation of user actions and finally the correction of the alert. Two parties are involved in this cycle: the work group that evaluates and defines alerts and the physician for whom the alert is displayed. We argue that the application of this alerting system model would improve the specificity and the sensitivity of alerts.

2.2 What is an alert?

An alert is a system feedback that informs the user about a potential risk. The alert can be visual, acoustic (Bourgeon et al., 2006) or may even be communicated by a haptic interface (Hayward, Astley, Cruz-Hernandez, Grant, & Robles-De-La-Torre, 2004).

In the present thesis, the term alert or medical alert is considered to be a visual alert. In the context of Clinical Information System (CIS), an alert is defined as “Information pertaining to a subject of care that may need special consideration by a health care provider before making a decision about their actions in order to avert an unfavourable health care event, or

(25)

manage alerts

generate alerts

definition of rules

alert user

evaluate user actions

Lovis, 2009

User interface (physician) Expert interface

System generated correct

alerts

Figure 2.1: System’s view on alert cycle

relate to the safety of subject or providers, or pertain to special circumstances relevant to the delivery of care.” (OpenEHR¹).

Research on medical alerts usually concerns feedback on therapeutic decisions by a clini- cian. However, physicians are confronted with other alerts in their daily practice. There are numerous reminders, for instance for signing a prescription, to consult newly arrived laboratory data, see a patient, etc. The aim of reminders and alerts is the same. They seek to guide the physicians’ attention to an information they might have been unaware of.

In the same perspective, Krall et al. (2001) defined four categories of alerts:

• Alerts: drug-allergy, drug interaction information when a drug is ordered

• Order messages: alternative drug or procedure recommendations in response to the placement of an order

• Reminders: health maintenance or disease state guideline recommendations

• In-Basked-Notifications: just posted lab result or a telephone message relating to a patient one saw yesterday

When constraining again our focus on drug related alerts, we find classifications that are based on pharmaceutical facts as they can be found on a package insert of a drug. A distinction is made between basic alerts that are based solely on information of the drug prescription, and advanced alerts that include information of the patient record. In the following two lists, representatives of the two kind of alerts are given (Kuperman et al., 2007; Schedlbauer et al., 2009):

basic alerts:

1http://www.openehr.org/

(26)

• drug doses

• drug route administration

• drug frequencies advanced alerts:

• drug allergy checks

• drug-laboratory value checks

• drug-drug interaction checks

• reminders about corollary orders

• drug guidelines

Other categorisations are possible. An important characteristic of alerts is the novelty of the content. Some information is likely to change over time, other information is likely to never change. On a scale from 1 to 5, an alert might be moredynamic(1) orstatic(5) (Iten A., personal communication, 10 August 2010). Subsequently an example for each level of novelty:

1. As learned recently, a drug is contaminated and must not be used.

2. A newly introduced drug has to be watched.

3. The prescription of a drug is not recommended in the given situation, for instance in the case of an influenza crisis, other drug prescription

4. A new procedure for drug medication has to be followed.

5. A patient wears a prosthesis and drug dosage has to be adapted.

So far, we have discussed the content of an alert. An alert however has two additional important characteristics: the severity and the urgency of an alert. Regarding the severity, Weingart et al. (2003) proposed three levels for Drug-Drug Interaction (DDI) alerts.

1. High severity, strong evidence 2. Moderate severity, strong evidence 3. High severity, moderate evidence

This scale illustrates an important fact in medicine: even if evidence based medicine is applied medical knowledge is rarely absolute. Knowledge in the medical domain is subject to constant change and development. Thus, the severity of a potential risk can be supported by a large body of evidence or might be based on some recently stated hypotheses.

The factor urgency is also an inherent characteristic of any alert. In a real work context physicians are often subject to time pressure. It is unrealistic to assume that physicians

(27)

can respond to every alert immediately. They will prioritise their interventions and react to alerts in function of their other tasks. Even if the alert does not give an indication of the urgency physicians will attribute it to the alert. The description so far of alert attributes is not complete. A comprehensive list of attributes of alerts will be given in the chapter Dimensions in CDSS and alerts.

Even if the potential of medical alerts is rarely contested, as we see later, a major problem of medical alerts in CPOE is their low impact on physicians’ prescription behaviour. More than half of the alerts are overridden without any action taken. In order to improve the beneficial effect of medical alerts, several approaches have been undertaken.

It is generally agreed that alert systems have to be better adapted to the needs and work processes of prescribing physicians. If alerts would be better timed, more specific and displayed in a user-friendly way, they would act as an even more powerful decision support system than today.

With the present thesis we want to respond to the problem of the large number of alerts that are ignored by the physicians. We do not try to make medical alerts more specific – certainly a necessary and important task – but we rather look at the human factors that needs to be considered in the research on medical alerts. Cognitive and ergonomic determinants may give us some insight in how medical alerts should be presented so that they are perceived without being more interruptive than necessary.

Alert presentation was identified to be a number one important factor in a retrospective review of DDI alerts in three hospital sites (Seidling et al., 2011). The authors based the scoring of the alert presentation quality on the review by Phansalkar et al. (2010).

In the European project Patient Safety through Intelligent Procedures (PSIP)² comprehensive research has been conducted on alerts in CPOE (Hackl et al., 2011; Nø hr et al., 2011). The goal of PSIP is the reduction of Adverse Drug Event (ADE) due to human factors. A large part of the project was aimed at contextualising alerts in order to render them more specific to physicians’ needs. Several prototypes were developed and tested during the project and the final phases of the project were engaged with the evaluation of the impact of these systems. However, the research did not aim to explore experimentally different ways to present medical alerts. Therefore, we think that the present thesis is complementary to the research done in the PSIP project.

The primary goal of the present thesis is to investigate the presentation of alerts in CPOE system. However, the alerts in the medical field cannot be reduced to their use in CPOE.

A requirement to an alert system is that it offers a coherent way of presenting alerts, independent from the source of the alert. In this respect, I will present some research which has been done on the subject of CIS and will later cover research on CPOE and Clinical Decision Support System (CDSS).

2.3 Models of Information System Success and Technology acceptance Even if the benefits of electronic systems (or artefacts) in health care have been identified the introduction of a CIS does not go without obstacles. Medical personnel has to be instructed

2http://www.psip-project.eu/

(28)

how to use CIS. Their work procedures may change considerably with the introduction of such a comprehensive system. Also, data entry can take considerably longer when using an electronic system compared to a paper based system. Physician’s acceptance of such a system is therefore not guaranteed.

Research has been conducted to identify the determinants of a successful introduction of CIS. We could identify two types of theories on a system-centred level. The first theories are concerned with showing the correlations of different dimensions of a Information System (IS). The aim is to construct a model that finally shows how these dimension are correlated with the dimension of IS “success”.

The D&M model of IS Success (DeLone & McLean, 2003) falls into this category. Ac- cording to the authors IS Success is depending on multiple dimension that are in correlation as Fig. 2.2 shows.

Information Quality

Use

System Quality

Service Quality

Intention to use

SatisfactionUser

BenefitsNet

Figure 2.2: DeLone and McLean information success model adapted from DeLone &

McLean, 2003

Besides the temporal aspect, DeLone and McLean also state that there is a causal relation between these factors. This view has been criticised. The authors clarify in their more recent model that causality has to be viewed in the sense of necessary but as not sufficient to the outcome. In consequence, the model’s predictive power becomes limited. The predictive power also lacks as the three independent factors “information quality”, “system quality”, and “service quality” can not suffice to explain the intention to use a system. The adoption of a system is also influenced by organisational factors, as the decision of management, which may not be solely based on these three variables. Moreover, users have prior experience with other systems and they might show resistance to use a novel technology etc. Moreover, in the D&M IS Success model actual use is used as a mediator of net benefits. However, the model does not evaluate quality of use which is an important factor in their point of view and which is the main focus of the human-centred approach we will see in section 3.

The second kind of models are those that aim at predicting information success. Tech- nology Acceptance Model (TAM) is a well known representative of this category (Davis, Bagozzi, & Warshaw, 1989). User satisfaction depends according to the authors on two variables:

(29)

• Perceived usefulness: “the degree to which a person believes that using a particular system would enhance his or her job performance”;

• Perceived ease of use: “the degree to which a person believes that using a particular system would be free of effort” (Davis, 1989).

External Variables

Perceived Ease of use

Perceived Usefulness

Attitude

Toward Using Behavioural

Intention to Use Actual

System Use

Figure 2.3: Technology Acceptance Model (TAM) adapted from Davis et al., 1989

The model predicts that only when perceived usefulness and perceived ease of use are high enough, the user will accept the system and use it by choice. The authors propose a questionnaire to evaluate the two dimensions. The model was later developed in order to identify the factors that that do influence perceived utility (Venkatesh & Davis, 2000; Venkatesh &

Bala, 2008). In the third version of TAM, the authors identified the factors subjective norm, image, job relevance, output quality, and result demonstrability that influence perceived usefulness. Perceived ease of use in contrast is influenced by computer self-efficacy, perceptions of external control, computer anxiety, and computer playfulness.

Van der Meijden et al. (2003) conducted a meta research on papers that evaluates CIS and attribute them to an earlier version of D&M IS success dimensions. The authors included clinical systems in inpatient care except decision support systems and guidelines. Most of the evaluations were retrospective, descriptive and correlational. Only a few of the reviewed studies are comparative studies.

In a a more recent research, Palm et al. (2006) designed an electronic survey in order to evaluate the determinants of user satisfaction in CIS. The questionnaire is based on the models TAM and IS Success as described earlier. They expect the following determinants to predict positively user satisfaction:

• user characteristics

• user satisfaction

• use

• system quality

• perceived usefulness

• service quality

(30)

They can partly corroborate their hypothesis as global satisfaction is significantly associated with CIS quality, CIS use and service quality.

Despont-Gros et al. (Despont-Gros, Mueller, & Lovis, 2005) conducted a literature review of articles within papers that address the topics “user evaluations”, “identification of dimensions” or critical reviews of them. They found that acceptance, satisfaction and success were the most referenced ones in literature.

The papers discussed so far are concerned with factors of success and satisfaction of CIS in the inpatient setting. Electronic patient records are also introduced and evaluated in general practitioners’ practices (Christensen & Grimsmo, 2008) or systems that are targeted to patients use (Winkelman & Leonard, 2004).

2.4 Impact of electronic prescriptions systems

A CPOE is in most cases an integrated part of CIS. As CPOE usually have some type of decision support linked to it, we will not distinguish in the following section between systems with or without decision support. Further, alerts and reminders do not have to be directly linked to the prescription process. Exceptions are for instance warnings that the patient is carrier of an infectious disease or administrative reminders. Alerts and warnings can be linked to CPOE or any other subsystem of a CIS.

The dimension prescription behaviour is describing whether and how the physician is reacting to a medical alert. This dimension could also be namedhuman interaction factors.

Patient outcome measures if a positive effect of an alerting on the clinical condition of the patient can be observed. In the following we present some research which aims at identifying both: change in prescription behaviour and change in patient outcome. Later we introduce studies which analyse each topic in more detail.

A meta-analysis including papers on the impact of electronic alerts and prompts on clinicians’ prescribing behaviour (Schedlbauer et al., 2009) aimed at evaluating which types of alert have a beneficial effect on prescribing or clinical outcome. From 27 types of alerts in 20 studies they identify 23 which showed to improve significantly prescribing behaviour and 5 showed a positive effect on clinical outcome.

Garg et al. (2005) found in their meta-analysis of 97 papers that a CDSS improved prescribing performance in 64% of the cases. 52 out of the 97 papers studied patient outcome.

However, only 7 (13%) of them reported significant improvement for patient outcome. Ad- ditionally, the authors found that automatic alerting shows better results (73% success) than passive alerting (47% success, p = 0.02).

In a study in a tertiary-care hospital, Koppel et al. (2005) investigated errors which are newly introduced with the adaption of CPOE. In a qualitative research they identified 22 situations with CPOE-facilitated risks and attributed them to two categories:

• Information errors due to fragmentation and system integration failure

• Human-Machine Interface Flaws: System’s work flow does not correspond to user’s work flow

(31)

CPOE-enhanced errors were common. 50% to 90% of house staff encountered frequently (weekly or more often) CPOE-related errors.

So far, we see that the literature gives indications that the effect on prescription behaviour seems to be stronger than on patient outcome. However, patient outcome is a variable that is difficult to measure. What period of time should be included to evaluate the clinical effects? Should the effect on third persons be included as well? When we take the example of prescriptions of antibiotics, the choice between different types of antibiotics might be therapeutically equivalent for the patient. However, it might affect the antibiotic resistance of pathogens which in turn would be a threat to the public health.

Further we have seen that CPOE might also introduce new risks that were not known in paper based prescriptions. For example a physician might by accident choose a wrong measure unit in a pull down menu. Or, on a larger level, a physician overlook a prescription because it is located out of the displayed area of the screen in a long list of prescriptions.

In the following sections, more findings on the effect of CPOE on prescribing behaviour and patient outcome are presented.

2.4.1 Effect of CPOE on prescribing behaviour

Even if some studies show a positive effect on prescribing behaviour, the potential seem to be far of being utilised. A meta analysis on alert overrides (van der Sijs, Aarts, Vulto, & Berg, 2006) shows that depending on the study 49 - 96% of alerts are overridden. The factors contributing to alert override are identified as:

• alert fatigue due to irrelevant and repeating alerts

• medical constraints: no drug change possible

• patients’ resistance to drug change

• inapplicable and useless alerts

• physician’s faith in own knowledge, disagreement with guidelines

• alert is based on incorrect information

• lack of time

• usability problems: misunderstood or unnoticed alerts.

This is a major finding for our thesis. We see that even if the system works as intended, there are several human factors that influence the use of alerts.

A subsequent research (van der Sijs, van Gelder, Vulto, Berg, & Aarts, 2010) shows using structured interviews after a prescribing simulation that even if alerts were handled correctly, underlying rules and reasoning were often erroneous. Eighteen residents did some prescribing tasks in a work-like context, where 234 alerts were triggered. An expert panel categorised 30% of the alert handling as incorrect because of the action (24%) or because the reasoning for the handling was incorrect (16%). The authors categorise the behaviour as skill

(32)

based (13%), rule based (63%) or knowledge based (24%) (see Rasmussen, 1986). Knowl- edge based errors designate errors that are the consequence of a decision that is based on erroneous information. Rule based errors designate errors where the physician uses correct information but the information is not relevant for the given case. Finally, skill based errors designate errors that originate from erroneous manipulation of the system. We find that most errors are rule based errors where correct procedures for the wrong reason are applied.

Another impact on work processes is the changed interaction between nurses and physicians (Beuscart-Zéphir et al., 2004). The previously distributed decision making process is replaced by a centralised decision making process as the conjoint medical round takes no more place.

A recent meta analysis (Shojania et al., 2010) studied the effect of reminders on physician’s behaviour. They calculated the median of effect and Interquartile range (IQR) for each comparisons in 28 studies and showed an improvement of 4.2% (IQR 0.8% - 18.8%).

In studies where physicians were asked to enter a response, the effect was by trend larger (median 12.9%, IQR 2.7% - 22.7% compared to 2.7%, IQR 0.6% - 5.6% with p = 0.09).

Many other reminder characteristics did not show any significant effect on prescribers’ behaviour. In particular, publication year, study design, setting (inpatient, outpatient), sample size, mode of delivery, and implication of users, did not show any effect.

In the following, we will give a non-exhaustive overview of studies that measured the impact of alerts on physicians in a specific hospital setting.

Overrides of drug alerts in five primary care practice affiliated with Beth Israel Deaconess Medical Center showed a similar picture of alert overriding (Weingart et al., 2003). 91% of the alerts were overridden. Most of the given reason were that the alert was not matching the given context. Based on medical expertise expert reviewers concluded that 36.5% of the alerts were invalid.

A randomised control trial (Judge et al., 2006) evaluates physicians’ behaviour during 12 month. In one unit (intervention group) the alerts were displayed and in the other they were triggered but not visible (control group). Experts evaluated physicians’ order behaviour in both groups. Physicians are only slightly more likely to take an action (relative risk = 1.11, CI95%1.00, 1.22) in the intervention group.

A retrospective analysis (Isaac et al., 2009) in Massachusetts (NJ) analysed drug alerts use during 9 months. Among high severity DDI alerts a 20 fold difference of acceptance depending on class of drugs was found. 9.2% of drug interaction alerts and 23.0% of drug allergy alerts were accepted. Highly severe alerts were only slightly more accepted than lower severity alerts.

Another study (Hunteman, Ward, Read, Jolly, & Heckman, 2009) showed in a retrospective analysis of allergy alerts that 643 alerts were triggered for 49,887 prescriptions in total.

Of these 643 alerts, 625 were overridden (97%). In most cases (49%), the patient has toler- ated the medication, the benefit outweighed the risk (29%) or the medication was therapeutically appropriate (24%).

In a randomised controlled trial over one year, Judge et al. (2006) investigate the number and type of triggered alerts and their impact on physician’s prescribing behaviour. They show no significant improvement for physicians who have received the alert (31%) compared to the control group (28%) where alerts have not been displayed. However, there have

(33)

been some specific alerts which resulted in an improvement. Alerts related to warfarin and central nervous system treatments have shown modest improvement with a relative risk of 3.5 (CI_95%: 2.1-5.7), respectively 1.4 (CI_95%: 1.0-1.9).

We see that many more recent studies fail to show a general impact of alerts on prescription behaviour. There are some studies that could identify factors that promotes the compliance with alerts.

For instance, some types of alerts work better than others. Thus, default dosing recommendations for a drug has a positive impact on prescription behaviour. However, recommendations which change the plan of care are less accepted (Teich et al., 2000).

Prescription errors are more likely to happen at certain points in time of a hospitalisation.

Caruba et al. (2010) evaluate the collected data of 18 days in 7 medical wards, the prescription errors and at what point in the hospital stay they happen. 52 % of the alerts were overridden. They find that 0.9% of prescriptions were erroneous and that 51% of these errors happen in the first days of hospital stay. They also show that the ward where the care was given has a high impact on drug prescription error. Taking the two extremes, prescription errors in nephrology were 6.31 (CI95: 1.94-20.46) times more probable than in diabetes care.

The authors explain the differences between the wards by different behaviour of medical staff and different degrees of collaboration with pharmacists.

One factor which might have an impact on user’s acceptance is the active or passive use of medical alerts. Tamblyn et al. (2008) designed a randomised single blinded control trial to demonstrate such an effect. One group had to open the drug related information by clicking on “drug review”. The other group got the additional drug alert information automatically. In a follow-up study, physicians could alter the level of alert at which they wanted to be informed. The results show that physicians in the computer-triggered group saw more alerts than in the on-demand group and made more changes to the level of alert they would see (50% in computer-triggered to 21% in the alert-on-demand group). An advantage of computer-triggered alerts was also found by meta analysis of Kawamoto et al. (2005). How- ever, there was no difference in overall prescribing errors. A review of the prescribing shows that most alerts were ignored for clinically acceptable reasons.

2.4.2 Effect of CPOE on patient outcome

As we have seen in the previous section, alerts may have in specific cases a positive effect on prescription behaviour. However, the final goal of such alerts is to prevent harm from the patients or any third party’s health. Even if the alert did not incite the physician to follow the proposed solution, the physician could take alternative actions to prevent patient’s harm.

Also, the system may lack information and produce therefor false positive alerts. Moreover, there are cases where a physician has to choose the least inconvenient option. Lastly, even if the physician may have committed an error it does not mean that it reaches the patient.

Other health care provider might interfere or the ADE was not severe. Literature defines patient outcome in relation with drug prescription as non-intercepted preventable ADE and non-intercepted preventable potential ADE.

One of the few studies that showed an advantage of CPOE on patient outcome comes from the home-grown system at Brigham and Women’s Hospital (Bates et al., 1998). In a

(34)

before-after study, they found that preventable ADEs declined by trend 17% from 4.69 to 3.88 (p = 0.37) and non-intercepted potential ADEs declined 84% from 5.99 to 0.98 per 1000 patient-days (p = 0.002).

When looking at studies with a high level study design (e.g. randomised controlled trials) the optimism is lowered. In a meta-analysis on patient outcome (Mollon et al., 2009) they find few positive results. Only 23 studies reported on patient outcome and only five showed a positive effect.

Rochon et al. (2006) point at the weak point of evaluation of CIS in long term care in regard of their effect on patient outcome. Even if it is shown that CDSS has a positive effect on prescribing errors, it does not automatically prove that patients suffer from fewer ADEs.

In there commentary they postulate that:

• ADE are common in long-term care

• Most errors occur at the ordering and monitoring stages of the prescribing process

• CPOE with CDSS has the potential to reduce medication errors which could lead to a decrease in ADE

Kaushal et al. (2003) conducted a meta research on five CPOE and seven isolated CDSS.

Among the CPOE studies two demonstrated a marked decrease in serious medication errors. For the isolated CDSSs, three demonstrated statistically significant improvements in antibiotic-associated medication errors. They showed that in two institutions with “home- grown” systems medication errors and serious medication error rates were significantly reduced. However, studies are often underpowered and fail to give significant results in randomised controlled trials.

In order to test the clinical outcomes, they defined the following levels:

• Clinical outcome: any measure of morbidity or mortality including ADE as defined in the “Outcome definitions”

• Surrogate outcome: Observed errors, intermediate outcomes (e.g., laboratory results) with a well-established connection to the clinical outcomes of interest

• Other: Other measurable variables with an indirect or unestablished connection to the target safety outcome

• None: No outcomes relevant to decreasing medical errors or adverse events

In the following we provide a number of studies on the impact of medical alerts on patient outcome.

In the Brigham and Women’s hospital, a study on overrides of drug-allergy alerts showed that 80% of the alerts were overridden (Hsieh et al., 2004). An expert panel reviewed the overridden drug-allergy alerts and classified them depending on the level of certainty that they caused an adverse event (Likert scale from 1-6). Severity of drug alerts are either significant, serious, life threatening or fatal. The panel evaluates also if the ADEs were

(35)

preventable, meaning for example that the clinical need for prescribing the drug was greater than the allergic reaction. None of the ADEs were judged to be preventable. The justification of physicians was in half of the cases “Aware/Will monitor” and in nearly a third of the cases

“Patient does not have this allergy/tolerates”.

In Massachusetts, an expert panel looked at the statistics of medical alerts and ADEs. They estimated that for every 331 alerts, 1 ADE can be prevented and that only a few alerts (10%) accounted for 60% of prevented ADEs and 78% of cost savings (Weingart, Simchowitz, et al., 2009).

In the University of Illinois Hospital and Medical Center, a positive effect on patient outcome could be found in a cohort study for alerts attributed to prescriptions which are contraindicated for different levels of renal insufficiency (Galanter, Didomenico, & Polikaitis, 2005). House staff accepted 42% of these alerts. Prescribing personnel’s behaviour was studied 4 month before and 14 month after the introduction of creatine clearance alerts. The results showed that the likelihood of prescribing at least one contraindicated dose of a drug decreased from 89% to 47% (p < 0.0001).

Earlier studies like Raschke et al. (1998) draw a positive balance of the introduction of drug alerts in their hospital in Phoenix, AZ. 53% of alerts were judged to be true positives.

In 44% of the cases, physicians stated that they were unaware of the potentially dangerous situation. The study sees a potential improvement of their system to reduce ADE-related injury in 0.064% of patient admissions.

In an outpatient context in Boston Ghandi et al. (2005) found that even if prescribing errors decreased between CPOE (4.3%) and handwritten notes (11.0%) the potential ADEs increased from 2.6% to 4.0%. These are only trends and not statistically relevant differences.

The outpatient sector is characterised by higher prescription error rates than inpatient sector, as they include typically more parameters (number to be dispersed, number of refills). The authors argue that 95% of ADE could be prevented when advanced decision support systems with dose and frequency checking would be in place.

Studies of CPOE on patient outcome show a mediocre result. Shojania at al. (2010) point out that: “if computer reminders do not improve patient outcomes, this may reflect inadequate connections between the targeted processes and outcomes of care rather than a failure to change physician behaviour”.

2.5 Improvements to alert systems

An often cited paper is the “Ten Commandments for effective Clinical Decision Support”

from Bates et al. (2003). Based on experiences and research with CPOE systems, the authors give following recommendations (which I reformulated to make them more concise):

1. Speed is everything: a CPOE has to have low response times

2. Anticipate needs and deliver in real time: an optimal decision support would respond to latent needs of physicians, providing them compiled information from medical data and knowledge

(36)

3. Fit into the physician’s work flow: give advice on the corresponding screen and at the right time.

4. Little things can make a big difference: the interface design has a large impact on how and if physicians use a functionality

5. Recognise that physicians will strongly resist stopping: if no alternative actions are provided, physicians will likely resist to stop an unadvised action

6. Change direction is easier than stopping: provide alternative actions when disadvising to continue a treatment

7. Simple interventions work best: make alert information concise and use simple work flows

8. Ask for additional information only when the system really needs it: physicians resist entering information if they do not see an immediate advantage. Design the system so that it handles correctly missing information.

9. Monitor impact, get feedback, and respond: evaluate how alerts are used and find the right balance between over- and underuse

10. Manage and maintain your knowledge-based systems: the alerts have to be monitored and the corresponding knowledge base has to be modified if requirements change A straight forward approach is to retrospectively analyse alerts and propose a remedy for each problem found. In a study on a CPOE in a Spanish general hospital (Lerma Gaude, Poveda Andrés, Font Noguera, & Planells Herrero, 2007) 20 drugs were responsible for 34% of the alerts. The authors identified the problem linked to these alerts (most often excess or omission of alerts) and proposed for each alert what actions to take (e.g., adding more advanced decision support, changing work flow, or change interface parameters).

Some research is concerned with the graphical representation of alerts (Pettersson et al., 2008). Instead of using exclamation signs, they propose a presentation as a 2-dimensional symbol which codes severity and the type of alert (hypersensitivity, important diagnoses, and treatments) with length and direction of bars.

In order to improve specificity, alert system knowledge bases can be cleaned from unnec- essary alerts. In a report on the redesign of a DDI alert system (Luna et al., 2007), the authors replace a DDI database which has been fed from different sources. For the redesign, experts evaluate each interaction from the existing index and choose the ones with a clear biblio- graphical and clinical background. Severity of the alert was evaluated and it was decided whether an alert should interrupt the prescription process (active alert) or only be displayed on the screen (passive alert). They defined an alert to be active when (1) simultaneous administration of a drug should be avoided or when (2) the alert has been classified to be highly significant and with great potential to harm the patient. Only 695 of a total of 3809 alerts have been categorised as active alerts.

A similar approach is proposed by a study (Shah et al., 2006) that conducts a classification and selection process on alerts of different types (duplicate class, DDI, drug-lab,

(37)

drug-disease, drug-pregnancy). Depending on the severity, the alert is attributed to 3 levels and thus 71% of the alerts could be assigned to non interruptive display mode. With level 1 alerts (2%) physicians could not proceed at all with the prescription. With level 2 alerts (63%) they could proceed when providing an override reason. Level 3 alerts (35%) are just displayed at the top of the page without interrupting the work flow. The selective set of alerts was accepted in 67% of the cases. The provision of selective sets of alerts seem to be a better choice than depending solely on default commercial knowledge database. Also, interruptions should be limited and thus avoid work flow disruptions.

In another context (van der Sijs, Aarts, van Gelder, Berg, & Vulto, 2008), the reduction of the number of alerts seemed to be difficult. The most often overridden alert (accounting for 72% of DDI alert overrides) are presented to 24 experts in the hospital. The participants view the alert as screen shots, in one version with a short alert description and in one version with the complete alert text. The experts have been asked if the alert could be turned off and to state the reason. They could not agree to turn off any alert. The highest agreement for turning off an alert was 13 of 24 of respondents. The study shows that different individuals from different departments have different requirements concerning medical alerts.

In the same direction is arguing a study of Weingart et al. (2009). Even if they could demonstrate a positive impact of CPOE on mortality, injuries and costs, taking into account the low compliance to these alerts (8.9 acceptance rate), the question remains whether the

“juice is worth the squeeze”. They propose to improve the specificity of the alerts to get a better yield of CDSS.

The usage of the alert can be optimised by improving the interaction with the physician.

The aim is to alert the physician at the right time in the prescription process and to do it with the least necessary interruption. Miller et al. (2005) designate seven steps in the ordering work flow at which the physician can be supported using decision support and the authors defined six different intervention approaches going fromsubtletointrusiveinteraction modes.

Yet another study (Strom et al., 2010) investigated if an alert which requires a confirmation of the physician (either clicking “acknowledge” or “view action”) lead to a better compliance than when just displaying the alert without any required action. The randomised controlled study failed to show a significant effect of the different interaction modes.

For the case of drug-allergy alerts Hsieh et al. (2004) propose seven recommendations how to improve alerting. In summary they give instructions which cases really need an interruptive alert and when non-interruptive alerts suffice. Further, they propose that docu- mentation from prior prescriptions should be displayed when renewed alerts are triggered. In order to do that, override reasons should be coded and not allowed to be entered as free text.

The severity of alerts should be visually recognisable. For instance, true allergies should be formatted differently from drug sensitivity.

Renewed prescriptions should not trigger alerts when they have been dealt with before (Tamblyn et al., 2008). Automated surveillance and computer-triggered alerts should be restricted to new drugs, allergies, or diseases. The review of patients prescriptions could be optional and may provide physicians with more detailed information on clinical risks.

The patient’s history and characteristics like age, number of medications, renal function, and comorbidity should be taken into account for decision support. Previously overridden alerts could even be silenced for future occurrences (Isaac et al., 2009).

(38)

In contrast to these approaches some authors (Stutman, Fineman, Meyer, & Jones, 2007) propose to use the same alert levels for all physicians and pharmacists in order to assure that all care takers have the same base of information. The introduction of CDSS should start with a few alerts and than the use should be monitored. The authors propose an iterative approach which bases new developments on findings that are made in the process. In doing so, they achieved a high adoption of these alerts (75-78%).

Zachariah et al. (2011) have developed guidelines for medication related decision support systems. According to their proposal an alert should have four components:

• a signal word: danger, warning, caution;

• the nature of hazard;

• instructions how to avoid;

• the consequences if not following the alert.

As we have seen in the introduction alert systems should ideally have a feedback loop where the user’s use of the alert system is monitored and where users can give feedback. In this perspective Zimmerman et al. (2007) propose a dashboard for alert usage where the use of alerts can be monitored and alerts can be modified as required.

2.6 Dimensions in CDSS and alerts

The meaning of dimensions is among others “a part or feature or way of considering some- thing” (Cambridge Advanced Learner’s dictionary, 2008). In contrast to a classification where the goal is to match each existing object to a corresponding class (as for instance species to upper class hierarchies in biology) a differentiation aims at finding characteristics which define the object without being exhaustive or mutually exclusive. In this perspective, we evaluate CDSS and alerts on different levels.

I start with the integrated form of alerts in their larger structure: the alert systems or CDSS. Whereas the characteristics of an alert can be described at a specific point in time, the perspective on an alert system reveals the alert’s changes over time and it’s interaction with other systems.

2.6.1 Dimensions in CDSS

In the following sections we will cover knowledge-based decision support systems and ig- nore non-knowledge based systems which are based on artificial neural networks or genetic algorithms (see Berner & Lande, 2007).

Edward H. Shortliffe, the main initiator of MYCIN, one of the first expert systems in the medical domain, proposes a classification of clinical decision support systems (Shortliffe, 1987).

First, he made a classification of the level of support a decision support system provides:

Decision making in dynamic work situations: building a framework for medical alerts in computerized drug prescription

Thesis

Reference

Decision making in dynamic work situations: building a framework for medical alerts in computerized drug prescription

Decision making in dynamic work situations:

Building a framework for medical alerts in computerized drug prescription

Contents

1 Introduction

Part I

State of the art

2 System-centred approach