BRIEF REPORT: Beyond Clinical Experience: Features of Data Collection and
Interpretation That Contribute to Diagnostic Accuracy
M a t h i e u R. N e n d a z , MD, MHPE, 1,2 A n n e M. Gut, PhD, i A r n a u d Perrier, MD, 2
Martine Louis-Simonet, MD, 2 Katherine Blondon-Choa, MD, 2 Francois R. Herrmann, M D , MPH, 3
Alain F. Junod, MD, 2 Nu V. Vu, PhD 7
1Unit of Development and Research in Medical Education, University of Geneva, Faculty of Medicine and Geneva University Hospitals, Geneva, Switzerland; 2Department of Internal Medicine, University of Geneva, Faculty of Medicine and Geneva University Hospitals, Geneva, Switzerland; 3Department of Rehabilitation and Geriatrics, University of Geneva, Faculty of Medicine and Geneva University Hospitals, Geneva, Switzerland,
B~CKOROb']~Z): Clinical experience, features of data conection proc- ess, or both, affect diagnostic accuracy, but their respective role is unclear.
OBJECTIVE, DESIGN: Prospective, observational study, to determine the respective contribution of clinical experience and data collection features to diagnostic accuracy.
METHODS: Six Internists, 6 second year internal medicine residents, and 6 senior medical students worked up the same 7 cases with a standardized patient. Each encounter was audiotaped and immediately assessed by the subjects who indicated the reasons underlying their data collection. We analyzed the encounters according to diagnostic accuracy, information collected, organ systems explored, diagnoses evaluated, and final decisions made, and we determined predictors of diagnostic accuracy by logistic regression models.
~ 8 ~ " Several features significantly predicted diagnostic accuracy after correction for clinical experience: early exploration of correct di- agnosis (odds ratio [OR] 24.35) or of relevant diagnostic hypotheses (OR 2.22) to frame ciinlcal data collection, larger number of diagnostic hypotheses evaluated (OR 1.08), and collection of relevant clinical data (OR 1,19).
CONCLUSION: Some features of data collection and interpretation are related to diagnostic accuracy beyond clinical experience and should be explicitly included in clinical training and modeled by clinical teachers. Thoroughness in data collection should not be considered a privileged way to diagnostic success.
KEY WORDS: clinical reasoning; clinical data collection; experience; expertise; medical education; internal medicine.
DOI: 10.1111/J. 1525-1497.2006.00587.x J GEN INTERN MED 2006; 21:1302-1305.
S
t u d i e s in cognitive psychology h a v e d e s c r i b e d t h e proc- e s s e s of clinical reasoning, t h e organization of memory, a n d t h e m e n t a l r e p r e s e n t a t i o n s of knowledge. 1.2 C h a r a c t e r i s - tics influencing d a t a collection or recognition have b e e n well d o c u m e n t e d in visual clinical disciplines like dermatology, or in c a s e s for w h i c h t h e p a t i e n t ' s physical a p p e a r a n c e leads to t h e diagnosis, a ~ For s i t u a t i o n s containing l e s s visible data, previ- o u s s t u d i e s including experienced p h y s i c i a n s 7 a n d s t u d e n t s 8 solving one single c a s e o u t of 4 possible s i t u a t i o n s s u g g e s t e d t h a t early h y p o t h e s i s generation provided a s t r u c t u r e to guideThis paper w a s presented in part at the annual conference qf the Swiss Society of General Internal Medicine, Basel, 2005.
Address correspondence and requests for reprints to Dr Nendaz: Department of Internal Medicine, Service of General Internal Medicine, Geneva University Hospitals, 1211 Geneva 14, Switzerland (e-mail: Mathieu.Nendaz@hcuge.ch).
1302
p h y s i c i a n s ' acquisition of key clinical data. F u r t h e r s t u d i e s 9'1° also s u g g e s t e d t h a t s o m e b e h a v i o r s in d a t a collection, s u c h a s detailed inquiry a b o u t t h e c h i e f c o m p l a i n t a n d f r e q u e n t s u m - m a r i z a t i o n o f t h e collected i n f o r m a t i o n , w e r e a s s o c i a t e d w i t h b e t t e r d i a g n o s t i c o u t c o m e s . D e s p i t e t h e e x i s t i n g evidence, faulty d a t a collection a n d i n t e r p r e t a t i o n a r e still i m p o r t a n t s o u r c e s of e r r o r s 11 a n d m a n y clinician e d u c a t o r s still r e w a r d t h o r o u g h n e s s of d a t a collection r a t h e r t h a n r e l e v a n c e d i c t a t e d b y initial d i a g n o s t i c h y p o t h e s e s . T h i s s t u d y a i m s to c o n f i r m t h e s e p r i n c i p l e s w i t h a larger s e t of c a s e s from different o r g a n
systems
a n d to d e t e r m i n e t h e r e s p e c t i v e c o n t r i b u t i o n of clinical e x p e r i e n c e a n d specific f e a t u r e s of d a t a collection a n d i n t e r p r e t a t i o n to explain d i a g n o s t i c a c c u r a c y .METHODS
Subjects and Research Design
We a s k e d t h e 10 e x p e r i e n c e d G e n e r a l I n t e r n i s t s heavily involved in t e a c h i n g in o u r service to v o l u n t e e r for o u r s t u d y . Six of t h e m a c c e p t e d , a c c o r d i n g to t h e i r time c o n s t r a i n t s . We t h e n r e c r u i t e d s e c o n d - y e a r r e s i d e n t s a n d s e n i o r m e d i c a l s t u d e n t s d u r i n g s u c c e s s i v e r e s i d e n c y a n d c l e r k s h i p r o t a t i o n s in o u r service, u n t i l we o b t a i n e d 6 p a r t i c i p a n t s in e a c h group. All s u b j e c t s w o r k e d u p t h e s a m e 7 c h i e f c o m p l a i n t s w i t h a s t a n d a r d i z e d p a t i e n t , t h u s p r o d u c i n g a total a m o u n t of 42 e n c o u n t e r s for e a c h g r o u p of clinical experience, a s a m p l e size e s t i m a t e d a d e q u a t e in t e r m s of p o w e r a n d feasibility. No specific review w a s r e q u i r e d in o u r i n s t i t u t i o n for t h i s s t u d y .
We u s e d c h a r t s of real p a t i e n t s to c r e a t e 7 c a s e s c r i p t s p o r t r a y e d by a s t a n d a r d i z e d p a t i e n t (SP). Their c h i e f c o m - p l a i n t s were: (1) h e a v y s e n s a t i o n in t h e a b d o m e n , (2) cough, (3) w e i g h t loss, (4) h e a d a c h e , (5) d i a r r h e a , (6) lower limb ed- ema, a n d (7) arthritis. The d i a g n o s e s of t h e s e c o m m o n c a s e s relied m a i n l y o n h i s t o r y a n d p h y s i c a l e x a m i n a t i o n . All s u b j e c t s e n c o u n t e r e d t h e 7 c a s e s in t h e s a m e o r d e r w i t h o u t time limitation. At t h e e n d of e a c h e n c o u n t e r t h e y provided t h e i r final w o r k i n g d i a g n o s i s . T h e e n c o u n t e r s w e r e a u d i o t a p e d a n d i m m e d i a t e l y r e p l a y e d for a t h i n k i n g - a l o u d s t i m u l a t e d recall, 1 d u r i n g w h i c h t h e s u b j e c t s i n d i c a t e d t h e p u r p o s e s u n d e r l y i n g t h e i r d a t a collection. T h e s e c o m m e n t s w e r e a u d i o t a p e d a n d r e t r a n s c r i b e d for a n a l y s e s . Two previ- o u s l y t r a i n e d i n v e s t i g a t o r s e v a l u a t e d a n d tallied t h e c h a r a c - teristics of e a c h e n c o u n t e r . Their i n t e r r a t e r c o r r e l a t i o n r a n g e d from 0.83 to 0.98.
Manuscript received December 3, 2005 Initial editorial decision January 18, 2006 Final acceptance June 19, 2006
JGIM N e n d a z e t al., Predicting D i a g n o s t i c A c c u r a c y 1303
Table I. Characteristics of The Encounters, According to Clinical Experience*
Experts Residents Students Experience Case effect
41 encounters 42 encounters 42 encounters effect (P*) (P*) Information collected
Encounter duration, mean/case (minutes) Unique findings collected, mean N/case Relevance score* of unique findings, mean/case Key questions t, mean N/case
Summary occurrences, mean N/case Systems explored t
Body systems explored: mean N/case Lines of inquiry, history, mean N/case Diagnostic hypotheses
Diagnostic hypotheses evaluated; mean N/case Relevance of diagnostic hypotheses*, mean/case Findings collected until final diagnosis first generated, mean N/case
Final decisions
Unique decisions made, mean N/ease Relevance of distinct decisions ~, mean/case
15.2 (13.8 to 16.7) 19.0 (18.0 to 19.9) 21.4 (19.6 to 23.3) .03 .90 61 (56 to 67) 77 (72 to 83) 73 (67 to 79) .19 .62 0.60 (0.57 to 0.62) 0.41 (0.40 to 0.42) 0.43 (0.41 to 0.44) <.0001 .68 9 (8 to 10) 8 (7 to 8) 7 (6 to 8) < .0001 < .0001 1.93 (1.63 to 2.22) 1.38 (1.07 to 1.69) 1.17 (.88 to 1.46) .11 .59 7.4 (6.9 to 8.0) 7.4 (6.8 to 7.9) 6.8 (6.2 to 7.4) .12 .21 14 (12 to 16) 18 (16 to 20) 17 (15 to 20) .41 .77 14 (12 to 15) 16 (15 to 18) 16 (14 to 17} .41 .04 0.69 (0.66 to 0.72) 0.49 (0.46 to 0.52) 0.49 (0.46 to 0.52) <.001 .83 9.8 (7 to 12) 24 (16 to 32) 23 (15 to 32) .008 .03 7 (6 to 8l 8 (7 to 9) 8 (7 to 91 .36 .005 0.69 (0.64 to 0.731 0.42 (0.37 to 0.47} 0.52 (0.47 to 0.561 <.001 .21 *ANOVA with subjects nested within experience levels a n d repeated measures for cases. Numbers in brackets denote 95% confidence intervals. *Relevance of information collected, diagnostic hypotheses generated, or decisions made, is their level of concordance (from O, 0% concordance to i,
100% concordance) among experts reaching the correct diagnoses. Key questions, decisions, or diagnostic hypotheses are those elicited by all members of this reference group.
tExamples of body systems: respiratory, neurological. One line of inquiry is a sequence of consecutive questions evaluating the s a m e diagnostic hypothesis.
Outcome Variables and Data Analyses
We analyzed 125 e n c o u n t e r s , 1 e n c o u n t e r b e i n g n o t r e c o r d e d for t e c h n i c a l p r o b l e m s . For e a c h e n c o u n t e r , we d e t e r m i n e d t h e diagnostic a c c u r a c y (binary variable, b a s e d on t h e a c t u a l pa- t i e n t ' s diagnosis), t h e a m o u n t , relevance, a n d s e q u e n c e of t h e i n f o r m a t i o n collected, t h e o r g a n s y s t e m s explored, t h e diag- n o s t i c h y p o t h e s e s evaluated, a n d t h e m a n a g e m e n t d e c i s i o n s m a d e . B e c a u s e t h e r e is no gold s t a n d a r d to w o r k u p specific c a s e s , we u s e d t h e level of c o n c o r d a n c e a m o n g e x p e r t s w i t h correct final d i a g n o s e s to d e t e r m i n e t h e relevance of t h e infor- m a t i o n collected a n d t h e diagnostic h y p o t h e s e s generated. 12-15 E a c h piece of i n f o r m a t i o n a n d d i a g n o s t i c h y p o t h e s i s received a relevance weight r a n g i n g from 0 (0% c o n c o r d a n c e ) to 1 (100% concordance). Key i n f o r m a t i o n or h y p o t h e s e s were t h o s e elic- ited b y all e x p e r t s (100% c o n c o r d a n c e ) .
We b u i l t a n ANOVA m o d e l in w h i c h t h e u n i t of a n a l y s i s w a s t h e e n c o u n t e r , i.e.. t h e p r o d u c t of s u b j e c t s (18) by c a s e s (7), s u b j e c t s b e i n g n e s t e d w i t h i n 3 e x p e r i e n c e levels. We a n a - lyzed t h e effects of clinical e x p e r i e n c e o n t h e v a r i a b l e s listed in Table 1, w i t h t h e 7 c a s e s a s r e p e a t e d m e a s u r e s . We also t e s t e d i n t e r a c t i o n s b e t w e e n c a s e s a n d e x p e r i e n c e levels.
We d e t e r m i n e d t h e f e a t u r e s of t h e d a t a collection p r o c e s s p r e d i c t i n g d i a g n o s t i c a c c u r a c y b y univariate, bivariate (cor- r e c t i o n for clinical experience), a n d m u l t i p l e logistic r e g r e s s i o n m o d e l s (corrected for all collected data). S t a n d a r d e r r o r s a n d 95% c o n f i d e n c e intervals (CI) were a d j u s t e d for i n t r a g r o u p correlation, t h u s t a k i n g into a c c o u n t t h e fact t h a t t h e s a m e s u b j e c t s a s s e s s e d m a n y c a s e s . All a n a l y s e s were p e r f o r m e d u s i n g t h e S t a t a "~" statistical software (release 9.1, S t a t a Corp., College Station, TX).
RESULTS
The c h a r a c t e r i s t i c s of t h e e n c o u n t e r s differed a c c o r d i n g to t h e s u b j e c t s ' levels of clinical e x p e r i e n c e (Table 1}. Overall, e x p e r t s differed m o r e from r e s i d e n t s a n d s t u d e n t s t h a n did r e s i d e n t s
from s t u d e n t s . C o m p a r e d w i t h e x p e r i e n c e d p h y s i c i a n s , y o u n g - er d o c t o r s collected l e s s r e l e v a n t data; e v a l u a t e d l e s s r e l e v a n t d i a g n o s t i c h y p o t h e s e s ; e v a l u a t e d t h e final c o r r e c t d i a g n o s i s later d u r i n g t h e e n c o u n t e r ; a n d m a d e d e c i s i o n s of lower relevance. No i n t e r a c t i o n b e t w e e n c a s e a n d level of e x p e r i e n c e w a s significant. The p r o p o r t i o n of c a s e s d i a g n o s e d correctly w a s , respectively, 81% (95% CI 66 to 90), 45% (95% CI 31 to 60), a n d 36% (95% CI 23 to 51) for t h e experts, r e s i d e n t s , a n d s t u d e n t s (P< .001).
The following v a r i a b l e s significantly p r e d i c t e d d i a g n o s t i c a c c u r a c y in t h e u n i v a r i a t e logistic r e g r e s s i o n : h i g h e r level of clinical e x p e r i e n c e (odds ratio [OR] 7.43, 95% CI 2.17 to 25.41), collection of key i n f o r m a t i o n (OR 1.23, 1.09 to 1.39}, s u m m a r i z a t i o n of available i n f o r m a t i o n (OR 1.50, 1.00 to 2.27), g e n e r a t i o n of t h e c o r r e c t d i a g n o s i s a t l e a s t o n c e d u r i n g t h e e n c o u n t e r (OR 15.45, 1.87 to 127.83), e v a l u a t i o n of t h e c o r r e c t d i a g n o s i s w i t h i n t h e first 1O q u e s t i o n s a s k e d (OR 28.29, 3.33 to 239.95), a n d evaluation of key d i a g n o s t i c hy- p o t h e s e s d u r i n g t h e e n c o u n t e r (OR 2.54, 1.54 to 4.18).
After c o r r e c t i o n for clinical e x p e r i e n c e (Table 2), f r e q u e n t s u m m a r i z a t i o n of i n f o r m a t i o n w a s n o longer significant a n d t h e total n u m b e r of d i a g n o s t i c h y p o t h e s e s e v a l u a t e d d u r i n g t h e e n c o u n t e r s b e c a m e a significant predictor. T h e n u m b e r of key d i a g n o s t i c h y p o t h e s e s r e m a i n e d t h e m o s t significant variable, even w i t h t h e conservative Bonferroni's correction for multiple c o m p a r i s o n s . 17
With multiple logistic r e g r e s s i o n a n a l y s i s , clinical experi- e n c e at t h e s t u d e n t level (OR 0.24, 0.07 to 0.83), evaluation of key d i a g n o s t i c h y p o t h e s e s d u r i n g t h e e n c o u n t e r s (OR 3.12,
1.55 to 6.25), a n d t h e late e v a l u a t i o n of t h e c o r r e c t d i a g n o s i s (OR 0.97, 0.94 to 0.99) r e m a i n e d significant i n d e p e n d e n t pre- d i c t o r s of d i a g n o s t i c a c c u r a c y (40% of t h e v a r i a n c e explained).
DISCUSSION
In t h i s s t u d y , several c h a r a c t e r i s t i c s in d a t a collection a n d i n t e r p r e t a t i o n p r e d i c t e d d i a g n o s t i c a c c u r a c y b e y o n d t h e a c c u -
1304 N e n d a z e t al.. P r e d i c t i n g D i a g n o s t i c A c c u r a c y ]GIM
Table 2. Bivariate Logistic Regression Analysis Corrected for Experience for the Prediction of Complete and Correct Diagnosis, with 95% Confidence Intervals Adjusted for Intrasubject Correlation
Odds ratio 95% Cl P*
Mean n u m b e r of key questions asked by case* Mean n u m b e r of lines of inquiry by case t
Mean n u m b e r of diagnostic hypotheses evaluated by case Mean n u m b e r of key diagnostic h y p o t h e s e s evaluated by case
Correct diagnostic hypothesis evaluated at least once during the encounter Correct diagnostic hypothesis generated within the first 10 questions a s k e d
1.19 1.04 to 1.36 .01 1.05 1.O1 to 1.11 .03 1.08 1.01 to 1.16 .02 2.22 1.34 to 3.67 .002 15.17 1.05 to 219.6 .04 24.35 2.66 to 222.50 .005
*If Bonferroni's correction for multiple comparisons is applied, the significance threshold becomes 0.005.
t Key questions or diagnostic hypotheses are those elicited by all members of the reference group of experts reaching the correct diagnoses. t o n e line of inquiry is a sequence of consecutive questions evaluating the s a m e diagnostic hypothesis.
CI, confidence interval.
m u l a t e d y e a r s o f p r a c t i c e , a m o n g w h i c h t h e m o s t i m p o r t a n t w e r e t h e c o l l e c t i o n o f k e y i n f o r m a t i o n , t h e e v a l u a t i o n o f r e l e - v a n t d i a g n o s t i c h y p o t h e s e s a n d t h e g e n e r a t i o n o f t h e c o r r e c t d i a g n o s i s w i t h i n t h e f i r s t 10 q u e s t i o n s a s k e d d u r i n g t h e e n c o u n t e r . T h i s h i g h l i g h t s t h e c r u c i a l i m p o r t a n c e o f a n e a r l y e v a l u a t i o n o f r e l e v a n t d i a g n o s t i c h y p o t h e s e s d u r i n g t h e w o r k - u p to d i a g n o s e s u c c e s s f u l l y a c a s e , a s it d r i v e s t h e s u b s e q u e n t c o l l e c t i o n o f r e l e v a n t i n f o r m a t i o n . O u r r e s u l t s o n s e v e r a l c a s e s in v a r i o u s d o m a i n s o f i n t e r n a l m e d i c i n e e x p a n d p r e v i o u s r e s e a r c h t h a t a l r e a d y s h o w e d t h e s e r e l a t i o n s h i p s w i t h f e w c a s e s 1'7'9 f r o m s p e c i f i c s p e c i a l t i e s (e.g., n e u r o l o g y ) o r c a s e s r e l y i n g o n v i s u a l c u e s . 3-6 I n a d d i t i o n , s o m e p r e v i o u s w o r k s r e l i e d o n w r i t t e n c l i n i c a l v i g n e t t e s r a t h e r t h a n h i g h e r - f i d e l i t y s i m u l a t i o n a l l o w i n g for a n o p e n - e n d e d i n q u i r y (e.g., s t a n d a r d - i z e d p a t i e n t s ) , a c o n d i t i o n k n o w n to a l t e r c l i n i c a l r e a s o n i n g b e c a u s e t h e i n f o r m a t i o n is i m m e d i a t e l y p r o v i d e d r a t h e r t h a n p r o g r e s s i v e l y c o l l e c t e d b y t h e s u b j e c t . 16.is O u r d a t a a l s o give a n a d d i t i o n a l i n s i g h t i n t o t h e role o f c l i n i c a l e x p e r i e n c e . W h i l e a f o c u s e d d a t a c o l l e c t i o n a n d f r e q u e n t s u m m a r i z a t i o n s o f t h e c o l l e c t e d c l i n i c a l d a t a a r e m o r e a t r a i t o f a h i g h e r level o f t r a i n - i n g t h a n a n e c e s s a r y c o n d i t i o n o f d i a g n o s t i c s u c c e s s , t h e e x - p l o r a t i o n o f a l a r g e r n u m b e r o f d i a g n o s t i c h y p o t h e s e s b e c o m e s a n i m p o r t a n t c l u e for s u c c e s s f u l y o u n g e r s u b j e c t s . M o r e t h a n a c c u m u l a t e d y e a r s o f p r a c t i c e , p r e v i o u s e x p o s u r e to s i m i l a r c a s e s m a y t h u s r e p r e s e n t a n i m p o r t a n t d e t e r m i n a n t o f d i a g - n o s t i c s u c c e s s , a s a l s o s u g g e s t e d b y t h e t i n y d i f f e r e n c e s o b - s e r v e d b e t w e e n t h e c h a r a c t e r i s t i c s o f r e s i d e n t s a n d s t u d e n t s . M a n y o f t h e s e p r i n c i p l e s h a v e a l r e a d y b e e n s u g g e s t e d b y m e d i c a l e d u c a t o r s b u t t h e i r i n t e r n a l i z a t i o n b y c l i n i c i a n - e d u c a t o r s r e m a i n s d i f f i c u l t i n p r a c t i c e . B y a c t u a l i z i n g t h e m , o u r d a t a r e i n f o r c e t h e g o a l s m e d i c a l t r a i n e r s s h o u l d s t r i v e to a t t a i n w i t h t h e i r t r a i n e e s a n d give c r e d e n c e to t e a c h i n g a c t i v i t i e s f o s t e r i n g t h e e x p l o r a t i o n o f d i a g n o s t i c h y p o t h e s e s r e l a t e d to t h e p a t i e n t ' s c o m p l a i n t a n d t h e i r u s e to f r a m e f u r t h e r d a t a c o l l e c t i o n . 19 W h a t e v e r t h e t e a c h i n g s t r a t e g y , it s h o u l d f a v o r t h e s i m u l t a n e o u s a c q u i s i t i o n o f k n o w l e d g e a n d p r o c e s s to r e m a i n o p t i m a l . 2° O u r r e s u l t s a l s o s u p p o r t t e a c h i n g p r o g r a m s t h a t offer e a r l y a n d s y s t e m a t i c a p p r o a c h to a v a r i e t y o f p r a c t i c a l c a s e s a n d d o n o t m e r e l y r e l y o n a r a n d o m a n d u n e v e n e x p o s u r e . T h i s s t u d y h a s s o m e l i m i t a t i o n s r e s t r i c t i n g t h e g e n e r a l i - z a t i o n o f t h e r e s u l t s . F i r s t , it w a s c o n d u c t e d i n a s i n g l e i n s t i - t u t i o n w i t h v o l u n t e e r s . T h e s u b j e c t s w e r e , t h e r e f o r e , p o s s i b l y m o r e m o t i v a t e d t h a n t h o s e w h o d e c l i n e d p a r t i c i p a t i o n , al- t h o u g h t h i s s e l e c t i o n b i a s w o u l d h a v e r a t h e r r e d u c e d t h e d i f f e r e n c e s w e o b s e r v e d a m o n g g r o u p s o f d i f f e r e n t l e v e l s o f c l i n i c a l e x p e r i e n c e . S e c o n d , a l t h o u g h t h e s t a n d a r d i z a t i o n o f t h e s e t t i n g i n c r e a s e s r e l i a b i l i t y , it m a y h i n d e r t h e n a t u r a l r e a s o n i n g t h e s a m e p h y s i c i a n s w o u l d h a v e w h e i 1 f a c i n g a re- al p a t i e n t i n a n a t u r a l s e t t i n g . I n c o n c l u s i o n , s o m e c h a r a c t e r i s t i c s o f c l i n i c a l d a t a collec- t i o n a r e r e l a t e d to d i a g n o s t i c a c c u r a c y b e y o n d t r a i t s m o r e di- r e c t l y r e l a t e d to c l i n i c a l e x p e r i e n c e . M e d i c a l e d u c a t o r s s h o u l d c o n s i d e r t h e m a s t r a i n i n g g o a l s for l e a r n e r s i n c l i n i c a l e n v i - r o n m e n t s a n d r e i n f o r c e t h e i m p o r t a n c e o f u s i n g a n e a r l y a n d w i d e e x p l o r a t i o n o f d i a g n o s t i c h y p o t h e s e s to f r a m e c l i n i c a l d a - t a c o l l e c t i o n . T h i s i m p l i e s a m o r e e x p l i c i t role m o d e l i n g o f c l i n - ical r e a s o n i n g a n d t h e a b a n d o n m e n t o f t h e still p r e v a i l i n g s e n s e t h a t e x h a u s t i v e d a t a c o l l e c t i o n i s t h e p r i v i l e g e d w a y to d i a g n o s t i c s u c c e s s .
We thank the faculty members, residents, a n d students who so willingly participated in this study.
Funding sources: Swiss National Science Foundation, Grant no. 3200B0-102265/1 a n d Elie Safra Foundation, Geneva, Switzerland.
REFERENCES
1. E l s t e i n AS, S h u l m a n LS, Sprafka SA. Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, MA: Harvard University Press; 1978.
2. Norman GR. Research in clinical reasoning: past history and current trends. Med Educ. 2005;39:418-27.
3. N o r m a n GR, Brooks LR, C u n n i n g t o n JPW, Shall V, Marriott M, R e g e h r G. Expert-novice differences in the use of history and visual information from patients. Acad Med. 1996;71:62-4.
4. Brooks LR, LeBlanc VR, Norman GR. On the difficulty of noticing obvious features in patient appearance. Psychol Sci. 2000; 11 : 112-7. 5. Leblanc VR, Brooks LR, Norman GR. Believing is seeing: the influence
of a diagnostic hypothesis on the interpretation of clinical features. Acad Med. 2002;77:$67-$69.
6. Leblanc VR, Norman GR, Brooks LR. Effect of a diagnostic suggestion on diagnostic accuracy and identification of clinical features. Acad Med. 2001;76:S18-$20.
7. Barrows HS, N o r m a n GR, N e u f e l d VR, F e l g h t n e r JW. The clinical reasoning of randomly selected physicians in general medical practice. Clin Invest Med. 1982:5:49-55.
8. N e u f e l d VR, N o r m a n GR, Barrows HS, F e i g h t n e r JW. Clinical problem- solving by medical students: a longitudinal and cross-sectional analysis. Med Educ. 1981;15:315-22.
9. H a s n a i n M, Bordage G, C o n n e l l KJ, S l n a c o r e JM. History-taking be- haviors associated with diagnostic competence of clerks: an exploratory study. Acad Med. 2001 ; 76:S 14-S 17.
10. Nendaz MR, Gut AM, Perrier A, e t al. Common strategies in clinical data collection displayed by experienced clinician-teachers in internal medicine. Med Teach. 2005;27:415-21.
11. Bordage G. Why did l miss the diagnosis? Some cognitive explanations and educational implications. Acad Med. 1999;74:S 138-S143.
JGIM
Nendaz et al., Predicting Diagnostic Accuracy
130512. N o r m a n GR, Barrows HS, F e i g h t n e r JW, N e u f e l d VR. Measuring the outcome of clinical problem-solving. Ann Conf Res Med Educ.
1977; 16:31 I-6.
13. CharUn B, D e s a u l n i e r s M, Gagnon R, Blouin D, v a n der V l e u t e n C.
Comparison of an aggregate scoring m e t h o d with a c o n s e n s u s scoring method tn a m e a s u r e of clinical reasoning capacity. Teach Learn Med. 2002; 14:150-6.
14. N e n d a z MR, Gut AM, P e r r t e r A, e t al. Degree of concurrency a m o n g experts in d a t a collection and diagnostic hypothesis generation during clinical encounters. Med Educ. 2004;38:25-3 I.
15. Norcinl J , S h e a J , Day S. The use of the aggregate scoring for a recer- tiflcation examination. Eval Health Prof. 1990; 13:24 i - 5 I.
16. N e n d a z M, R a e t z o M, J u n o d A, Vu. Teaching diagnostic skills: clinical vignettes or chief complaints? Adv Health Sci Educ Theory Pract. 2000;5:3-10.
17. P e r n e g e r TV. What's wrong with Bonferroni adjustments. BMJ. 1998;316:1236-8.
18. G r u p p e n LD, Wolf FM, Billi J E . Information gathering and integration as sources of error in diagnostic decision making. Med Decis Making. 1991; 11:233-9.
19. K a s s i r e r JP, Teaching clinical medicine by iterative hypothesis testing. Let's preach w h a t we practice. N Engl J Med. 1983;309:921-3. 20. E v a KW. What every teacher needs to know about clinical reasoning.
