INFORMATION TO USERS
Thismamacripr.basbeenreproducedfromthemiaofiImmaster. UMI filmsthetextdirectly fromthe original or copy submitted. Tbu.s.some thesis and dissertation copies areintypc:writerface,while othersmaybe fromanytypeof comput erprinter.
Thequalityofthisreprvductioaisdrpndent apoa the quality of tbe copy submitted. Broken orindistinct Print. coloredorpoorquaIity illustrations andphoto grap hs, print bleedtbrough.substandard margins.
and improperalignment can adversely affect reproduction.
Intheunlikelyevent thaI the authordidDOl:sendUMIa complete manusaipt andthereare missing pages.,thesewill benoted. Also,if unauthorized copyrightmaterialhadtoberemoved,anotewillindicate thedeletion.
Oversize materials(e.g.,maps.drawings,charts) are reproducedby sectioningtheoriginal. beginning at theupperleft-handcomerand continuingfromlefttorightinequalsectionswith smalloverlaps.Each originalisalsophotograp hedinoneexposureandisincludedinreduced fonn atthe backofthebook.
Photo grap hsincludedin theoriginalmanusaipthavebeenreproduced xerogBphicaUyinthiscopy. Higher quality6'"x9"'black:andwhite photographicprints are availableforanyphotographs orillustrations appearinginthiscopyfor an additional charge.ContactUMIdirectlyto order.
UMI
ABcU.t HaweUlDfolllWionCompany 300Nonh Zeeb Road, ADnAJbor MI48l06- 1346 USA
3131761-4700 &lJOI3;11-0600
i i
ABSTRACT
The goalof an alternati ve funding plan (APP) is to crea t e a fundingsystem wh ich ackno wledge s thatthe responsi bi l i tie s ofacademi c phys i c ians ext.endbeyo n d th e provisionof clinicalservicesto significant ro les in teach ing, researchandadminist.rative service.An alternat. i v e fund ing plan was pr o posedfo r St. John's acad emic pe diatr i ci ans. in the Ch i l d He a l thPr ogram , Healt.h Ca re Co rporat ion ofse.John's,Depa rt men t of Pediat r i c s , Me mori a l Un ive r s i t y of Ne wf o un d l and.
The ob j e c t.iveof thisstudywasto dev e l opa compreh e ns ive eva l uatio n protocol fo r the prop o sedAFPwhi ch ca n be usedtoas s e ss t.he impact of t.he AFP on: (1) pro v iders, interms of rese arch, t.e aching, ad min i st ra tiv e and clini ca l ca redutie s . (21co nsumer (p a r ent /guardian) satisf acti o n , DI unde rg raduateme dical stude nt co urs e sati sfact i on , (41 pos t-graduate pediat ric re sident sat is facti on, an d (51ccmmunity-Bas ednon-academic pro viders ' pediatric....ork loads.
The pro t ocol deve lopedin th isstudyemplo yed (11 que s t i o nna ires (provid e r an dco nsumersat i s factio nand st u d entasse s sme n t of teachi ng quality) . 121 provincial medi caldatabase inf o rma t i o n for non- a c ade mi c phys icians,
i i i (3 1in f o rma ti onglea n e d from the ChildHealthProgr am, Janewaydatabases (pa t ie n t volume , services, andwai t i n g time s), go v e rnment informat ion(o v er a ll budg e t ch anges, provide r inc o me s) and (4) faculty informa t i on (r e s e a rch acti v it ies, in c o me, admin istrat i on !orqa niz at i onal activities,con t i n ui n ged uca t ion, physician re c rui tm ent and turnover and departmen t and faculty innovations l.Sele cted data collectioninstruments and procedures forthe ev a l ua t i o n protocolwe r e prete s t e d to determine their ap p r opria t e n e s s and completeness .
Dedica t io n
for mygra ndm o t. h e r
Ro••Anna (Moor e) Kennedy
(1910-19 96)
Awoman of qreat love, faith, .t re nq'th, wi.dom,
And Peac••
iv
Acknow1edgments
I would l ik e tograteful lyacknow l edge theguida n c e an d en co urageme nt ofmysup e rv isors Dr. Doree n Ne v i l leandDr. Wayne Andrews. And thankDr s. Jor g e Se g o via an d Vareesh Gadagfor the ir participationas members of myth e si s commi ttee .
Man y thankstothe people, in myria ddepartments, in the Ch i ld HealthProgram, HealthCa r e Corpo r a tionof St.John's, who providedinvaluab leinformation anddi r e c t i o n dur ingmy study.
Thank you toJo hn H.C. Pippy, fortec hnic a l guida n ce . TomymotherBe t t y Lou, andmyfath erPat r ick , I am ever thankfu l for their encouragementand support.
Tomygrandmot her Viv i e nn e, thankyou for your love and fait h inme.
To Doug l a s C. Pip py, my moral supporter, wh ose friendship an d lov ehe l ped make this proje ct a good expe rience.
This thes i s was produced while I held a graduat e fellowship fromtheSc hool of Graduat e St ud ie sand the Facult y of Me d icine,Memor ial Univ e r si tyof Newfoundland .
vi TABLB OF CONT ENT S
Abstra c t . . . Acknow l e d gme n t s Li st of Tables
i i
ix Glos sary
Ch apt erI Int roduction .
1.1Relevant Research . . . . 1. 1.1 Reimbu rs e me nt Met.hods and Th eir Impa cts 1.1.2 Alterna tive Fund i ng Plans For Academic
Pediatricians . . 9
1.2 The Proposed AFE' . . • . 12
1.3sc a ceeenc of the Problem 1)
1.4 StudyObj ect i v e s . 14
1.5Assumption s 15
Ch a pter II Meth ods . . . . . . . . . 16
2.1 sel ection of Prot.oco l De sign 16
2.2Setting . 17
2.3 Sample. . . . . . . . . . . 17
2.4 Instrument.a tion . . . . . . . . . . . . . 18 2.4.1DomainandInd i c a t o r Identifica tio n 18 2.4.2Loca tionofDataSOurces . . . . . 21 2.4.2.1ProviderSatisfact ionSurvey 21 2.4. 2.2Cons u merSat i s factionSurvey 24 2.4.2.3 Undergraduat e Medi calSt uden t
Surve y . . 33
2.4. 2. 4Post- g radu at e Pe diatri c Re side n t
Survey . . . . . . . . 34
2.4.3 Identificationof Acces s Requirements 35 2.4.4 Pre tests . . . . . . . . . . . 35 2.4.4.1Provid erSurveyPretest 35 2.4.4.2al Consumer Satisfaction
Pre testI . . . . . 37
bl ConsumerSa t i s f a c t i o n
Pret e s t II . . . . . . . . . 37 2.4.4.3 Under gra d u a t e Studen t Survey
Prete s t . . . . 39
2.4. 4 .4 Post-g r aduat e Pe d i a t r i c Resid e n t Surveyare cee c . . . . . . . 40 2.4.6Finalizationof a Forma l Evaluation
Pro t.ocol 41
2.5 Et.hical Issue s . 41
vii 2.5.1 Ethical Issue s Arisingfro m theUs e
of Exi st i ngDatabases . . . . . . . 41 2.5 .2Ethic al Issues Aris i ngfrom the Use
of Questionnaires . . . . . . 42
2.6Data Analysisforthe Pre t ests . . . 43
Cha pte r III Resul t s 45
3.1 Domai n and In d icato r Ident i fica tion 45
3.2 Data So urces . 45
3.2 . 1 ExistingDa t a base s . . . . . 45 3. 2. 2Acce s s to Exi st i ng Data b a s es . 55 3.2.3 Prete stResults for Sa tisfact i o n
Ques tionna i r es . 56
3.2.3.1Prov i de r Survey . . . . 56 3.2.3.2Consume r Sa t is fac tion . 63 3.2.3.3Unde rg rad u a te Medica l
Stu d e n t s . . . 73
3.2.3.4 Po st -graduate Pediat r i cReside nt
Survey . . . 75
3.2 .3.5No n-academ i c Co mmuni t y-based
Pr o v ide r s . . . . 77
Chapte r IV Discussionand Recommend a tions . . 78
4.1Protoc o l De sig n . . . . 78
4.1.1 Identif i cationof Scheduleof
In s t rument Administra t i on . 79 4.1. 2 Pot e ntia l Con f ounde rs . . . 80
4.1. 2 . 1History . . . . 81
4.1.2.2 Seasona l Va riation . . 81
4.1 . 2 . 3 Instrumentation . . 81
4.1. 2 .4 Uncont rolledSelec tio n 82 4. 2 Data So u r ces . . . . . . . . . . . . 82 4.2.1 Existi n g Da t abases Changes . 82
4.2 .2 Satisfact ionSurvey Changes . 83
4. 2 . 2. 1 Prov i d e r s . . . . . 84 4.2.2.2 Cons u me r s . . . . . . . . . . 85 4.2.2.3 Und erg ra d u at e Medical Students 92 4.2.2.4 Post-g r adua te Pe d iatri c
Reside nts . . . . . . . . . 94 4. 3 DataAnalysis for the Eval ua t i on 94 4.4 Sug g e stions fo r Complement a ry Progra ms and
Further Res e a rch . . . . 95
4.5 Limi t a t i o n s ofthe Study 97
Cha pter V Con c lus ion 99
Referen ces . . . . . 101
ChapterI INTRODUCTION
Academicphysicians are clinicalprov idersofcare, but alsofil l role s as researchers, educa t o rs, and
admin istrato rs.These phys icians find it achalle nge to balancesuch distinct activitiesef f e c t ively(Cad man, 1994 ) . Recen tly, several authors have explored the stres s escause d byineffective bal ancingof such profe ssional activities amongac a d e mic physic ians (Linn, Yager, Copeand Leake, 1985).
Diffic u l tiesexper i e nced by academi cphysic ians in ba l a ncingduti es have a mul t i tude of potentia l ca us e s. The most commonlycitedcauseshavebeen those based on remune r ationmetho ds (Glaser, 1970 an dBa bson, 1972).There are four broadmethodsof paymen t th a t have been explored . The traditiona l th r e e incl ude ; (1)fee-for-service, a service volu mebasedmethod , (2)s a l a ry, a time-basedme t h od., and
(3) ca p i t at i o n, a popu lat i o nsize basedmethod (Fourn ier, Con ta n d r i o po u los andPi n e aul t, 1984) .The fourthis a more recentde s i gn exclusivelyus ed in aca d e michealthca r e ce nte rs- th e al t ern a t i v e funding plan' (AFP ) method, which
'I t couldbearguedtha t thisactuallydescribesan alternativepayment plan (AP P) , no t an AFP, but after
involves mixturesof the three t raditionalmethods (Ha s lam andwa l ke r . 19 9 3, ccvee, 1995, and-Que e n 's Health Policy Un i t's AFP Evaluati on wo r k s h o p Background Paper", 199 6) .
Any remun era tio n method invol vesince ntives (Wright.
19911 whichare essential considerationsin physician behav ior. Indeed, it is the belief of some researchersthat inefficientbalancingbehaviorexhi bit edby somephys i c ians canbeexp l a i n ed in part by the setof in centives as sociated with the particular remunerationmethodch o s e n (Hick so n, Alt e me ier and Perrin . 19 8 7, Casa lino. 19 9 2, and Birch.
1994).Sinceeac h remun e ra tio nmet h od ha s a un iquesetof incentivesat wo r k,changi ng the remunerat ionmethod potentiallyaffects the physicians' balanceof ac tiVi ti es.
{Birch, 19 94}.
Recently, several academic he althsciencecenters acrossCa n a d a have introducedalternative fundingplans;
Univers ityof Toronto, 199 0, Que e n ' suni v e r s ity, 1994 and Dalho us i e Unive r si t y 19 95.One of the goalsof anyAFP is to cre at e a remu n e ratio n systemwhich acknowledgesthat the responsibilities of academicphysiciansextendbeyond the prO Vi sion of clinicalservices tosign ificantrol es in teaching. re s e arc h and administrativeservice. It is hoped that the ne win c e n t i ves createdwi l l facilitatean effective
considerationof the difference. the term AFPwa s retai ned in thisdiscussion.
ba l a n cin g ofac t iv i t ie s forac ade mi c phys icians(Has l a man d Wa l k er, 19931.
Academic pediatriciansin the Department of Pediatric s, Facultyof Me d i c ine, MON, haveexploredthepossibilityof des i gningan d implemen t i ng an alt e rna t ive fundingplan . Their cur re nt system of remuneration is primarilybased on earningsfor clinica l servicesthrougha fee -for-service mechanism .Aproposed AFP has been developed by eneee pedia tri cia n s. It wi l l enta il a sh i ft fr oma primar i ly fee- for -service mec h a n ism to remunera t i on by sala ry.
The develope:rsof th e AFPforSt.John'sacademic pedi a t r i c i ans identifie:dt.he needfor a compre:hensive ev a l ua t i o nof the proposed AFP.
1.1Relev-.nt Reaearch
There were two broadareasof researchrelevant to th is stud y , (1) ReimbursementMe t hods andTheir Impac t s,and (2) Alt e rnat i v e Pa yme n t Plans forAcadem i c Pedia tric ian s.
1.1 . 1 Reimbura ementMe thods andTheir Impacts Studyof reimburse me nt methods in he a lthcar e has con s i s ted largel yof opinion s conc e rn ing behaviorchanges of th e ph y s i c i a n (Glas er, 1970and Babs o n, 19721.Gl ase r and Babsonwh o summarized various methods con cl u d edthat
rei mburs e me nt met hods resear c h was bas edex clu siv e l y on shadowcontrols , judge mentsofexpe rt s, progr a m administrators, andpa r ticipa n t s. This typ e of methodology usuallylackssub s t a n ti a l evident ialbasis (Ro s si, 1993).
The study of remunerationme t h ods doesnot have to ad d ressthequestionof howmuch a ph y s i c i a n is paid but how tode s i g n a paymentmechanismthat fac il ita te s theal i gni n g of ro l es and functions and the common sharedhealthca r e objectives wi t h reasonable levelsof compensation (Ba r e r and Stoddart. 19911.
Co n c e rnove r the adverse incentives associatedwit h va r iouspa ymentmethods have been exp ressedinmany pa pe rs. Co n t a n d rio poulos, Champagne and Pineault (l986) gave a Canadian pointof view when dis c ussingthe advantagesand disadvantagesof fee -far-service, ca p i t.a t i o n and salary arrangementsfor physician remune ration.Some advan tagesof salaryremuneration identifi e d inthis paperin clu ded:
enhancement of t.eamwork, pa r ticipa t i o ninmedical - administrationactivitiesand participat i on in prevention ori en tedacti Vities. The nega t iveaspectsof salary remunerationcons i d e re d in the samepa per centered around pr od u c t i v i ty . Babson (1 972) , Reinh a r d t (1 9 84 ) and Co nta n d r i opou los. Champ a g n e and Pineault (1986) asserted that salary payment underm ines pat.ientand physician
autonomy and lowers physician motivation, productivity and type and numbers of services provided.
Others have concentrated on one form of payment. For example Casalino (1992) advocates that the use of a fee-for- service system ensures adequate incentives for physicians remain in place. Consulting vs. procedural distortions caused by fee-for-service payment were identified by Opit,
(1984) and noted by Barer and Stoddart, (1992) in their Canadian Health policy review. Charles J. Wright highlighted the shortcomings of the current system for reimbursement in his 1991 article, "rhe fee-for-service system should be rep Laced". Wright summarized the beliefs reflected in the papers mentioned earlier in this paper by saying:
The current fee-for-item-of-service system causes gross inequities in physicians' remuneration ...
It also results in disproportionate recognition of procedures, undervaluing of counsel ing services and disincentives to productivity in all activities but clinical ones - areas such as medical education, planning and research suffer.
(Wright, 1991)
Alternatives to fee-for service were presented as capitation (National Health Service, United Kingdom),
contracting an dsala ry (Fo u rn i er, Contandriopoulos an d Pineau1t ,1984 and Thornhill, 19911~.
Pineault, Contandriopou losand Fournier(1 9 84) conducteda study of Ouebec ph y s i c i a n s in19 81 to determine their ac ceptan c e of an alt e rna t i v e to fee-for-service pa yme n t. Two- thirdsof thephy s i c i a n s surveyed woul d have accepte d a ti me - b a s ed remun e r a tion sys tem to re p l a ce fee- for-s e rv ice payme nt. Howe ver, concerns remainedove r pr ofessional au to no myin content of workand pract i c e or g a n i z atio n. Of par ti cular impor t ance to thisdi scu ssion were theresul ts from the questi o n s on pract i c e change s . The aut ho rsfound that general pr a ctitionerson a t ime- b a sed re mune r ationsystemwould de c rease,or at mostmaint ain, timedevotedtopa t i en t careand increasetime spent on continuingeducation, teaching, research, community health and prevention.
Lahaie an d Chopyk 1199 21and Guilfoyle II98 S) studied the useof salary remune rationfor rural physiciansin Manitoba and withina heal t hcenter environment. The prog ra m evaluationundertaken in1990 observedincrease s inservices such ascou nsel i ng, healthpro motion, home visits,
lca pita tio n is a payment method des igne d topaythe phy s i cianaccordi ng to theamount of re s p on s ibi l i t.yheld (Ifof patien t .inpracticed .Salary is a time-ba s ed met ho d and fee-for- service isa servi c e ba s ed met.hod.(Fo u rni e r, Contandriopoulosand pLneau Lt; , 19 84 ).
prevention . sat.ellitecli n ics andspec ialt.y cl in i c s.
improve ments inpee r relationshi ps (no longer see nas compe t.i n g for patients . mo r e equ a l i t y ) . in c r e a s ed conti n ui ng educationactivities among salariedph y s ic ia n s.and in c r eas e s in time spent wi t hpatients. There were concerns ove r lossof aut o n omy expre s s e d by moreexperienced phys icians (t ho s ewhohad practi c edunderthe fee-f or- servi c e system). th e op p res s i v e tax struc ture of personal incom e andunevenl ydistribu ted wo r kloads .
Accordingto Hickson. Alt e meie r and Perrin (1 9 87), behavior, attitud e, practicepri ori tyan d moti vation modificati o nsaccruing topaymentmethodchang e s mus t be id en ti f i edandmeasuredinan evaluation of a ne w payment mechanism .They ascertained th a t pediat ri c practic e will likelybe infl u e n c e d by new re i mburs e men t techniques.Us i n g a ra ndomizedexpe r i mental desi gn , the y al l o c a t e dphys i cians tobe paidby fee-far-servi c e or salaryandfo l l owe d a set of va r i abl e s for nine months . The findings in clud e d that fee-far-service physic iansscheduled more visits per pat ient.
than didsalar iedphysiciansandsawtheir patientsmore often (1. 4 2 visits versus .99 visits). Fee- f ar -service physiciansalso providedbe t te r continuityof car e by attendi ng86.8\'of the vis it s made bytheir patients
(s a l a r iedphy s i c i a n s attended 78.3\ of visits) and encouraginglesseme r g e n cy vis i tsper enrolledpatient.This
studywa s the on l y publis hed syste ma tic st udy of the effects of salaryor fee- f or- servi cepayment onpediatri ci ans.
However, i twas a study of pediatri c residents, not academic pediatric ians (Hi c kso n , Alt.emeier and Perrin, 198 7 ) .
Oth erst udiessu c h as Bjorndal , Artnt zenandJo h a n s e n (1 99 4 ), stud i edthe effec t s of sala ryversus fee - f ar -serv ice paymenton the behaviors of general pract i t ion e rs con c e rn ing wo r k ing hours, pat ient turnove r and pat ient characterist i c s. The y found tha t fee-for-se rvi c e phy s ic ians ·worked atecec exc l us i vely wit hpa t i ent s in their own pract ice, wh i le salar ied GPs spent mor e ti me out of offic e - (5 5\ ofwor ki ng hours ). Th e fee-fa r- s erv icegro up had more consultati ons (a vg. 2.68 patients per hourversus 2.37pat i e n ts perhour forsa l ar i e d ph y sic i a ns ) .These findingswer e not suffi c i entlysigni fi cant tosupport the authors' hypot h e sis tha t sa lary reimburs e ment le ads to fewe r and longe r consul t a tions. Salariedphysici anshadmor etelephone co nsu l t at i o nsperhour thanth e fee-far - s e rvi cegroup.Also, the fee- f o r- servi c e ph y sic ian swork edanaverag e of42ho urs perwe ek; whereas, the sa l a rie dgroup averaged38 hours per we e k. In thi s st udy cons u l t ati o n ac t ivityandpati ent populations we r e not foundto have evo lve ddif fer e n t l y betweenfee-for -service and salariedgeneralpracti c e pro f e ssi o n a l s.
The pr o po sed AFP fo r St.J o hn' s acade mi c pedi a t ricians wil l entai la reo rga n izat io nof the payment pl an. The payme nt mechan ismwi ll beestabli shedona sal a ry rat h e r thanfee-f ar - s e rv i c e billing.Therefore, as pa r t of the searc h foranappropriateevaluationtool, re s earc h regardingbehavior, work effort andou t p ut ch ang e s mustbe explor e d.
1.1.2 Alterna t i v e Funding Plans !"orAc a d emic Pediatricians
In Apr i l 199 0 the universityof Toronto' s De p a rt mentof Pediat ricsat The Hospitalfor Sick Chi l drensigne d a AFP agreementwit h theOn ta rio Ministryof Health. The plan was structuredto fulfi lth e principal goa l of est a bl ish ing a stableandsecur e fundingba seforacademic medic ine . The depa r t ment wa s facing shrin k ingresourc e s, includi ng st a gn a n t fee sch e d ule s , and increas ing time sp e n t dev ot edto clinic al care , wi th a resultan t de-emphasison teachi n g and re s e arc h (Has l a man dwa l k e r, 1993) .The reorganiz a ti onof th e pa yme nt planfor the departmentas awhol e requ i red changes. For individual clinicia nsth e new pl anenta i l ed:
ashift fromfee-for -se rv i c e co nt ribu t i o ns to the departm entIscentral fund, wi tha guaranteed annu al salary, to a ne goti ated annual sala ry poss i b ly au gme ntedwi thmerit pay tore fl ect achievement inthe areas of res e a r c h, se rv i c e andte aching. tc c vce, 1995)
10
According toCo yt e (19 95). the int r odu c tionof the Un i versityof Toro nt o, Departm e n t: ofPedia trics AFP increas ed the financial stabilityofth e department andalso redu c e d the ince nt ivefo r it s members to see large numbers of patients in orde r to "earn thelr-fu ll-sal a ries ·(Coyte. 19 95 ).The AFP al l ows forthe me mberstospe n d moreti me in re searchand teaching, and correspondinglylesstime in cli nic al service .
After a sh o rtperi od of negoti ations. the Ont ario Ministry ofHea l t h andth e Departmen t en d ors edan AFPwhi c h held theDepartment to ope r a t e wit hi n a globalbudge t ba sed onclin i c al services billings tothe OntarioHealth Insu r ance Plan (OHI P ) an d ot h e r source s, na me l y un i v e r s i ty funds and rese arch gran t s.The ful l· t ime facu l ty members were ass ureda sala rytoprov ide clin ical ca re, research , teaching and administrative services andwere accountableto the Chair of Pedi atrics.Altho u gh the plan has beenactive fo r six yea r s , th e r e ha s bee n no eempr-ehens tve evaluation compl e t e d.
Thear e a s of the Unive r sityof Toronto 'sDepartment of Pediatricsat SickChi ld r e n ' s Hospital's plan reviewedto date inc lud e the following va ri ab l e s: number of ho spital admi s s i ons, numbe r of emer g encyvis i t s, phy sici anturnove r
11
and recruitment, innovations, Bub-specialty clinic visits, publications and impact and research funding. The number of hospital admissions has remained constant since the introduction of the AFP (a p p r o x i ma t e l y BODO per annum) (Haslam. 1996). However, there has been a drop in patient days due to decreases in length of stay and shifts to ambulatory setting care. The acuity level of pediatric patients has risen more than in other departments in the Faculty. There was a decrease in emergency visits but it was coupled with a 20\ increase in Bub-specialty clinic visits.
The department has experienced a 14\ increase in shadow billing since the introduction of the AFP. There has been a 7\: increase in the number of University of Toronto graduating medical students seeking pediatric residencies at Hospital for Sick Children and pediatric residents from the university have improved MCCQE scores. The number of annual peer-reviewed publications per faculty member rose to 4.8 from 3.23, from before the AFP was introduced. And research funding rose, with number of investigators remaining almost unchanged over the 5 year period (Haslam, 1996).
The AFP at Queen's University, Faculty of Medicine, took effect in July, 1994. This AFP is similar to the University of Toronto, Department of Pediatrics at Hospital
lpre-AFP calculations based on data in Haslam, (1995).
12 for sick Children'Bplan. with the exceptions of (1) the breadth of inclusion (the entire Faculty of Medicine) and
(2) they did not initially include part-time faculty. The Queen's University (Kingston) AFP included SEAMO, South Eastern Ontario Academic Medical Organization. which is comprised of five members: Queen's University, Kingston General Hospital, Hotel Dieu Hospital, Providence Continuing Care Center and Clinical Teachers Association of Queen's University (Sinclair, 1996). The granting of the funding envelope has allowed the inclusion of all academic medical professionals. Queen's University Health Policy Research Unit recently developed a provider satisfaction
questionnaire designed to gauge the faculty response to the AFP. The results of this survey will be available in late May 1997'.
Other provinces are in the process of developing AFPs;
University of Ottawa, University of Saskatchewan, Dalhousie University and trnfver-s Lt e de Montreal are in the preliminary stages of AFPs.
'Personal communication with Malcolm Anderson and Jarold Cosby of Queen's University Health Policy Research Unit.
13 1.2The ProposedUP
The goals ofthe pr o posed Dep a rtme n t.of Pedia t.ric. Me mor ial Unive rsit.yof &ewfoundland 's AFPfor ac a demic pe d iatric i answere ou t line dina brief prepar e d for the Department of Healthin1996. It calls for the establishment of a remu n e rat i o n sys t.em wh i c h: (1) prov i d es a stable fundingbase for theac a d e e j,c Department. of Pe d ia t r i cs, wi t h educational, ree e a r e n , clinical an d adm i n is t rati ve responsibilities en c omp ass e d therein, (2) offerscompetitive sal a ries, re f l e cting yearsoftrai n ing. seni o r ityand rank, intens i tyof work, te achingcommitments , academi c pr odu ceiv iey. andamountof on-calland ou t of towncl in ics heldper an num , {] l offers a fair and mutual oppor tunity for annual salary adjustment s and disput.eresolut.ion , (4)recognizespart.- t.i mefac u l tyas ess e n t. ial to the Departm ent of Pedi a tr i c s, (5) ensur e s t.he main t.enanceof compet.e ncefor ac a d e mi c medi c a l faculty (t.r ave l foracademic paper present.at.ionsand cou r s e exp e n s es) and maintenanceof app r o pria t e le ve lsof admini s t r a t. i ve su p port. (s ee Append i x A fo r Role , Mission and Goalsof theAFP for the Dep artme nt. of Pedi a t.ri c s, Me moria l universit.y of Newfo u ndl a n d ).The pro p osedAFP fo r academicpediat r iciansin St.John'a ess e nt. ial l yfollow s theoutl ineaof th e Universit.y of To r o n t.oand Que en'sUniversi t yAFPs, wi t h modificationsmade
14
to reflect the unique regional requ i r e men t s of pediatri c academi c medic ine in New f ound land .
1.3St .t ement of the Pr obl ea.
Thepu rpo s e of th i sstudy wa s todeveLcp a
co mp rehe ns i veeval ua tion protoco l fo r the propo s edAFPfor ac a d e mic pediat r i c ian s in the Departmen t of Pedi at. r ics, Memori al Unive rsi tyof Newfound land (MON) .
1.4StudyObj e cti v es;
1/ To develop th e proto col an d
inst rum e n ta t i on fo r the eva luationof anAFP forac adem i c pedia triciansint.he Child He alt h Program, Healt hCareCorporationof St.John's {HCCSJJ and the De p a rtment. of Pe di at ric s, MON.
2/ To prete stselect ed datacollection instruments andpr o c e d ures forth eeva l u a t i o n protoc o l.
Thisstudy develo pedthe inst rume n t a t i o npr ot o c o l fo r the future dat a collection an d an alysi s fo r the
comprehensiveevaluationof theAFP.The comp l eteeva lua tion
15 of t.he pr oposedAFPwil l t.akepl a c e over anextended pe r iod of thre eyear s andwi l l beunderta ken at. a la t e r dat. ebya re s e arc htea m usi ngthe protoc oldevelope din thisstudy.
Therefo re, focus of the analysis inthi s studywason the as sessmentof theappro p ria t e n e ss .completeness, val idity and reliabilityof the inst rume n tat i on.
1.5A8SUll1Ptione
The as s umptions basi c to thisstudywer e:
1. Acade mic pedi at ricians ar e chall e n gedwi t h a no n -optima l balance of ac t i v i t i e s and remuneration'.
2. Thepropose dAFPwil l introduce a new set of fi n a n c i a l incentiveswhic h will facilitatethe acad em i c pe di a t ric ian s' eff ecti v e ba lanceof acti v ities.
,Thispr o b lem has beenid e nt.i fi ed by the Departmentof Pediat ri cs, Memorial University of Newfoundl a nd , an dthey have chosen to add res s the problem with a newalternati ve funding plan.
16 Chapter II
Methods 2.1 Selection of ProtocolDesign
The most rigorous evaluation of the impact of an AFP on pediatric health care in the province would be a randomized trial, wherein physicians would be randomly allocated to the AFP or the current fee for service remuneration system and variables such as cost, quality of work, productivityand satisfaction could be monitored over time. This design however is not currently feasible within the region as only 31 pediatricians will be potentially affected.
A quasi experimental design, in the form of independent pre and post tests', was therefore chosen for the protocol. This design will require one experimental group (the academic pediatricians in the province) and multiple observations of this group before and after the intervention (in this case, the intervention will be the introduction of the AFP) .
"Depend i.nq on the dependency observed between tests, the analysis may have to consider the study as a repeated measures test.
17 2.2S.~ t i ng
Theeva l ua tion protocol was develope d incollaboration wit h seve ra l institutions including: the Chi l d Health Program, HCCSJ, and the Depa rtmentof Pediat.rics, MUN.
Fa c u l ty ofMedi c i ne,St. John 's.This work involv e d four differentsi t es; theJanewa yHospi tal. Departmentof Pediatrics . Fac ul tyofMe d i c i n e (Heal t hSc i e n ce Centre ) . Me dic a l Care PlanOf f i c e and the Departmentof Health for theProv i nceof Newfo undlandandLa b rador.
2.3 Sample
The six target groupsforth i s study in c ludedchose whichwouldpotent-iallybe imp a c t ed uponby the introduction of the pr oposedAFP. The sampleincluded membe r s from the followinggroups: (i ) al l providers (f u ll - t i me andpart-time academic pedia tricians in the ChildHealth Program. HCCSJ.
and the Departmen t of Pedi a t ri cs . Facultyof Medicin e. MUN) . {2 1 a convenience sample of 15 consumers {p a r e n t s or guardians ofpat i e n ts in theChildHealthProg r am, HCCSJI .
l31 all Undergra duateMedicalStudents {s t u d en t s in the Growth andDevelopment cour s e inWi n t e r semester 97 1 . (4) all po st-gra duat e Pediatri c Residents (16 inWi nt e r semester 97). and (5) a conve n ience sample of no n-acade mic communit y basedpr o vid e r s (five individual s froma group of prac t icing
18 non-academic general and family practitioners in the se.
John's Metropolitan region). The sixth group consisted of a convenience sample of personnel at the Department of Health, Medical Care Plan Offices, and Memorial University, who provided information about the overall environment setting to the investigator.
2.4 Instrumentation
Identification of appropriate strategies to evaluate the impact of the AFP proceeded in several steps: (1) domain and indicator identification, (2) location of data sources to appropriately measure selected variables,
(3) identification of access requirements for data sources, (4) pretesting of instruments, (5) identification of appropriate schedule administration for instruments. and
(6) finalization of a formal evaluation protocol for the proposed AFP. Each step is discussed below.
2.4 .1Domain and Indicator Identification: The variables chosen were based on the objectives of the plan and the principles for alternate funding of the Department of Pediatrics at Memorial University (Appendix A). They included all areas and groups of people which could be potentially affected by the AFP. Five groups were identified
19 tha t cou l d be potentially impa c t e d upon. Th e primary domains of impact and appropriate indicators foreach group were ou t l i n e d through extensive consultation wi t h members of each potentiallyaffected group (s e e Table 2.1).
Tab] e l' Groups Doma i n s and Indicators
Group Domain Indicator
1. Providers A. Clinical a.work satisfaction and
Care workload
b.number and type of services provided c. length ofstay
d.waiting times (r e f e r r a l to consultation) e. emergency room visits f. number of admissions
B. a. overall budget
Administrat i on
b.provider income c.physicianturnover and recruitment
d. degr e e of continu ing education
e.number and depth of innovations Ce g:
travelingcl i n i c sl f.activityin professional
organizationsand public orcOllITIUnity service
Gr ou p Domain Indic ato r Providers C. Re s ea r ch nUl1lberof academic
Con't. publications
(pee r-.ndnon peer revi ew)
b.numberof citatlODJ1 trOll! publiabed_ter1al s
Propos.lsvri tten (a l l re v iewe d, fund"'dor unfunded) d.Bxt e rnall y funded re search projects
DwnMrofclini ca l tri.ls{f und edlUld
\,ID.fundedl
t.GroSII ResearchFunding S·II.cademicawards D. teac hi ng seecell (3Aa) 2. Cons umers A. Quality of a,Satisfact i o nwith
Care care
L Onder- A. Sati . fac tion a,student course graduate wi thTe a c hing evalua t ion
b. studentresearchand pa pera
4.Pediatric A. Sati s f action a ,student program Post Graduate wit hTeach ingand evaluati o n Medical St ud en ts Supe rvi sion
b. supervi sory positi on s of facu lty S. conmunity A.Clinical a.MCPbi l ling
Base d Non- services practice changes
acad emic {pe d iat ric popula t i on
providers only}
20
21 2. 4.2Lo cation of Oat.. Sou r c ••: Identificationof data sources to measure changes in the selected indicators resulted from consultation with members of the potential ly affe c t e d groupsand a review of the relevant instrumentat i on literature.Appropriatein s t rume n tswere designedif no su itable data collection sourc e cou ldbe lo cat e d.Data sourceswer e of twomajortypes : (1) existingdatabases, and (2) surveyinstruments.
Mostindica t ors (ex c l u d i n gtho se re latedto sa t is f act i o n ) cou l dbemeasuredthroughex i s t i n g da tabas e s. However, no existing data bases provided a measure of satisfactionfor eachof the fourgroups pot e ntially imp ac t edup o n by the proposedAFP. 'rne rercr-e, validatedand reliable in s t ruments were loca te d through a li terature revie w andas se ss e d forthei r appropriatenessand completeness for use inthe target settings . Fo u r survey questionnaires were adapted or developed for the purpose of this study , as no t e d below .
2.4.2.1 ProviderSa t i s fa c t ion Survey
Provid e r satisfac t i o n is animporta n t compone ntbecause it allowsfor an assessment of physicianbehaviourchanges attributabl e to the AFP (CosbyandMidd l e t on , 1996).
TWoexist ing quest ionnaireswe r echo s e n to measure the impac t of the AFPon proViders. The fi r s t questi o nnairewa s
22 a work satisfaction measure ent::.itled "The Hospital for sick Children Department of Pediatr::i.cs Physician Survey 1996", which was developed by the HMR~(Sandra Leggatt) at the University of Toronto (April, :::t996), and adopted by the investigator as section one of the Child Health Program, Department of Pediatrics. HCCSo,J, questionnaire.
The second questionnaire "Ch o s e n , entitled "rhe Alternative Funding Plan and t~eProfessional Activities of Medical FacultyCl1996"was deedsqn e d at the Queen's Health Policy Unit (Cosby and Middlet on, 1996) at Queen' s University, Kingston. and had "been administered to all academic physicians in the Spr ing of 1996 (Cosby and Middleton, 1996). This questionnaire focused mainly on measuring self-efficacy (perce ptions of the opportunities created) and outcome-efficacy (perceptions of the value of change) of the providers affec:ted by the AFP (Cosby and Middleton, 1996). The Universi..tyof Toronto questionnaire made up the second section of the Child Health Program, Department of Pediatrics, HCCSJ, questionnaire and was also designed to gauge overall int~ntand interest of faculty in changing their behaviors. It i...s ideally suited for an ex- ante evaluation because it me easu ree bot.h intent and actual behavior shifts, in terms of tcIme allocation to activities and the relative perceived im~ortanceof the AFP to the faculty pediatricians.
23 The twoinstruments , from the University of Toronto and QueenIS Health Policy described above. were combined to form the Child Health Program, Department of Pediatrics, HCCSJ, questionnaire entitled -The Proposed Alternative Funding Plan : The Physic ian's Perspective" (see App e n d i x B). The work satisfaction questionnaire from Toronto (Leggatt, 1995) formed section one and the AFP attitudes questionnaire from Kingston (Cosby and Middleton, 1996) formed section two. The questionna ire from Kingston was slightlymodifiedby changing the verb tense (to future from past) and by the term "pr-oposed" being added before all references to the AFP. The original designer did not feel these slight modifications would seriously affect the re l i a b i l i t y or face validity of the questionnaire (personal communication with Jarold Cosby, February 1997).
Inter-item reliability for the questionnaire from the Queen'sHealth Policy Unitwas measured by Cronbach's alpha coefficient and found to be .8 1 ; a significant level of reliability <Cosby and Middleton, 1996). Factor analysis and principal components were used by Cosby and Middleton to ascertain construct and face validity. They found that
the majority of the variables are measuring a similar construct that is based on faculty perceptions of the AFP ... a three factor solution which accounts for over 40% of the variance.
24 Reliabilityand validity scores are not ye t av a i l a b l e for the University of Toronto work satisfaction survey.
In adherence with the Queens University, Kingston, designers' specifications, the physical appearance of the childHe a l t h Program, Department of Pediatrics, HCCSJ, questionnaire administered at the Child Health Program, HCCSJ, followed that of the original Kingston questionnaire, with one exception. The booklet for the ChildHealth Program, Department of Pediatr ics, HCCSJ, questionnaire had a buff cover in place of the original grey cover (see Appendix B for questionnaire) .
2.4 .2.2ConsumerSatisfact ion Su rve y It is important to attempt to measure
satisfaction because, according to Larsen et al. (1979) , wit h o u t the clients' viewpoint , an evaluation of services is bound to be biased toward the evaluatorsI perspective.
There is a tendency for patients, or in this case, their proxy (parent/guardian), to report high levels of satisfaction re g a r d l e s s of the service provided (Brown, Sheehan, Sawyer, Raftos and Smyth, 1995). The patient satisfaction data from young guardians (under 18 years) patients older than 60 years is particularlyvulnerableto acquiescent response bias, whereby the respondents tend to
25 answer all questions in a positive manner indicating high satisfaction (Linn, 1975 and Simonian, Tarnowski, Park and Bekeny, 1993). The higher levels of satisfaction reported from some pediatric satisfaction studies resemble the primarily positive responses found with satisfaction data from elderly patients (Ross, Steward and Sinacore, 1995 and Rees Lewis, 1994). And i t has been suggested by Ware (1981) that this phenomenon of higher levels of satisfaction reported, called upper limit clustering, is caused primarily by lack of item variability. Upper limit clustering could be primarily solved bythe assignment of both positively and negatively worded questions. Positively and negatively worded questions were used throughout the Child Health Program, St. John's, questionnaire to address this possible problem. However, as described by Nguyen, Attkisson and Stegner (1983) and Linn (1975), in patient satisfaction evaluations of health care, levels of satisfaction are very high regardless of the method used or the population sampled, and this is particularly true in pediatric settings
(Meterko et al., 1994).
Another procedure to correct for upper limit clustering is to regress satisfaction data on disease seriousness measures? (Strasser and Davis, 1991 and Ross et al., 1995).
1I n Strasser and Davis (1991) this is called "PatLent; Acuity·
nd in Ross et al. (1995) this is referred to as ·Sickness Impact
26 Education and socio-economic levels <Linn, 1975 and Ware et. a I .• 1976 and Hulka et a L. , 1975) of respondents may influence data and response bias. Patients with less formal education and lower socio-economic status tend to evaluate their physicians more positivelythan patients with more formal education (Linn, 1975 and DiMatteo and Hays, 1980).
Demographic information on the education or socio-economic levels and ages of parents was not collected in this study. Although information such as this may provide for greater insights into satisfaction determinants, measuring these was beyond the scope of the current study. However,
appropriateness of vocabulary for parent/guardian literacy levels was addressed by approximating the literacy level of the parental or guardian group using census data.
The instrument chosen to measure satisfaction in the evaluation used university letterhead because of the findings of Etter, Perneger and Rougemont (1996), concerning high scores for questionnaires printed on medical practice letterhead. However, i t is not known whether University letter head is better than child Health Program, HCCSJ, letterhead for response rates. This should be pretested. A study by James et al. (in press)8 found a better response
'rafile" .
'Personal Communication with Bonnie James, Co-ordinator of .he Enhanced Cancer Surveillance Project, Ontario Cancer
27 eaee among physicians wi ththeuse of canceragencyVB.
Un i v er s i ty letterhead andwi t h a Ph.D. as reques t erin s te ad of an MD.
Asummaryof the fact ors influenc i ng responsebias is pr e sentedinTable2.2 bel ow .
Ta ble2 2· FllCtorsInflyencing Re s p onse aLts
Year Researchers Fact or
1975 L i = ed u c a t ion lev e l , age .
satisfaction withlifl!!in the i rcommunity 197 5 Hulka. ec al. education leve l
1976 War e etal . educationlevel
198 0 DiMat teoand.Hay a socioeconomicst a t us 19113 Wa re. Sn yd er, Rus a ell socioeconomi c sta t us . self
Wr i g ht and Davis v•.su pe rv i sed surve y completion. income and age.
andsoci a l de sirabilit.yof ans wer
1986 Lewi set .1. adherenceinte n t:
signi fi c an tlyassociated wi t h total5at is f a c t i on 19115 Lin der-Pe l z and doctorconduct ref l e c cing
Struen i ng satis f a c t ion 1991 Strasserand Davi s patientacu i ty
Treat.me ntResearchFound ation, Toro n t.o (Ma y , 1997 ).
2.
Year Res e archers Fac to r
1975 Linn ecucecrcn level.age,
sat. i sfac t.ion wi t.h li fe 1n their cOlmIUni ey 19 94 Meter koec at. pediatric i anacquiescent
respon ses
1996 Etcer.Pernegerand. ques tionna ire sponsorship Rougemont
Theconc e rnove r bi as dueto ene mor e or less satis fied paren ts/guardians being mor e likelyt.oreturn quest ionnai r e s remains. Howe ve r, Me t e r k o et al. {1 9 9 4} did carry outa studyof th e responsecharaccerist icsof those parene s/guardiansresponding.The y te s t e d the hypothesis that the surve ywasbeing fi lled out by cbeee individuals who we r e greatly imp r e sse d or discontentedwi thca r e.
However. theydid not observe big negativeco r r elat i o n s betweensample sizeandsc ale scor es, thus disprovingtheir bias theory.
Acc ordi ng toWa re (1 9 8 1), good measuresof patient satisfact ionare characterizedby acceptability, practi c a l i t y, scoreva ri a bi l i t y, reliabil ity, valid i ty and preci sionfor hypot he sis testi ng. Itwas foundby Ware ee al. (1976), and Linder-Pe l z an d Struening, (1985) that the key determinant ofove r a l l andclinic encounter pat ien t satisfaction is doctorconduct. Severalvalidatedand
29 reliable measures of patient satisfaction were reviewed by the investigator but found to be lacking in appropriate pediatric vocabulary or were not for use in a pediatric care setting (eg: Form IV of the Patient Satisfaction
Questionnaire (PSQ) Ware, Snyder, Russell Wright and Davies, 1983) .
Surveys Chosen for tbj5 study: Two surveys were chosen by the investigator according to the criteria, outlinedbyWare (1981) and the limitations of satisfaction measurement, outlined in the paragraph above. The two chosen surveys were combined into a single instrument for this study entitled
"Parent/ Guardian Perspectives on Child Care at the Janeway". The instrument measured consumer satisfaction wi t h care received by their children in the Child Health Program, HCCSJ. Section one was made of selected sub sections from a questionnaire entitled The Parent Medical Interview Satisfaction Scale, P-MISS (Lewis et al., 1986), and section two was composed of selected sub sections from the
questionnaire entitled the PRF-23 (Davies and Ware, 1991).
The Parent Medical InterviewSatisfaction Scale, P-MISS was chosen by the investigator to measure satisfaction with communication and interpersonal scales of satisfaction with an episode of patient care. The PRF-23 sub sections form
30 assessed consumer satisfaction with access to care and technical quality (see Appendix C for questionnaire) .
The effective measurement of satisfaction entails both quantitative (use of Likert scales) and qualitative (Nelson and Larson, 1993) data. The last three questions on the questionnaire developed bythe investigator for use in the Child Health Program, Department of Pediatrics, HCCSJ, were qualitative questions based on work done byNelson and Larson (1993).
Seriousness of the patient's illness, has been found to affect satisfaction levels and satisfaction levels can be adjusted with seriousness data to give a clearer picture of real satisfaction with medical care (Strasser and Davis, 1991 and Ross, Steward and Sinacore, 1995). Strict objective illness severity data could not begathered in the pretest of the consumer satisfaction questionnaire; however, a rough subjective proxy question for parenti guardian perception of illness seriousness was added to the questionnaire (-In general, would you say your child's health is:-) to allow for the discussion of illness seriousness effects on overall satisfaction scores in this paper. However, provisions were being made to measure overall pediatric population illness seriousness levels in the evaluation (through the use of Resource Intensity Weights in Case Mix Groups). In the study by Meterko et al. (1994) satisfaction levels of
31 parents/guardians assessing care given by pediatricians when compared to adults aaee ss Inc satisfaction levels with internists' and family practitionersI care are much more favorable. This will not POse a problem in the evaluation since only pediatric facult:y will be evaluated using the questionnaire.
Validity and Reliabjlity: "I'he validity and reliability of the selected sub scales from the P-MISS and the PFR-23 questionnaires that made up the child Health Program, St.
John's, questionnaire for this study, have been established (Lewis et al., 1988, WareandDavis, 1991 and Meterko et al.,1994l. Cronbach's alpha coefficients (Cronbach, 1951) were used for measuring internal consistency of the subscales in both instruments.
The P-MISS survey's reliability was established by Lewis et al. (1988) and the Cronbach's alpha coefficients (Cronbach, 1951) were reported, by sub section, as follows: Physician communication wi th the parent (eLphae . 81) , Physician communication with child (aLphae . 93), Distress relief LaLphae .85), and Adherence intent (aLphae .86) ).
For group-level compa r-Leone , Nunally (1978) suggested that the alpha coefficient. be >.70 and Meterko et al.
suggests for inter-item COmparison an alpha coefficient
>.90. The PRF-23 designers (Meterko et al., 1994) defined
