Development and implementation of an assessment tool to evaluate technical skills in the insertion of implantable venous access devices, a Prospective Cohort Study

(1)

Article

Reference

Development and implementation of an assessment tool to evaluate technical skills in the insertion of implantable venous access devices,

a Prospective Cohort Study

ABBASSI, Ziad, et al .

Abstract

Based on the Competency Assessment Tool, herein we developed an assessment instrument suitable to evaluate the implantation of central intravenous devices.

ABBASSI, Ziad, et al . Development and implementation of an assessment tool to evaluate technical skills in the insertion of implantable venous access devices, a Prospective Cohort Study. Journal of Visceral Surgery , 2020

DOI : 10.1016/j.jviscsurg.2020.10.016 PMID : 33184018

Available at:

http://archive-ouverte.unige.ch/unige:149925

Disclaimer: layout of this document may differ from the published version.

1 / 1

(2)

Pleasecitethisarticleinpressas:AbbassiZ,etal.Developmentandimplementationofanassessmenttooltoevaluate technicalskills inthe insertionof implantablevenous accessdevices,aProspective Cohort Study.JournalofVisceral Surgery(2020),https://doi.org/10.1016/j.jviscsurg.2020.10.016

ARTICLE IN PRESS

+Model

JVS-1108; No.ofPages7

JournalofVisceralSurgery(2020)xxx,xxx—xxx

Availableonlineat

ScienceDirect

www.sciencedirect.com

ORIGINAL ARTICLE

Development and implementation of an assessment tool to evaluate technical skills in the insertion of implantable venous

access devices, a Prospective Cohort Study

Z. Abbassi

^∗

, B. Nebbot , A. Peloso , J. Meyer , T. Thomopoulos , M. Jung , W.L. Staszewicz , S.P. Naiken , N.C. Buchs , C. Toso , F. Ris

VisceralSurgery,DepartmentsofSurgery,UniversityHospitalofGeneva,rue Gabrielle-Perret-Gentil4,1211Geneva,Switzerland

KEYWORDS Surgicaltraining;

Competency assessmenttool;

Skills;

Implantablevenous accessdevices

Summary

Objective:Basedon theCompetency Assessment Tool, herein we developedan assessment instrumentsuitabletoevaluatetheimplantationofcentralintravenousdevices.

Background:Surgicalassessmentisbasedmainlyonthesubjectiveimpressionsoftheteacher.

Basedonthe‘‘CompetencyAssessmentTool’’(CAT)developedfortheevaluationoftechnical surgicalskillsinminimallyinvasivecolorectalresection,wedesignedanassessmenttoolsuit- abletoevaluatetheimplantationofcentralvenousaccessdevicesperformedbyjuniorsurgical trainees.

Methods:Fourmajorassessmentsduringthedifferentstepsoftheinterventionwereusedinthis evaluation.Eachofthesetaskswasdividedintofoursub-domainsaccordingtosurgicalskill.In additiontotheCATscore,theapprentices’skillswereevaluatedusingavisualassessmentthat wasquantiﬁedusingananaloguescale(valuefrom1to10).Thecandidateswereclassiﬁedinto juniorandseniortraineesdependingonthenumberofprocedurestheyhadalreadyperformed andontheirsurgicalexperience.

Results:71procedureswereevaluatedduringthestudyperiod.Sevenseniortraineesconducted 43 proceduresandfivejunior traineesperformed 28interventions.The senior trainees had significantlyhigherCATscoresthanjuniorcandidates,andthescoresfluctuatedaccordingto surgicalexperience,usuallyreachingtheirpeakafter10procedures.

Conclusions:TheCATmodeliswellsuitedfortheassessmentofsurgicaltraineesduringcentral venousaccessdeviceimplantation.Itenablesacloseassessmentofthelearningprocessand thetechnicalskillsoftrainees,whichhelpsthemimprovinginasafe,standardizedmanner.

DOIoforiginalarticle:https://doi.org/10.1016/j.jchirv.2020.09.005.

∗Correspondingauthor.

E-mailaddress:[email protected](Z.Abbassi).

https://doi.org/10.1016/j.jviscsurg.2020.10.016

(3)

Pleasecitethisarticleinpressas:AbbassiZ,etal.Developmentandimplementationofanassessmenttooltoevaluate technicalskills inthe insertionofimplantable venousaccessdevices,a Prospective CohortStudy.Journalof Visceral Surgery(2020),https://doi.org/10.1016/j.jviscsurg.2020.10.016

ARTICLE IN PRESS

+Model

2 Z.Abbassietal.

Introduction

Surgical traininghastwo mainaspects. First,the acquisi- tionof the theoretical concepts related tothe pathology and its management; second, the acquisition of surgical techniques, operative strategy and manual skills.

Whereasassessmentofcandidatesurgeonscanbesuccess- fullyachievedwithobjectivityconcerningtheirtheoretical knowledge,evaluationoftheirsurgicalskillsandtechnical maturityaresubjectiveand,therefore,remainamatterof debate [1]. Over the past 20 years, several authors have tried to develop tools to evaluate surgical skills includ- ing scale evaluations and scores. Some are speciﬁc to a procedureand structuredasachecklist ofsteps [2]. Oth- ersusemoregeneralmeansofevaluatingthesurgicalskill andknowledgerequiredfortheproceduraltechnique[1,3].

The OSATS (Objective Structured Assessment of Technical Skill) is among the most cited evaluation tools found in the literature. It is a good tool to evaluate specialised candidatesbasedontheirhands-on experience.Ithasthe advantage of usingdetailed criteriatodescribe technical skills[1]butdoesnottakeintoaccountthedifferentsteps andstageswithinasurgicalprocedure[4].Duetoreduced work hoursand increased demand for objectiveevidence ofqualiﬁcation,theneedforstructuredlearningstrategies andskillassessmentshasbecome essentialin thesurgical ﬁeld [5]. A low number of laparoscopic procedures per- formed for elective colorectal surgery in Britain has led to a national training program in 2008. They developed andvalidated asystematic evaluation measurecalled the

‘‘CompetencyAssessmentTool’’(CAT).Itusesachecklistof the tasksrequired subdivided intoa four-step procedure, thereby enabling the evaluation of candidates’ technical skillsincludingerrors.Thisprogramincreasedtherateofthe colorectallaparoscopicprocedureby50%in4yearsandthus decreasedthelearningcurveforthisprocedure[6].During surgicaltraining,implantationofthecentralvenousaccess device is carriedout mainly by traineesunder the supervision of experienced surgeons. This procedure is highly standardisedandallowsfor thedevelopmentofan evalu- ationtoolwithseparate steps.It combinesstandard open surgery, dissection of a vessel (cephalic vein), catheteri- sation techniques and x-ray interpretation at the end of thesurgery[7].Moreover,itisaverycommonprocedure:

400centralvenousaccessdevicesareimplantedinourunit annually.Basedonthemodelalreadyusedforlaparoscopic colorectalsurgery, ouraimistodeterminetheefﬁcacyof theassessmenttoolwedevelopedfortheinsertionofcen- tralvenousaccessdevices.

Materials and methods Evaluation method

We developed our adapted version of the assessment table used by Miskovic. The CAT designed for laparo- scopiccolectomystudiessubdividessurgeryintofourstages:

exposure, control of the vessels, mobilisation and resec- tion/anastomosis.Eachstepwasevaluatedtoassesstheuse ofinstruments,manipulationoftissues,complicationsand the result.These sub-skillswere rated usinga four-point scale:0ifthestepwasnotperformedbythecandidate,1 for incompetent, 2for novice, 3for competent and4 for expert[6].

Avideodetailingthestandardprocedureforimplantable venousaccessdeviceinsertionthroughthecephalicveinwas availablefromourserviceandshowntoallnovicetrainees.

Thesubdivisionintofourstageswasapprovedbythesuper- vising surgeon.Similarly, the second principle of Miskovic wastodetailtheassessmentofeachstepinfourskillsand was applied to the new tool: use of instruments, tissue manipulation, damageandquality ofthe ﬁnalresult.The expectedresultwastoobtainaCATsuccessfullyadaptedto ourprocedureinvolvingcriteriaforeachspeciﬁcstageand evaluationoftechnicalskills.Avisualanaloguescale(VAS) was included in the samedocument toevaluate the can- didate’s surgical performance without direct supervision.

A rating scale with seven possible answers ranging from

‘‘clearlyyes’’to‘‘clearlyno’’wasused.ThemodiﬁedCAT wasdividedintofoursteps(Table1):

• installationanddissectionuntillocationof thecephalic vein;

• preparationofthecephalicvein;

• introductionofthecatheter;

• creation of a subcutaneous space, placement of the deviceandclosure.

AVASrangingfrom1to10allowedustoeffectivelyeval- uate the autonomy of the trainee. The autonomy of the traineeswasassessedsimultaneouslyusingaVAS.

Each step was sub-divided into four skills assessment stepsthatwereevaluatedbyascorerangingfrom1to4(1 forincompetent,uncontrolledordangerous;2inadequate, inefﬁcientorvague;3safeorgood;4expertorperfect).We reservedtheevaluationforonlythosesurgeonswhocarried outthewholeprocedure.

Data collection

We prospectively analysed 71 evaluations. The recorded data included the CAT, EVA, age and sex of the patient, identity ofthecandidateandtheassessor,positionofthe catheter, surgical indication, ASA score of the patient, patient’sbodymassindex,difficultyofthecase(classifiedas standard ordifficult),numberofproceduresperformed by theapplicantbeforetheevaluation(>10or<10),junioror seniorstatusofthecandidate(linkedtothetypeofsupervi- sor,juniororsenior)andthenumberofyearspriortosurgical training.

Interventions requiring help from theassessor, suchas help for venepuncture,catheterizationof thevein or any otherdemonstrativehelponthepartoftheassessor,were excluded.

Statistical analysis

Thestatisticaltestswereselectedandperformedwithhelp from theMethodological SupportUnit of the GenevaUni- versity Hospitals using the CRAN R v. 3.0.3 software (R FoundationforStatisticalComputing, Vienna,Austria.URL http://www.R-project.org/).

TheMann-WhitneyUtestwasusedtoestimatethediffer- enceinthenumberofproceduresperformedbyjuniorand senior candidates aswell theirrespectivesurgical experience.TheReceiverOperatingCharacteristic(ROC)method wasused toexaminethe CAT scorethreshold requiredto identifyautonomouscandidates.

Given the unequal number of observations per candidates, mixedstatistical modelswere used tostudy other continuous variables, and generalised mixed models for

(4)

Pleasecitethisarticleinpressas:AbbassiZ,etal.Developmentandimplementationofanassessmenttooltoevaluatetechnicalskillsintheinsertionofimplantablevenousaccessdevices,aProspectiveCohortStudy.JournalofVisceralSurgery(2020),https://doi.org/10.1016/j.jviscsurg.2020.10.016

AR TICLE IN PRESS

+ModelJVS-1108;No.ofPages7

Developmentandimplementationofanassessmenttooltoevaluatetechnicalskills3

Table1 CaracteristicsofthemodiﬁedCAT.

Taxx Abilitytouse

surgicaltools

Tissuehandling Damages Finalqualityresult

Patientinstallation, surgicalincision, useofscalpel

Subcutaneoustissue dissection

Damagesdueto Cephalicvein

identiﬁcation Patientinstallation,

surgicalexposure, veindetection

1.Unqualiﬁed Wrongposition Wrongincision Suboptimal exposure

1.Unqualiﬁed Stiffand uncontrolled movements

1.Unqualiﬁed Wrongsurgical incision(or requiring correction) Improperdissection

1.No Cephalicveinnot

identiﬁed

2.Unsatisfactory Multipleattempts Suboptimal ergonomic

2.Unsatisfactory Controlledbut hesitant(and sometimes) ineffective movements

2.Unsatisfactory Bleedingduring dissection, muscularinjuries

2.Partial Identiﬁcation requiringhelp

3.Safe Safesurgical technique Goodergonomic

3.Safe Controlledand

effective movements

3.Safe Effectivedissection 3.Yes Successful

identiﬁcationafter multipleattempts butwithouthelp 4.Expert Optimalinstallation

andergonomic

4.Expert Abilitytopredict Perfectmovements

4.Expert Effectivedissection withoptimaltissue preservation

4.Expert Perfectly anatomical dissection

X.U/E Invaluable X.U/E Invaluable X.U/E Invaluable X.U/E Invaluable

Taxxx Abilitytouse

surgicaltools

Useofmixterright angleforceps

Veinhandling, denudationand distaltying

preparationfor catheterism Surgicalvein

approach

1.Unqualiﬁed Wrongusewith uncontrolled movements

1.Uncontrolled Poorexposure High-riskhandling

1.Uncontrolled Majorvenousinjury 1.Uncontrolled Impossible

2.Unsatisfactory Controlledbut hesitantand multiple movements

2.Ineffective Hesitanthandling 2.Ineffective Bleedingdueto minorvenousinjury

2.Imprecise Unsatisfactory preparation

3.Safe Gooduseofsurgical tools

3.Good Effectivedissection 3.Good Novascularinjuries 3.Good Satisfactory preparation 4.Expert Verygooduseof

surgicaltools

4.Expert Effectivedissection Low-riskhandling

4.Expert Perfectdissection 4.Expert Verygood preparation

(5)

Pleasecitethisarticleinpressas:AbbassiZ,etal.Developmentandimplementationofanassessmenttooltoevaluatetechnicalskillsintheinsertionofimplantablevenousaccessdevices,aProspectiveCohortStudy.JournalofVisceralSurgery(2020),https://doi.org/10.1016/j.jviscsurg.2020.10.016

AR TICLE IN PRESS

+ModelJVS-1108;No.ofPages7

4Z.Abbassietal.

Table1(Continued)

Taxx Abilitytouse

surgicaltools

Patientinstallation, surgicalincision, useofscalpel

Subcutaneoustissue dissection

identiﬁcation

Taxxx Abilitytouse

surgicaltools

Venotomy Insertionofvenous

accessdevice

catheterismand radiologicimages Insertionofvenous

accessdevice

1.Unqualiﬁed Completevenous section

1.Uncontrolled Poorexposure High-riskhandling

1.Uncontrolled Majorvenousinjury orsterility compromised

1.No Impossible

2.Unsatisfactory Multipletrieswith hesitant

movements

2.Ineffective Multipletrieswith hesitant

movements

2.Ineffective Vascularinjury (w/oconsequences)

2.Partial Procedure correctedbythe teacher 3.Safe Gooduseofsurgical

tools

3.Good Effective

introduction, low-riskproﬁle

3.Good Novascularinjuries 3.Good Goodplacement

4.Expert Verygooduseof surgicaltoolswith highsecurity

4.Expert Verygood

introduction

4.Expert Perfect

introduction

4.Expert Perfectautonomous Resultw/oanyhelp

Taxxx Abilitytouse

surgicaltools

Electrocautery, needledriver

Subcutaneous pocketpreparation, connectionandport placement

Damagesdueto Functionalport

placement

Subcutaneous pocketpreparation andportplacement

1.Unqualiﬁed Uncontrolleduse withhigh-risk proﬁle

1.Uncontrolled Uncontrolled,and hesitant

movements

1.Uncontrolled Tissueormaterial injuries,

compromised sterility

1.No Wrongport

placement

2.Unsatisfactory Multipletrieswith hesitant

movements

2.Ineffective Multipletrieswith hesitant

movements

2.Ineffective Portplacement ineffective

2.Partial Helprequired

3.Safe Gooduseofsurgical tools

3.Good Controlledand

effective movements

3.Good Noinjuries 3.Good Goodplacement

4.Expert Expertuseof surgicaltools

4.Expert Perfectprocedure 4.Expert Correctdissection andportplacement

4.Expert Perfectautonomous Resultw/oanyhelp

(6)

ARTICLE IN PRESS

+Model

Developmentandimplementationofanassessmenttooltoevaluatetechnicalskills 5

Figure1. ThemeanvalueoftheCAT,±SDerrorbarsinfunction ofdegreeofexperience(0=juniorstatusor1=seniorstatus).

Figure2. CATscoresinfunctionofsurgicalexperiencetrainees.

categorical variables. Only univariate analyses were per- formed.

Results

Theprocedureswereperformedby12differentcandidates (Table2).Sevenseniortraineescompleted43procedures, and5juniortraineescompleted28procedures.Therewas no signiﬁcant difference between the average number of proceduresperformedbyseniorandjuniortrainees(Mann- WhitneyUtest,P=0.62).Seniortraineeshadmoresurgical experiencethanjunior trainees(P<0.005).Patient demo- graphicsandcharacteristicsarepresentedinTable3,there were no signiﬁcant differences between the two groups.

The meanCAT score wassigniﬁcantlyhigherin thesenior groupthaninthejuniorgroup(Fig.1),withameanof3.41 (2.9—3.93) versus 1.43 (0.78—2.07), respectively (Fig. 2).

Moreover,theyearsoftrainingsignificantlyinfluencedthe CATscore(P=0.007),respectively.Seniorcandidateswere significantly more autonomous comparedto junior candidates (P=0.003). The CAT mean scores were higher for the independent trainees than the others trainees: 3.5

Figure3. CATscores’inﬂuencebynumberofprocedures.

Figure4. ReceiverOperatingCharacteristiccurveofthemodiﬁed competencyassessmenttool.

(2.96—4.04)versus2.05(1.57—2.53),respectively(P<0.01).

AthresholdCATscorewasdeterminedusingtheROCmethod to ascertain when a trainee needs additional supervision (Fig.3). When the score wasabove 3.3, the trainee was considered to be autonomous with a sensitivity of 98.5%

anda speciﬁcityof 90.1%.CAT scores followed alearning curvesmoothlyandremainedinﬂuencedbythecandidates’

experience(Fig.4).

Discussion

This study assessed the CAT ability to evaluate surgery trainees. The CAT allows for the objective evaluation of trainee’sperformanceusingastandardisedprocedure,with aclearcutoffvaluefortheaveragenumberofprocedures requiredtoachieveexpertiseinthetechnique.

(7)

Pleasecitethisarticleinpressas:AbbassiZ,etal.Developmentandimplementationofanassessmenttooltoevaluate technicalskills inthe insertionofimplantable venousaccessdevices,a Prospective CohortStudy.Journalof Visceral Surgery(2020),https://doi.org/10.1016/j.jviscsurg.2020.10.016

ARTICLE IN PRESS

+Model

6 Z.Abbassietal.

Table2 Demographyofcandidates.

Candidateidentity A B C D E F G H I J K L

Experience(years) 3 0 2 3 1 4 0 3 5 5 0 0

Junior(J)—Senior(S) S J S S J S J S S S J J

Procedures(n) 6 2 3 11 5 4 1 3 10 6 6 14

Table3 Patientdemographicsandcharacteristicsofsurgicalproceduresandtheirdistribution.

Mean(CI95%) Juniors Seniors P

Age(years) 55.9(50.4—61.4) 57.8(53.3—62.2) 0.62

Sexratio(Male/Female) 0.7(0.4—0.9) 0.5(0.3—0.7) 0.28

ASAscore 2.3(2—2.6) 2.4(2.2—2.7) 0.42

BMI(kg/m²) 25(22.3—27.1) 25.1(24.1—27.4) 0.6

Difﬁcultratioofsurgery 0.9(0.7—0.9) 0.8(0.6—0.9) 0.36

TheCATmodelwassuccessfullyadaptedtotheinsertion ofimplantable venous accessdevices,which supportsthe applicationofthismethodtotheevaluationofotherproce- dures.Analysisoftheresultsshowedthatthedemographics ofthe operated patientsand thenumberof interventions evaluatedinthecomparativeapplicantgroupsdidnotdif- fersigniﬁcantly.Moreover,themeanCATscoresobtainedby experiencedcandidateswerehigherthanthoseobtainedby juniorcandidates, which wasasexpected,reassuringand servedtofurthersupportingtheuseoftheCATasaneffec- tiveassessmenttool.Manyassessmenttoolscurrentlyexist.

Some models areapplicable tocadavers or animals, oth- ersonsimulatorsandfinallysomearesuitedforuseinthe operating room. Those tools were initially developed for thelaboratory studiesand, assuch,were notsuitablefor clinical practice. They were mostly adapted for a single, specificproceduretoscorespecializedskillsandtechnical abilities[4,8,9].Theywere appliedtosimulators bymea- suringgenericskillsinlaparoscopicandroboticsurgeryand displayed shorter learning curves but are insufficient for opensurgery[4,6,9,10].Meanwhile,someauthorsproposed arating scale thatconsiders only thestages of an opera- tion,asproposedbyEubanksOBJECTIVESBOSATS[2,10].To date,therearemany modelsthatmeet thegeneralstan- dardsof competenceand surgical dexterity of the OSATS withtheabilitytobeadaptedtomultipletypesofinterven- tions[1,4,11,12].

The CAT has two major advantages compared to the aforementionedtools. Itrepresentsacombination ofspe- ciﬁccriteriatomeasuretechnicalcompetenceandsurgical precision. This tool is well suitedfor adaptation toother surgical interventions [6]. This type of tool is useful to objectivelyassessthecandidate’sbaselevel;theirprogres- sion and improvement can be evaluated throughboth an analysisof their strengthsandweaknesses duringa surgical procedure [6,8]. Several studies are underway in our hospitaltovalidatesimilarmodelsfor othertypesofsurg- eries.

The main limitation of this study is the smallnumber ofcandidatesandthelack ofvalidationusingotherbasics procedures. Future studies will include a larger sample populationandadditionalsurgicalprocedures(e.g.,chole- cystectomy,appendectomy).

Conclusions

ApplyingtheCATspeciﬁcallytotheinsertionofimplantable venous accessdevicesproved thatitwasefﬁcientfor the assessmentofthetechnicalskillsandprogressofyoungsur- gicaltrainees.

We believe this tool is useful in tailoring the surgical trainingaccordingtotheprogressofeachcandidate.

Acknowledgements

TheauthorsthankMr.LorenzoOrciforhiscriticalreviewof themanuscript.

Disclosure of interest

Theauthorsdeclarethattheyhavenocompetinginterest.

References

[1]MartinJA, RegehrG,ReznickR, etal. Objectivestructured assessmentoftechnicalskills(OSATS)forsurgicalresidents.Br JSurg1997;84:273—8.

[2]EubanksTR, ClementsRH, PohlD,et al.AnObjectivescor- ingsystemfor laparoscopiccholecystectomy.JAmCollSurg 1999;189:566—74.

[3]Birkmeyer JD, FinksJD, O’Reilly A, etal. Surgical skilland complication rates after bariatric surgery. N Engl J Med 2013;369:1434—42.

[4]Mariette C. Apprentissage de la chirurgie laparoscopique:

quelles méthodes pour le chirurgien en formation? J Chir 2006;143:221—5.

[5]VassiliouMC,FeldmanLS,AndrewCG,etal.Aglobalassesment toolforevaluationofintraoperativelaparoscopicskills.AmJ Surg2005;190:107—13.

[6]Miskovic D, Ni M, Wyles SM, et al. Is competency at the specialistlevelachievable?Astudyofthenationaltrainingpro- grammeinlaparoscopiccolorectalsurgeryinEngland.AnnSurg 2013;257:476—82.

[7]Orci LA, Meier RP, Morel P, et al. Systematic review and meta-analysisofpercutaneoussubclavianvein puncturever-

(8)

ARTICLE IN PRESS

+Model

Developmentandimplementationofanassessmenttooltoevaluatetechnicalskills 7

sus surgical venous cutdown for the insertion of a totally implantablevenousaccessdevice.BrJSurg2014;101:8—16.

[8]DaRosaDA,ZwischenbergerJB,MeyersonSL,etal.Atheory- based model for teaching and assessing residents in the operatingroom.JSurg2012;70:24—30.

[9]MiskovicD,WylesSM,NiM,etal.Systematicreviewonmen- toringandsimulationinlaparoscopiccolorectalsurgery.Ann Surg2010;252:943—51.

[10] CalatayudD,AroraS,AggarwalR,etal.Warm-upinavirtual realityenvironment improves performance in the operating room.AnnSurg2010;251:1181—5.

[11]Zevin B, Bonrath EM, AggarwalR, et al. Development Fea- sibility, Validity, and Reliability of a scale for objective assessmentofoperativeperformance inlaparoscopicgastric bypasssurgery.JAmCollSurg2013;216:955—65.

[12]AggarwalR,MoorthyK,DarziA.Laparoscopicskillstrainingand assessment.BrJSurg2004;91:1549—58.