An
Evaluation of the National Coaching Certification Program Level Two Techn ical Course for the Sport of Judo
by Thomas L.Gallant, B.A.&B.Ed.
Subminedinpanialfulfillmentofthereqvimnc:nts (orthe MasterorEdUQrion (LeamiDgR.esoun;cs:ColnmllD.ieatiorcs4.TedmoJogy)
inmeScboolof Education. MemorialUniversityofNewfoundland June.1998
.+.
Nationallibtaryof"""'"
Acquisitions and BibliogfaPhic5ervices 39SW8IlingIcnStrHI OltawaON K1A0N4
eo.-
Biblioth8quenationale duCanada Acquisitionset services bibliographiques 395, rwW..1irlgIon
= C N K 1A0N4
The author basgranteda non- exclusive licence allowing the National Library of Canada to reproduce,loan, distnbute or sell copies of this thesisinmicroform.
paper orelectronic formats.
The author retains ownership of the copyright in this thesis.Neither the thesis nor substantial extracts fromit may be printed or otherwise reproduced without the author's pennission.
L'auteur aaccorde une licencenon exclusivepermettant
a
la Bibliotheque nationale du Canada de reproduire, preter,distribuer au vendre des copies de cettethese
sous la forme de microfiche/film., de reproduction sur papier au sur format electrcnique.L'auteur conservelapropriete du droit d'auteur qui protege cette these. Ni la thesenides extraits substantiels de celle..ci ne daiventetreimprimes au autrement reproduits sans son autorisation.
0-612-36124-1
Canada
Abstract
Thepurposeofthisevaluatio n wastoassesstheNationalCoachingCerti fica tio nProgram (NCCP)technical level twocompooem: for the sportofjudo. Thereviewoftbe literatureprovided backgroundoftheNCCP.thesportof judoandtheevaluati on methodolo giesthatwereconsidered and/orusedin theevaluation. A modifiedversion of Stake's(1995)responsiveevaluation model wasusedas aguidein the project.The modelwasusedbecauseof its flexibili ty.use of audience concerns. aswellasits standardsdevelopmcnLQuantitativeandqualitativedatawere gathered from all provincesandterritoriesinCanadaoveraperiodofa year andabaIf. Thestakeholders wereiden tifiedanddivided intotwogroups.primary(expert)andsecondary(Ievel twe]
coaches. Theevaluation itse lfwasseparated intotwo phases.Inphaseone theprimary stakehold ers setthestandards.Inphasetwo the primaryandsecondary stakeholders evaluated theexistingcourse by applying thestandardsfrom phase one.The consensus amongall stakeholderswasthatthecoursewasmeeting the standards;however.theydid offer recommendati onsfor improvement of lbeNational Coaching CertificationProgram..
Acknow ledgements
I wish to express my appreciation to those people who supported, encouraged and assisted mein completing this study.
lowemygreatest debt of appreciation toDr.Mary Kennedy,my supervisor, who spent tireless hours guiding my every step,evenwhileinretirement and livinginBritish Columbia. [ feelprivilegedtohave been her student.
I wish also to thankDr.Basil Kavanaugh, theother member of my thesis committee,who spent many hours reading the draftsandoffering input. His assistance was most valuable. I would also liketothank.Mr.GerryWhite who spent a great deal of timehelping me with the statistical analysis.Thanksalso gotoMr.Brian Kerr for proofreading and advising on thefinaldocument.
Inadditiontothe above,I would like to thank JudoCana da for its cooperation withthe study and theCoaching Association'sNational Coaching Committeewho provided necessary background materials to aid in thecompletion of thedocument.
Special thanks have to go to the Master Course Conductors,Course Conductors and coaches who participated in tile surveys and offered comments and suggestions to helpwith thestudy.Also[0Mr.Raymond Damblant for offering a special prize[0help withexpedient data collection.
[ would likealso to thank my familyandfriendsfor their supportand encouragementinthewritingof this study.
This thesis is dedicatedto the memory of my parents:
Ronald T.Gallant (1907·1994) AlfredaGallant
(1926-1986)
iii
Abstract . Acknowledgments...
Listof Tables listof Figures
TABLEOF CONTENTS
...i ..ii .xi ...•...xii
iv
CHAPTER ONE;BACKGROUND OF TIIE SlUDY Introduction•.
The Program•
Significance afthe Study Limitationsafthe Study Definition ofTenns.
Organization ofthe Study .
...•••1
. 1
• 1
...3
...__ 6
. 6
. 7
CHAPTER TWO:REVIEWOF THE LITERATURE Introduction .
History ofJudo Introduction
Bujutsu .
Feudal Japan..
FoundationsofKodokanJudo..
The Founder of Judo .
...9
. . .
... ....9
.... .
. 10
...__11 . 13
. 14
Judo as Physical Education....
Judo Outside Japan . JudoinCanada...
Program Evaluation...
Historical Perspective. The Age of Reform...
TheAgeof Efficiencyand Testing 1900·1930...
The TylerianAge 1930 -1945...
The Age of Innocence1946 - 1957 The Age of Expansion1958 -19n The Age of Professionalism1973 to the Present VaryingPhilosophical Stances
Objectives-OrientedApproach Overview... . Proponents Characteristics .
Benefits .
Limitations . Consumer-Oriented Approach
Overview Proponents..
Model...
Characteristics.•...•••..•....••
...15
. 16
. 19
. 20
20 . 21 ... 21
. 22
...22 ... ...23 ...25 ....26 ...27 ...27
. 27
. 28
. 28
. 28
..29 .29
29
. 30
. 30
Benefits . Limitations....
Management-Oriented Approach..
Overview ....
Proponents .
Model .
Characteristics Benefits ..
Limitations Expertise-OrientedApproach
Overview .
Proponents..
Model...
Characteristics...
Benefits . Limitations.
Adversary-OrientedApproach Overvi ew Proponents .
Model .
Characteristics Benefits Limitations .
. 31
. 31
. 31
..31 ...32
. 32
... 33
. 34
...34 ... 35
. 35
...35
. 35
...36
. 36
. 36
. 37
...37 ...37 .37
. 38
...38 ...39
vi
CHAPTER 3:EVALUATIONMETHODOLOGYANDDESIGN....
EvaluationDesign EvaluationModel Evaluation Procedures
PhaseOne .
Phase Two .
EvaluationMethodology . StakeholderIdentif ication ConcernsandIssues . CollectionandTreatment of Data
CHAPTER FOUR:EVALUATIONANALYSIS. Introduction
Audiences
... 51 ....51
...52 ... ... 53
54
. 54
... 54
....55
. 58
...60
. 62
...62 ... ...62
viii
PhaseOneAnalysis 65
StakeholderDetermination 65
EducationalBackground . 65
AgeGrouping. . 66
Language . 66
Geographic Grouping 67
Categories
Standards•...•....••...••...••...•.•...••.•...
Summary
...•••....•... 67 ...69
. 74
Physical and MentalTrainin g . PhaseTwo Analysis .
Scoring .
QualityCoaches . Instruction . Discussion . Curriculum .
Discussion Learning...
Discussion Historical DevelopmentofJudo
Discussion ClubSet-upand Administration.
Discussion .
Teaching..
Discussion Skill AnalysisofAthle tes
Discus sion.
Discussion InjuryPrevention .
Discussion. Theory ofludo Techniques.
Discussion .
. 75
. 75
... 76 ...76
. 77
. 78
...79 ...80 ... ... 82
..83 ..84 ...85
...87 ...88 ...89
... ....90 ..92 ...93 ..94 ... ...95
. 96
. 97
. 99
ix
Overview ... 99
CHAPTER 5:SUMMARY. CONCLUSIONS ANDRECO~ATIONS 100
Summary ....• . 100
TheStake Responsive Model . 101
Advantages of TheResponsive Evaluation Modelfor NCCP Evaluation.... . 102
Limitations ofTbeResponsiveEvaluation Model 103
Conclusions.. . 103
Recommendations forFurtherStudy 104
REFERENCES APPENDICES
A.Issues&Concerns Survey...
B.Standards Evaluation Survey.
C.Correspondence...•
D. ProgramOutline
.107 ... 111 ... ... 112
. 122
. 130
•...140
xi
LIST OF TABLES
...91 ...94 ...96
. 98
Table
1.1.NCCPCourseCertification Requirements 2.1 Martial ArtsCategories
2.2. TheFourEvaluationTypesof the ClPP Model ..
4.1Demographics of PhaseOne 4.2 Evaluation Scale...
4.3 Recruitment of Coac hesandInstructors...
4.4 Coac hesNeeds . 4.5 KnowledgeTransfer..
4.6 HistoricalDevelopment..
4.7 Club OrganizationandAdministration 4.8 Teaching Methodologies 4.9 SkillAnalys is 4.10Physicaland MentalTraining 4.11Injury Prevention..•
4.12JudoTheory .
Page
... 3 ...13
. 33
...67 ..75 ...77
. 79
. 81
. 84
...•.86 ... ...89
Figure
LIST OF FIGURES
Page 2.1 ResponsiveEvaluation Cleek, modified from Worthen&Saunders. 1987 42 2.2 NeepEvaluationModel,Neep (1996),from Six-StageEvaluation Model 47
3.1 Ne e pjudo evaluation . 53
4.1 Qualitycoaches who enter the
Neep
LevelTwo forthe sport ofjudo 71 4.2The course materialsatisfies the needs of coaches 79 4.3Theknowledgetransferis adequate to serve the needs oftheLevelTwo coach 82 4.4The adequacyof the historical developmentof judoinCanada 84 4.5Theorganization and administration needs of coaches arebeingmet.. ..87 4.6The teaching preparation needs of coaches... . 894.7 Traininginskill analysis... . 92
4.8 Physicaland mental trainingskills acquisition 94
4.9 Injury prevention training plan ...96
4.10Theory ofjudo as representedbyrandori nokata acquisition 98
Introduction 1
CHAPTER I Backgroundof the Stu dy
Introduction
The basis of'this study was to conduct an evaluation ofone component of the coaching education program for the spon ofjudo. The program is jointly sponsored by the Coaching Association of Canada,underit's National Coaching Certification Program (NCCP) and the coaching committee ofJud o Canada. The purposes of thestudy were to ascertain the usefulness of the Level Two Technical coaching course for judo,andto build a framework on which to evaluate other courses in this and othersports.Withthis inmind,chapter one discusses the concept of coaching educationin Canada fromits broadhistoric perspectiveto the founding and development of theNCCP.andho w coaching sportinCanada canbe reflected in the more modem form of theart and sport ofjudo.
TheProgram
The National Coaching CertificationProgram (NCCP)wasestablishedin19n,and since that time it has trained over 500.000 coaches from across Canada. The purpose of the program is totrain Canadian coachesinailsportstomeet the increasedneeds ofCanadiao athletes.One of the goals of the NCCP is to ensure the competence of coachesby requiring theattainment of certain predetermined progressive standards foreach levelof the training programs.
Injudo there are1434 coaches involvedinthe NCCPprogram.Fifty-eight percent[58%]
of these are certified atlevel1,twenty-sixpercent[26%] atleveltwo.andonly five percent [5%]
Introduction 2 at level three (NCCP.February1996).The figureslistedabove reflect alltenprovinces and one territory. the Yukon.AsweU, therewerethreecoaches participatinginthe program from the United States
The program consists of five levels. each designed to offer progressive components of expertise eventually leading to certification. Levels one throughthreeare designed for coaches whotrainrecreational and developing athletes, or athletes who compete up to and including provincialteams.Levels' four and five are designed for the training of eliteathletesof national andinternational caliber.
Each levelhasthreecomponents: theory,technical and practicaLThe theory courses arc offeredbyNCCP representativesinall provinces and territories and are generictoallsports. The technical components are developed and offered by the National Sports Governing Bodies (NSGB)ineach of the individualsports. The practical components entail coaches puttinginto practice what they have learned in the theory andtechnicalcomponents by coaching for a predetermined number of hours. When the time requirements arc fulfilled.certification is granted after an evaluation process by the NSGB. Tobefully certifiedata given level, allthree components - theory. technical and practical mustbecompleted.Table l.Lgiveean example of a coach whohascompletedall threecomponents of leveltwo.but only theory and technical at level three. This coach is certified at level two.
Introduction 3
Table1.1NCCP CourseCertific.tioDReqairem eDts
Level I 2 3 4 5
NCCPCertificationRequ irem ents Theo ry
., .,
Tft:hn ical.., .,
Practical., .,
., .,
Certified C C
Sinceitsinceptionin1972 and with one revisionin1989.theNCCPhasbad hundreds of thousands ofdollars allocatedinresearch anddevelopmen t(Robinson, 1993).Thefundswere allocatedto the differentSportGoverningBodiestodevelop technical courses that were sport specific.Inthe spring of 1996theNCCPdid an overallevaluation oftbeprograminorder to effectpositive changesinall programs.The Coaching Association ofCanada (CAe)released only thepre liminary report.InMay1997.the evaluationcommittee madeitsmotions for change to the NationalCoachingCertificationCouncil (NCCC). To ensure that changes willbepositive.
the evaluation mustbedone at all five levelsof the NCCP program andinallthe participating sports.Inaddition, each component at each level mustbeeva luated,e.g.Level TwoTheory, LevelTwoTechnical and LevelTwo Practical.
Significanc eof tbeStu dy
There has been considerableeffort made over thepast decadetoelevate thelevel of coaching skillsacrossCana da. Theintroduction of the NCCP was done to makeCanada more competitiveat Olympic andworldchampionshipevents.To determinethe effectivenessofthe overallprogram . allexi stin g coursesshouldbeevaluated summatively.Anevaluation ofthe
Introduction 4
NCCP Level Two for judo was necessarybecauseit would provide"every one associated with the operation of the program. theabilityto measureresultsandto determine if itis theprogram that produced the results(and to wbat extent) or some otherfactor"(Robinson, 1993). Thisstudyis very important tothesport of judo,asthere bas neverbeenany formal evaluation performed on any of its componentcourses.
Along with baving the potential to improve judo technical courses, this summative evaluationcomplements evaluations previouslycompleted on theory courses and technical courses from other sports. This evaluation also contributes to theexistin g pool of evaluative knowledge gained from applicationsinCanadaandother countries.
Before the 1996 evaluation, the NCCP bad not been formally evaluated in almost 20 years. Thisissurprising, consideringthatthere are over500,000registered coaches in Canada who have taken courses in the NCCP.There are some possible reasons why the NCCPhas not hadany formal evaluation. Robinson(1993 ) recounted some of the ideas from a meeting between senior management of the Coaching Association of Canada(CAe)and members from the National Coaching Certification Council (NCCC) Evaluation Sub-Committee:
1. It may unlock 'Pandora's Box.'Stakeholders were a little unsure of what an evaluation study wouldfind.In other words, evaluation may reveal serious problems with the NCCP's approach,that, until now, havebeenhidden.For example, upon completing a thorough assessment of Alpine Ski'sLevel Two Integrated Course, itwasfound thatthree particular modules failed to achievethestatedcourse objectives. The evaluation may
Introduction 5
indicatefault with the course conductor.the learning activities.thecontent, or someothe r factor. Thisinformationwould probably stimulatea hostof reactions that some stakeholdersmay notbepreparedtoaccept.
2.Peoplegeneral ly dislike chang e. TheNCCPhasbeen operating fornearly20 years andis considered successful.Inthat time.theprogramhasacquired a privileg edposition of beingwell funded and dearly lovedbyits "keepers andcusto mers."Inotherwords, peopleinvolvedinthe program havegrown extremely co mfo rtab le with thestatus quo.
].Lastly,although evaluation bas beenpraised by educatio nalists and program desi gners as the keyto instructional effecti veness.the comprehensiven essof thelearning objecti ves contained inthe NCC P makesthe taskof implementing avali devaluationoverwhe lming (Robinson,199] .p.25).
TheNCC P recentl ycompleted its evaluationon the entireNCCP.Theevaluatio ndid not include technical courses,which arethe responsibili tyof the individualsportgoverning bodies suchas Judo Canada. The compl eti ondate of tbis projectwasMay1996.This evaluation proj ect servedthreegoals:
1.It producedcredibleinformationforthecoaching advisory commi tteestoaid in improvingthe NCCP Program.
2.It produc ed credible informationforthe coac hing advisory committeesto assistwith the revision of the Theory Component.
3.It includedthe involveme nt ofthe keystakehold er groupsintheongoingevaluation of theNCC P.(NCC P Evaluation Project, April1996)
Introduction 6
Limitation ,oftb eStu dy
Theevaluationofthe LevetTwo forthesport ofjudobad some inherent Iimitenons.These limitations include:
I.Itexaminedonlyone ofthe threetechnicalcoursesdesignated forcoac:bes whoinstruct developing athletes.TheevaluatordidootincludeLevelsOne'&:ThreebecauseLevel OneTechnicalhadj ustbeenrevisedandLevelThreecourseswereonly offeredevery otheryear.A coursewasofferedOctober1995andanotherinJune 1997.Thenumberof coachesparticipatinginthesecourseswastoofew to effect a comprehensi ve and meaningful evaluation.
2.Therewerea limitednumberofparticipan ts evaluated.The evaluationstookplaceinall tenprovin ces.andthe Yuko n.Thecourse wasnotofferedinthe North WestTerrito ries becausetheirsmall association has noqualifi ed course conductors.Their coachesreceive NCCPcoursetraininginSaskatchewan.
3.Theevaluatorisa member ofJudoCanadaanda Master Course Conductorforthecourse being evaluated. Thereisthepotential for bias.andtheselection of an evaluation model orapproach.,alongwithits implementation, wasdonewith this awareness.
DefinitionofTerm,
Thefollo wingare someoftheterms anddefinitions thatwillbeusedthroughout this study.
Coachin gAssocia tio n orC aDada.The Coaching Associationof Canada (CAe)isthesportarm of the GovernmentofCanadawhoareresponsible foradministeringthepolicies of government and allocatingfundsas theypertaintosportsinCanada.
Introduction 7
NationalCoachingCertifica tionCoun cillNCCC).TheNationalCoaching Certification Council(NCCC)isthe bodyresponsiblefor the activitiesanddirection oftheNeep.
Nation alCoachi n gCerti ficati onProgram(NCCP).TheNational CoachingCerti fica tion Program (NCCP) isa primaryprovider of educational informationand courses for thosecoaches who work with athletesinCanada (CAC,1996).
Judo. Frederick(199 1) defined judo as"Way ofgen tlen ess, a nonviolent,basically defensi ve martial art. createdin1882 by KanoJigoro (1860- 1938).In1964 judo emerged from its former martial arts status to become a trueOlympic sport.
Kodokan Judo.The proper name for the sport and art. ofjudo which was foundedbyKana Jigoro,more commonly knownasjudo.
LevelTwo Tech nic al.There are three component parts to coaching certification foreach sport withinthe NCCP:Theory,Technical and Practical.Eachcom ponent of thecertification process isaccomplishedbytaking and passing acourse atthatlevel.
Organizati onoftbe Stu dy
The evaluation of tbe NCCPTechnicalTw o course for judo wasorganiz ed around two phases.Thefirstphase included a descriptionof the NCCP technical two course for judo .Italso includedan identification ofthe stakeholdersthatwereaffected bythe course, the audience
lntroduction 8
concerns and issues,the evaluation process and methodo logy,theevaluati onstandards.and their representativecriteria.The secondand final phase provided a briefdescription ofthe evaluation process, and included thefindingsfrom the standards'evaluation.TheLevel TwoTechnical for judowasthenevaluat ed against thestandardsfromphase ODe.The finalreportwasproduced withrecommendationsandconclusions derived from quantitative and qualitativedata derived from the phase two evaluation process.
Thisthesisis organized into fivechapters . ChapterOnerepresents anintrod ucti on to the study,and a brief history of the NCCP (National Coaching Certificatio nProgram)inCanada.
Thischapteralso discusses thesignificanceof the study and itslimitations , as wellas the organizationoftbe study.ChapterTwo presents a review oftheliterature on coachingeducation andevaluation. as well as educational program evaluation.ChapterThreepresents the rationale behind using the procedures thatwere followed and the selectionof the evaluationmodel.In addition, this chapter describes the methodologyusedinthe implementationof the study.
Chapter Four presentstheresults of theevaluation,andChapter Fivepresents the evaluators ' conclusionsand recommendationsfor course revision and improvement.
Literature Review 9
CHAPTER 2 Review of the Lite ra tu re
Introducti on
'This chapter provides an overviewofthehistoricaldevelopment of both the sportofjudo and evaluation.Itdiscussesthechronologyofjudoevents. fromitsinception to its inauguration into themodemOlympic games.Italsodiscussesthecategorization ofevaluation methodo logies fromitsfoundingto modemevaluatio n methods.Aswell, this chapteraddressesthe commo nalties and differences of evaluationtheory,bygrouping thedifferent modelsintosix differentcategories .The remainderof lhis chapter concernsitself with providinganoverviewof evaluatio n within thespons conununity.
Historyof Judo Introductio n
Theart ofKodokanJudoderived from the bujutsu (ancientmartial orwarlike) arts of feudal Japan.Predomi.nanJ:amongtheseartswasthatofjujutsu.Thefoundcr ofKodo kanJudo (judo for short) wasDr.JigoroKaoo,aRhodes scholar.and anotededucatorinJapan,who took theartof judoand"used hisinfluence to establisbjudoas the basisof a revitalizedphysical educationprograminJapan{ese schools}"(Reay&Hobbs,1992.p.IS).Thisstart was the instrumentthatput judo on theworldstageandlater into theOlympic Games.Thesportof judo ispracticedinover 92 countriesthroughoutthe world(JudoCanada,1994 ).
LiteratureReview10
Bu jutsu.
Theartsofcom bat have long been associatedwithJapanandcome from a longtr3d.ition embodiedin'"avarietyof fo rms.methods,andweapons,eachofwoicb constitutes apartieular specialization of tbatart"(Ratti&.West brook,1992,p.21).Rani&Westbrook (1992) further categorized"'the entire bodyof these specializations.thegenericartof combat,...(under the term}bujutsu"p.22.Thederivation ofthe wordisfromtheChinese buormilitary dimension, andjutsu orart ofmilitarycombat.Ratti&Westbrook. (1992)furtherdividedbujutsu intosub- categories ofspec::ialization.Eachspecialization,intum,is known as ajutsu,a[Japan ese ) word whichmaybetranslatedas method,art, or tec hniq ue andis indicativeof theparticular waysin whichcertain actions are performed.Historically, eachart or methodhasdeve loped certain procedures orpatterns, whichsetitapanfrom theproced uresandpatterns of otherarts. A specializati onconsists ofa particular.systema ticme thodof using a specificweapon.Veryoften, aspecialization of combat was identified bythenameof thc weapon usedbyits practition ers.
Anexampleofthis[kindof)systemwouldbekenjutsu...theart(jutsu)of tbesword(ken) (Ratti
s:
Westbroo k,1992,p.2I ).ThisidentificationsystemdidDOlusethe nameofthe weapon exclusivelyasameans for identification.Iden ti ficati o n was alsoacccmpaniedbythe principlesusedinthe art.Anexam ple of thisisthat oftheunarmed methods of combat,knownasjujutsu.The wordjuj utsu cameto meantheart(iutsu )ofsuppleness(iu)usedinacertainwayinordertodefeatanoppo nen t(Ratti
&.Westbrook.1992 ).Inothercases,originalstyleswereimpro vedupon orwerechangedto suit the particular styleofthatstudent, orto suitsomeother agenda.OnesuchstyleisAikido.Ai
UteratureReview II (union, harmony),ki(vital breath, energy),do (way).Aikido originatedfrom a more ancient stylecalled aikijutsu(Frederic,1991,p.3-4).
Another method ofidentificationin use wasthe naming of the style after the master of the school.Anexample of thismethodwasKodokan Judo.TheKodokan was the
first scbool where the founder of judo,Jigo roKane,firsttaught.
FeudalJapa n.
Theperiodof perfectionof thevariousmartialarts was, according to Ratti&Westbrook (1992),"the span of nine centuries. from the lateninthand earlytenthcenturi esupto1868, the year of the Meiji Restoration.This was theyear the feudal erainJapanesehistorywas officially proclaimed atanend"(p.22). Kanesuggests:
Theorigin of jujutsu [the forerunner of modern-day judo]islostinthemists of antiquity.
TheNihon Shaki "Chronicle ofJapan",a history,compiledbyimperialcommandin720 AD.refers toatournament of cbikara-kurabe,the contest ofstrength, which washeldin theyear ofthe EmperorSuinin,230B.C.Some historians regard this asthe beginning of sumo,orJapan ese wrestling, whichhassomethingincommonwithjujutsu.The eventis recorded as an important authentic historical proof showing theem bryonic stage ofbotb sumo andjujutsu (Kano,1970,p.21).
LiteratureReview12
Itwas during the Tokugawaperiod,1600 to 1867,thatRatti&Westbrook (1992) suggest was the period where specialization ofthc arts ofbujutsu took place.Among these arts was jujutsu, thepredecessor ofmoclcm day judo.
During the Tokugawa Period, also known as the Endo Period, schools of martial combat became popularwherea novice or student could study bujutsuina specialized school (ryu).
These bujutsu ryuandweretaught and attended by professional fighting men offeudal Japan.
Publicly acknowledged expertsinsome weapon or fighting style taught these schools.The primary purpose of which was education."in the sense:that it involved the transmission of systematic knowledge in the specialization of jujutsu through the use of teaching specialists who were considered capable of producing fighting specialists" (Ratti&Westbrook, 1992,p. 154).It wasfrom these ryu that judo matured.
The Japanese martial arts are classifiedina number ofways. The most prominent of these are armed and unarmed bujutsu. Ratti&Westbrook (1992) havetaken this division a step further by subdividing the armed forms into major, minor and collateral. (See Table 2.1).
LiteratureR.evi.ew13
Table2.1 MartialArtsCatego ries
BujutJuba.Feodal Japaa
s,ear..llUip:
... -
)Wl;-
~ulSU(ll&ginata1 sodcprwnijUl3U
5&SUIl'laIajutlU Swonb••asbip:
ll:IJutsu ltcrIjutsu
aenee il.ijulsu il.ido laIllOjulSU Horw . . ..bip:
lMjulSU jom,jutsu JUibajUlSU S....· · I:
suicijUlSU
.,...,-
bld!ulO_"YO&i
Art.flllutall:
joj_ l bol
...
jodo Artof....jitl:"juuejuGq(jitte)
Art.fdM dlaieA etM.-_. . . .Q::
baDrij_
- ...-
dli&iricijuau--
Ott.narts:
lliajutsu
w_
sIIinobijutsll chibriri-no-jU1$ll
ohum. erljuav )'IlbijUtlU
k....
lWciIwi suijoboltojulSU
Uunnod
...
-
--- - ... ... ...
J...juj
...
k<mpoulSU kiaijulSll...
Itoshi-oo-mewui kosIli-oo-wUari kwniuchi roilwrniuchi
"'
ohinob i....
"''''''''''
~-
WdojuUu
....
-
... ,.-
Source:RJaiandwestbrook.l992.,p.23
Founda tioD!!of KodobnJudo.
During the later partof the nineteenth century. sometwentyjuj utsu ryuexisted.Chie f amon gthese.accordin gtoKana(1970),were:"theTakenouchiryu,Sekiguchiryu.Kyushinryu.
Kilo ryu.and Tenshin- shen 'yoryu.Thelasttwo of which were espec iallystudiedby thelate ProfessorJigoroKane"(p.2).
LiteratureReview14
JigoroKanawasborninMikagea seaside townnear Kobein1860.Atageeleven his family moved to Tokyo.During thisperiodofJapanesehistory,lmperialrulewasrestoredwith theresignationoftbe last Shogun (military dictator)oftbe Tokugawa Sbogunate, Tokugawa Yoshinobu,in1867.The administrativepoweroftbe emperor was restoredin1868.Afterthe collapse ofthe feudalsystem,Japancast outalIremnants offcudallifc, includingthebujlltSU ryu,andlookedtowardEuropeandAmerica asrole models toestablishaDeWorder.According toKaoo (1970),thesinglemostimportanthappening whichsignaled a swift decline ofbujulSU WB.'i"theordinancein1871,prohibiting theSamurai[feudalwarrior) fromwearingtheirswords.
Juj utsu wasnoexceptio nto this.
The Founderof Judo.
TheyoungJigoro Kana,astudentatTokyoImperialUniversity,took.upthedying artof jujutsu in order to protecthimsclffrombullies.Kanaheardofthe powerofjujutsu,"anexercise bywhicha man ofsmall strcngthcanbeata man ofberculean strength"(Kano,1970,p.7).Since juj utsuwasin a state ofdisreputebythe members ofJapanesesociety for beinga violentart which producedruffians,youngKanohada difficult timein findingaschool. He finallymetand studiedunderTeinosukeVagiand laterunderHac hinosuk eFukudaandMasotomoIso,ofthe Tenshin 8hinyo Ryu as wellas Tsunetoshilikubo of the Kito Ryu schoolsofjujutsu(Kano, 1970).Reay&Hobbs(1992) further add thatFukudo&Iso werebothinstructors atthe presti giousKomusho(central martialartscollege).Foll owin gthedeathofFukuda,Kana trainedbriefly underMaster Iso,beforehefinishedhistutelageunder Kuto,master ofKito Ryu school.TheKiloryu schoo ldatedback totheseventeenthcentury.
Literature Review 15
In1882,when Kanohadmastered the ruditnentsoftbese styles, hetaughtjuj utsuathis own dojo(hallwherethemartialarts are practiced).Kananamed his dojo the Kodokan.Instead ofcalling itjujutsu howe ver,he termed it judo,the wayofsuppleness or harmony(Frederic, 1991,pp.65).Frederic(1991 ) referstothe term judo asbeingpreviousl yusedin the Jikishin-ryu styleofjujutsu.When asked whytheterm judowasusedinsteadofjujutsu, Kano replied "whatI teachisoot simplyjujutsu.OfcourseI teach jujutsu,butit isupoo'do'[way or principle]"(p.9 ).
Inaddition toencompass ing thebestof manyjujutsuarts, Kano's judoalsoreflected a broader range of'tecbniq ues.Buttherewereother reasonsfor avoidin gthe term jujutsu.Theseinclude:
Somejuju tsuschools ofte n indulgedinviolentand dangero ustechniquesinthrowing or twistingarmsandlegs.Many people ...believedit was harmful.Kano wishedtoshow thathis techniqu es werenot dangerousand would not needlessl yinj ureanyperson.
Jujutsuhadfall en into disrepute.Some jujutsu masters wereforcedto exhi bit theirskills bywayof demonstratio njusttomakealiving.Othersstaged professionalbouts with other[martial]arts(Kano,1970,p.9).
Judo as Physical Education.
Judobasaninherentduality.Kano(1990)suggestedjudoisamentaland physical discipline whoselessonsare readilyapplicableto the management ofour daily affairs.The fundamentalprinciple ofjudo,andone"thatgovernsall thetechniques ofattackanddefense,is that whatevertheobjective, it isbestattainedbythemaximum-efficientuseof mindandbodyfor thepurpose"(Kano,1990,p.25).
LiteratureReview16
The secondandperhapsthemost importantrole forjudois thatofphysical education and sport.Kano (1990) gavewhathesawastheaim.of physical educationas"makingthe body strong,usefulandhealthywhile buildingcharacter throughmentaland moral discipline"(p.20).
Hefurther concludedthat "asphysical education, many sports cannotberatedhighly - in fact, sho uldbediscardedorimproved-for they failto makethe most efficient use of mentaland physicalenergy and impedepro gress toward the goal of promotingheal th, strengthened usefulness"(Kano, 1990,p.20 ).Itwas forthesereasonsthat judo was promotedasa physical educationprogramforJapanese schools.Kano also feltthatjudofitthe maximumofa physical educationaswell asor betterthanmostsportsin existence atthetime.Inhisjudo Kano created theSeiryoku Zen'yoKokumin Taiiku(maximum-efficiencyfor physicaleducation)aspartofthe do(way).This part ofjudoisone oftherecognizedkata(form) stillpracticedbyjudoka (practitionersofjudo ) throughouttheworldtoday.The kata ofjudo also train practitioners in basic principlesand skills of self-defense.
JudoOntsideJapan.
Intheearlypart:of the twentiethcentury,judo, as a competitivesport, spread its web throughout the world(Ratti&Westbrook, 1992).Judo came fromJap an toNorth Americafirst, before goingtoother continents.
In1902 TheodoreRooseve ltbecame interested injudoand as asignof good will,Jigoro Kanosent Yoshiaki Yamashita,oneofhisbeststudents,to theUnited Statestobehispersonal instructor.A roomwasevensetasideat the WhiteHouse fortraining purposes.In1903.judo
LiteratureReview17
was establishedin Seattle,Washington State,andin Los Angelesin1905.Judo never really reached its growthpotentialinNorth America or Europe until theyears followingWorldWar D.
World WarDturned out tobea mixedblessin gfor the growth and developm entof the sportofjudo.Theinternment ofUnited States andCanadian citizensofJapanesedecentand the occupation ofJapan afterthe war wasthe primary cause for thespread(Reay&Hobbs, 1992).
Theinternmentcam psinboth theUnitedStates and Canada saw the governments of each of these countries forciblystripping moneyand property away fromcitize ns of Japanesedescent Whenthewar was over these refugees, homelessandwith no placeto go,spread across both countriescarrying their judo skillswiththem.InCanada, thefocus of this exodus was Toronto andMontreal. The occupation forcesinJapan mastered judo from skilled teache rs at the Kodokan.These soldiers broughttheirnewfoundskillswiththemto theirbomes all across NorthAmerica,when theirtours ofduty wereconcluded.
Thespread of judo to AustraliaandNew Zealandcame earlyin the twentieth centuryas it didinCanada and theUnited Statesof America. In1928, a clubwasfoundedin Brisbane, Australia byDr.AJ. Ross,whose parents lived in Japan. Ross studied judoat the Kadokan from the age offourteen.Ittook a whilelonger before it reached NewZealand.In1948,Mr.G.
Grundy,who studiedjudoinAustralia, openeda clubinAuckland(Reay&Hobbs, 1992 ).
According to Reay&Hobbs (1992),Russia was themost successful newcomerto judo.
Inan efforttodeterminethebest wrestling systeminthe world,Anatoly Kharlampriesandhis
Literature Review 19
Judoin Cauada.
JudoinCanadahadits start underthe tutelage ofMr.Shinzo Takagaki, a native ofTokyo, Japan. who cameto Canada to further his studies at the Universityof British Columb iain Vancouver.Mr.Takagaki studied there for three years,andduring that time hewasinstrumental insetting up The Vancouver Dojo (practice hall where judo is performed)(fakagaki&Sharpe, 1974). One of the members of the original dojo,Mr.SteveSasaki, who immigratedtoCanadain 1922,helped start thefirstdojoin1924 (Judo Canada, 1994).Mr.Sasakitook over the leadership of the club afterMr.Takagaki went back to Japan.In1932,Mr.Sasaki became the firstofficial judo instructor for the RCMP."In1936 ,Professor Kano cameto Vancouver and invited Mr.Sasaki to accompanyhimona tour of North AmericaandEurope.They traveledto the United States, France.Germany and acrossCanada"(Judo Canada. 1994,p.18).Itwas during thistrip that theclubwasgiven a new name byDr.Kano.Thename of Canada'sfirst dojo was called the Kidikan (Judo Canada. 1994).
OnOctober 25,1956,Mr.Sasakiwasinstrumental in forming theCanadian Kodokan BlackBeltAssociation (CKBBA later Judo Canada).In1958,he flewto Tokyo toconfer with the members of the International Judo Federation in order to have Canada as a full member.The bid was successful andMr.Sasaki became thefirstpresident of the CKBBA.Histerm of office lasted until1959 (JudoCanada, 1994).
Another prominent judoka (practitioner of judo)wasMr.Umitsu, a student ofMr.Sasaki. Mr.Umitsu served as presidentofthe association from 1958-1961.In 1958,Mr.Umitsu rep resentedCanadaas both a competitor at the second World Championshipsandas a delegate
LiteratureReview20
tothecongressoftheInternationalJudoFederation (JudoCanada.(994). 101961,Mr.Frank Hatashita becamethethirdpresident oftheCKBBAandheldthe positionuntil 1978.Itwas during this timethatCanada won itsfirstmedalininternationalcompetition.Mr.DougRogers, anairlinepilotwithCanadianAirliDes,won a silver medalatthe1964cham pio nshi p(Judo Canada, 1994).
In1993 .undertheleaders hipofMr.JimKojima,Canadahosteditsfirstworld cham pionshi psinHamilto n,Ontario,andtookits secondsilver medal.NicholasGillofMontreal.
wona silvermedalintheunder78 kgweightdivis ion.Thiswashissecond world medal.At the BarcelonaOlympicsinSpainin1992.Mr.Gill wonabronze medalforCanada.
Program Evaluation
Hi,tonc.1 Pen pectiy e
According toMadaus,etaI., (1984)sixperiodshave elapsedinthe lifeof program evaluation. Thefirstisthe periodpriorto1900,whichhe called the Age ofRefonn.Thesecood time period,from1900until 1930,wasreferred to astheAge ofEfficieocyandTesting.The third.from1930 to1945,wascalledtheTylerianAge.The fourthperiodineval uationhistory, from 1946 to about1957,wasreferred to as theAge ofInnocence.The fifthperiod,from1958 to1972 , was referredtoas the Age of Expansion.The sixthandfinalperiod,from1973tothe present,was referred to asthe Age ofProfess ionalization(Madaus,1983).
UteratureR.eview21
The Age of Reform
The AgeofRefonn sawmany societal changeswhichled00.apathfrom which evaluatio n couldnever retum..Priuwyamongthesewasthe 1ndustriaIRevolution., which transformed theverysttuctureof 19th century society.Thisperiodwasalsomarkedbyattempts toreformeducationalandsocietalprogramsandagenciesinboththeUnited StatesandGreat Britain. lnGreatBritaintherewere continuingattempts toreform.education,thepoortaws, hospitals,.orphanages,andpublichealth. Evaluationsof thesesocietal agencieswereio.formalin nature (Madaus, et,al.,1984).IntheUnitedStates, dwingtheperiodbetween1838and1850, Ho raceMann, Henry Bernardandlater William TorreyHamsinitiated thepractice of data collectiontorational ize educatio naldecisions (Worthen&Sanders,198 7,p.12).
TheAge ofEffi c:ien£)'and Testing1900·1930
Itwastheworkof aDOted American behaviorist oftbe early19OOs,Edward Thorndike, alsocalledthefatherof tbe educationaltesting movement, whopersuaded edUC310rs that measuricg human cbangewasworth whi le.Inthefirsttwo decades of the20thcentury, Thorndike ledthetesting movementto whereit becametheprimarymeansofevaluatingschools.
Thetestsheusedvariedinpurpose.Primarilytheywereusedtodiagnosespecificweaknesses, tostandardize curricula, to evalua teexperiments,and10 assess the overallperformance ofa systemaswell asto makeimportantdeci sionsaboutindividualsinthese systems(Worthen&
Sanders.1987.p. 13).
UteratureReview22
The Tyleria nAge 1930 - 1945
Tbereweremany criticsofthetesting movement.Manyofthese camefromeduca10rs whoarguedthatthenotio nof progressive education.,apragmatic approachtoeducation(Madaus.
et,aJ.1984,p.8),wasunsoundandthat thestudentsfromthesetypesof institutionswould fare poorly in highereducatio nprograms.as compared to studentseducatedinconventional Carnegie- unit curricula(Worthen&Sanders. 1987,p.1S).Because ofthesecriticisms,many leadingunivers itiesrefusedtoacceptprogressive-schoolgraduatesintotheirprograms.Toprove theirnotionscued, the Carnegie CorporationhiredRalphW.Tyler,a noted educator fromOhio StateUniversi ty, intheUnitedStates ofAmerica, todoa study(Madaus, 1983).
In1932.Tyler managedto convince300 collegesto waivetheir entrance requireme ntsfor graduatesfrom30 progre ssi ve schools.Tyler'sapproach consisted of measurin g bytheuseof behavioral objectives.Hedevelopedinstrumentsand proceduresto measurea wide rangeof educationaloutco mes (Worthen&Sanders,1987,p.IS).Eval uation., asenvision edbyTyler, wasa compariso nofintendedoutcomestoactual.outcomes. The approac hwaspopular among thescientificcomm unity because it reflectedthe scientificparadigm.
TbeAgeofl noo«oce1946-1957
Theendof WorldWar ITmarkedthebeginningof lhisera.Civilized societyhadjust come out of a periodof mass destructi on,anditwas ready to moveswiftlyinto getting theworld back onttaekandintorapid growth and developm ent. Societyappearedto givelittle regard to conservation orthesafeguardingof the environment.
Literature Review 2J
Thereseemedto be an eodIesssupply of moneyfrom taxesto fundanythingthatwould enhancethisexpansion,and educationwasregardedasthe core to society'sgrowth...
Accountabilitywas not lookedupon.asbeing important.IneducationmanystUdiesweredone and datacollectcd,..buteducation'sratio nal ewastojustifyexpensice,therewas littleevidence thatthese data were usedtojudge or improvethequali ty of programs oreven.thatthey couldbe usefulforsuchapurpose"(Madaus.etaI.,1994,p.10.).
Thefield ofeval uatio n did dev el op.However,thisperiod markedthe use ofmany standardized tests and theuse of new technologi esto scorethem.Duringthe19505and1960 s, RalphTyler'srationalewasused extens ivelyto train teache rsintestdevelopment(Madaus.,et al.,1984 ).Testin gwas fundedand handledlocall y.Thispractisecame toanend,withthe onset of thc AmericanfSovietrace forspace.
TheAge of E:lpa DSioD1958·1912
TheageofexpansioDwas markedbythelaunchof tbe Russ ianspaccaaft.Sputnikin 1957.TheAmericanpublic,was undertheassumption theU'iA was losingtheracetobefirstin space .lbis was interpretedas a failureinAmericanknow-howandingenuity.They feltthat some ho wthewhol estructureof American scienceandeducatioobadtobeimprovedinorder for themtobeworld leadersinspace.
Thiswasthe era thatsaw educa tio n aleval uati on emerg einto a profe ssion,which cameto dependon taxpayers'money foritsexist e nce.Asa result cf thisdependence,theUnited States federalgovernmentenactedthe National Defense EducationActof1958.Theactprovided for
LiteratureReview24
new educationalprogramsin mathematics.science,andforeign language;andexpanded counse llingandguidance servicesandtestingprogramsinschool districts(Madaus. et. al., 1984).
During thisperiod.aUthe[existing] approacheswereused.Mwell,evaluators evaluated curricul um deve lopment effortslhrougb.theuseoffieldexperime nts(Mad eus.,et aI.,1984),the ideabeingto improvethecoreof American educationinorder to ensuretheleadership ofthe United Slatesinworldtechnologyandbusiness.Cronbach(1963)noticedthatdespite allthe effortsandfunds,theresul ts were far from promi sin g.He criticized the guiding conceptualizations ofevaluationsfortheirlackofrelevanceandutility.Cronbachadvisedthem to tumaway fromthe experimentalapproach,where a treatmentgroupis compared to a control group.usingnorm-referencedtests.HewantedevaluatorsinsIeadtofoc us ongatheringand reporting informationthatcouldhelpprogramdesigners developcurriculum.
MaresultofCronbachandothers,evaluati oninthe UnitedStates ofAmerica became more focused. Theeffortsof politicians such as SenatorRobertKennedyandhiscolleagues were instrumentalinchanging the ElemenwyandSecondary EducationAct of1964 (£SEA), which included specialevaluationrequirem ents .These requirements forcededucators toshift theirconcerninevaluationfrom theory to practice and implementation(Madaus,et al., 1984, p.
13).M aresult ofthis shifting emphasis ,educators found the tools provid edbystandardized testingdid nor work wellwithevaluation.Insteadofmeasuring outco mesdirectly,theywere,at best,indirectmeasuresoflearnin g(Mada us.AirasianandKellaghan,1980).Wheneval uators could not perform the tasks outlinedbyESEA,theprofessional fraternity, PhiDellaKappa.set
LiteratureReview2S
upa nationalStudyCommitteeon Evaluation(phiDeltaKappa.(97 1).ThePhiDeltaKappa Comminecpinpointedmanyproblemsassociatedwiththetraditional Tyterianapproach.
Criterion- referencedtestingwasloo ked at asthealternativetononn-refereocedtesting.This wastheerawhereDCWmodels ofevaluation tookshape.Stufflebeam.AIkin andProviscame up with their management-orientedapproach.andMichaelScriven with hisconsumer -orien ted approachtoevaluati on.
The AgeofProfessionalis m1973 to tbePresen t
Atthisstage in its evolution,evaluation faced aturningpoint.Iteitherhadtoattachitself to other professions .most predominant among these beingresearch,orfonnanewparadigm.
There werethosewhotried"'unsuccessfully to fit their method s toprogram evaluation"(Gube, 1967);however.researchersloo k for differen toutcomesthanevaluatorswereespousing to achieve.Inorder toaddressthis lack of direction, journalswerepublished as a measure of cohesiveness in evaluationideas and methodologies.Inaddition. manyuniversitiesbegan offerin g evaluationcoursesat themastersanddoctoralleve ls.
Toaddressthedemandforsomefonnofdirection.a JointCommitteewasformedin the USA to solidifysome formof professional standardization forevaluationsandevaluators. This committee was the resul t oftwelveprofessionalorganizations pooling theirexpertise.Madaus, et al.,(1984)postulatedthat duringthisperiod evaluatorsincreasingly realized that the techniques of evaluati onmust achieveresultspreviouslyseenasperipheral toseriousresearch;
servethe informatio n needsof tbe clients ofevaluation;deal withsituational realities;meetthe requirementsof prob ity;andsatisfy needsofveracity"(p.16).Thenewprofession,althoughin
LiteratureReview26
its infancy,continued to develop techniques and resultedina newparadigm.andcontin ues to developnewtechniquesandmodelsthatare beingusedtoday.
VaningPbilo!oDb italStanus
Evaluationmaybeclassifiedaccordingtomany differentapproac hes.Madaus (1983)uses nineapproaches. House (1980)proposed a taxonomyof eight major evaluationmodels.Worth en andSanders(1987) divide thevario usapproachesintosix categories,each of which includea numberof models. The authoruses thistaxonomybecauseofitsclearandconcisebreakdown.
Thesixcategoriesproposedare:theobjectives-orientedapproach;thetnanagement -oriented.
approach;theccesum er-ori ented approach; theexpercse-ceieetedapproach;theadversary - orientedapproach;andthenaturalistic and participant-oriented.approach (pp.152-155).
Worthenand Sanders(1987) did a comparative analysisof the six groupingsand analyzed eightareasofcomparison.This analyticalmatrixincluded:
"theproponents,orindividuals whohave written about the approach;thepurpose of tbeevaluation; the distinguishingcba.zacteristicsof each approach;thepastusesof each approach;contributionstotheconceptualizationoftheevaluation. its destincti.ons, new tcm1S or concepts, logicalrelationships.andotheraids suggested by proponentsof eachapproach;criteria forjudgingtheevaluations,i.e.explicitly or implicitlydefined expectationsthatmaybeusedto judge the qualityofevaluations that followeach approach ;the benefits that maybeattributedbyeachapproac h;andthelimitationsor risksassociated withthe use of eachapproach"(p. 151).
Literature Review 27 Obi~tives-Ori~D tedApproacb
~
ODeprominentproponen t ofevaluatingcurricul um. wasRalphW.Tyler .afaculty member at Ohio Stare University since 1929.whoinsistedthatcurriculabe organized around certainobjectives. He didastudycalled the Eight YearEval uation Study oCOhio School Curriculumfrom 1932 -1940.'"Objectives werecriticalbecausetheywerethebasisfor plannin g. becausetheyprovided an explici t guideto teachers,andbecausethey servedascriteria for selectio nof materials,outl iningof conten t,developmeot ofinstructionalprocedures.andthe prepara tionoftests andexamin ations"(Guba and Lincoln,1985.p.4).It was the beliefthatthis approachserved as asystema tic and intellectual approach totheevaluation ofinstructional material.Tyler ' s approachis consideredas theobjectiv es -ori e ntedapproachtoevaluati on.
Tyler' sobjectives-orientedapproachtoevaluationisessentiallytheprocessof deten:nining to what extentthelearning objectivesof any course are beingrealized.Objectives, fromTyler's viewpoint, arecbaogesthatoccurinstudent behaviorpatterns during the course.
Evaluati on.from this perspective,isthe process ofdeterminingthe degreetowhichthese changesinbehaviorare actually taking place"(Tyler,1950.p.69).
Propon~nts.
Some ofthe more notedproponentsoftheobj ectiv es-o riented approachto evaluation , in additio n toTyler,are:Proves,Popham,Teba,Hamm o nd, Metfesseland Michael,andBloo m.
They advanced Tyler' sideas.butessentially their ideasandmethodologies werethesame.The approach wasusedchiefly bycutricul um specialistsand educatorsinthe education system.
Literature Review 28
Characteristics.
Tyler's philosophy of measurementincludedonly measurable objectives .These were tested using pre-test and post-test approaches. Tests were given at the beginning ofa course:
offering in orderto determine the knowledge level of any course: entrant. At the end of a course, the student was again testedinorder to determinethelevel of knowledge gain. Gainhadto be measurable.reliable and valid (Worthenand Saunders, 1987 p.152).
Benefits.
The Tyler approach, accordingto Worthen and Saunders (1987) was simplistic and logical.Itfollowed the scientific paradigm and its simplicity allowed non-evaluators to use its methodologies.It focused on outcomes pre-determined from set objectives.Inaddition, the approaches fosters large amounts of empiricaldata.
~
Theobjectives-oriented approachhasthe following limitations:
I.it canbeover simplistic,implying a linear or inflexible approach.
2. it assesses only theobjectives.ratherthannecessarily judgingthe worth of the program.
3.the worth of the objectives are not assessed;
4. there is no standard fromwhichto measure;
5. the approach also neglectsany transactions that occurwithinthe program. focussing entirely on pre-determined objective of the course offering;
Literatu reReview29
6. Tyler'sapproachignoressignificantoutcomesofthe programifthey are not pertinent to theobje ctivesbeingassessed.The objectives-orientedapproachis primarily scientific,dealingwitha verynarrow scope (Worthenand Saunders,1987, pp.72-73).
Dealing withcontro lledvariablesis easy in alaboratory situation,but very difficultto accomplishina setting wherethesubjectshaverights.
The Consumer-Oriented Approach Overview.
The consumer-orientedapproachis "predominant lya summative evaluation approach"
(Worthen&Sanders, 1987,p.88). Those whowant to know if coursematerialis working advocatethe approach.These are, forthemost part, educatorsandthose whoproduce material forthe educationalcommunity.As well,governmentagenciesare heavyusers.
Proponents.
Most notableamong those whoadvocate the consumer-orientedapproachto evaluationis Michael Scriven. His approachis referredto as Goal-Free.One of MichaelScriven'smajor contributionsto evaluation washisdistinction between "formative"and "summative" evaluation.
Summative,he reasoned, was the basis forthe decision by administrators,whether the entire finishedcurriculum,after beingrefinedby formativeevaluationprocedures,made a significant contributionand alternativeto the schoolsystem thatit warranted theexpense of purchase (Scriven,1978,pp.41-42).
LiteratureReview30
ModeL
MichaelScrivensuggested asevenpointprocessbywhichtoeval uate educational products.This seven pointprocessincluded:
I.evidence ofachievement ofimportanteducationalobj ecti ves;
2.evidenceofachievement of important oon-ed ueationalobjectives (for exam ple.social objectives) ;
3.follow-upresults;
4.Secondaryandunintended effects, such aseffects on the teacher.theteacher's colleagues, otherstudents,administrators, parents.the school.the taxpayer.and otherincidental positiveor negative effects;
5. rangeofutil ity (forwhomwill itbeuseful);
6.moralconsiderations(unjust usesofpunishmentorcontro versialcontent );
7.costs (Worthen and Saund ers, 1987.p.88).
Cbanc1eristics.
lbisapproach advocatesthe useofcheckliststo eval uate educati onalproducts . Some advocateshavediscussed usingguidelinestodeterminetheworthofeducational prod ucts,using standardformsto compileandthen disseminate evaluation informati on.Onesuchsetof guidelines wasproposedby Sanders andCunnin gham (1974),whoaddressed fouraspectsof a product. which include:educational proces ses. content,transportability,andeffectiveness.
Literature Review 31
Bencfits.
Theconsumcr-orientcd evaluation approach bas produced many benefits to educatorsin the field. Some ofthese include giving educatorsin thefield alistof evaluated products that theywould DOthave thetime,or the knowledge to dofor themselves.Consumer-evaluations haveadvancedthe knowledgeof educators aboutthecriteriamostapproprite for their usein selectin g educational products (WorthenandSaunders,1987,p.96).
Limitations.
The consumer-o rientedapproachto evaluationhasthe following drawbac ks:
1.The cost factor.The cost of providingthe serviceofdoing evaluations hastobeabsorbed by the consumer,in this case,the educationsystem.
2.The local educators maylose initiativeindoing their own evaluations on the products they use.Therebas traditionallybeena place for local initiativeintrying untested materialin pilot projects.
Mn agemcDt"()ricDtedAppro ach Overview.
11Jefocus of the management-orientedapproachisdirected primarily at managemenL Withinthe system. itisthe decision-makers ccecems, information needs,andcriteriafor effectiveness that guidethe direction of theevaluatio n. According to Worthe n and Sande rs (1981)the developersofthismethod have relied ona systems approach to evaluati oninwhich decisions are made about inputs,process,and outputs(p.77).The focus ofthe management- oriented approach to evaluation is the ability ofthe management team to effect aquali ty
Literature Review 32
relationshi pinthe evaluation process.Madaus et al.,(1984)refers to thisapproach as the decision making approachto evaluation (p. 48).
Propone nt!.
The two major proponents of the management--orientedapproach to evaluation areDaniel L.Stuffl ebeamandMarvinC.AIk:in.Themost notableof the modelsinthiscategory isthe Context, Input, Process,Product or CIPP Model.developedbyStufflebeam.The modelwas developedinthe late 1960sas an alternativeto the objectives-orientedapproachwhich was the most prevalentat thetime (Madaus,etaI.•(984).
Mod el
The ClPP Model ofevaluationis formativeinnature.Itsmaingoal is to provide improvementinthesystem. Madaus etaI.•(1984) described theClPP Model as an approachthat
"seesevaluation as a toolby which tohelpmake programs workbetterfor the peopletheyare intendedto serve" (p.118).TheClPP frameworkfor evaluation is brokendown into four evaluation areas:contextevaluation, toinform planningdecisions;input evaluati on,toserve structuring decisions, process evalua tion, toguide implementingdecisions ;and product evaluation, to serverecyclingdecisions (Madaus.et al.,1984,p.122).Table2.2representsthe cross section of the four typesof decision makingand accountability frameworks of theCIPP Model.
Literature Revi ew 33
Tabl e2.2 The FourEnluario DTypes
orne
ClPPMood EvaluatioDTypesCODtnt uput Proceu
Prod."
Decision-making Chao", Setprogram Implement Decision to:
objectivesand mategy evaluation terminate,
set priorities Setprogram continue.modify
des;gn theprogram
AttOUI1tability R"",n1 Give chosen Record the Givethe objecti vesand strategyand process. recycling
rational e.keep reasons. decisions.
recordsofneeds.
opportunitiesand problems.
Source:Madaus.et al••1984,p.122
Cban cteristics.
1beinformation derived from theCIPP approachtoevaluation wouldcomefrom decision·malcing oraformative approach,andaccountability orasummativeapproach.The type of information thaltheapproachwould yield isasfollows:
I. Whatneeds areaddressed.bowpervasiveandimportantwerethey.andtowhatextent werethe project'sobjectivesreflective of assessedneeds(addressedbycontext information).
2. Whatproceduralandbudgetingplanwasadopted toaddresstheneeds.whatalternatives werecons idered,why wasit chose n overthem,andtowhatexternwasitreasonab le, potentiallysuccessful .and costeffectiveresponse totheassessedneeds(ad dressed by inputinformation).
3.To what extentwastheprojectplan implemented.andhowandfor what reasons didit havetobemodified(addressed byprocessinformatio n).
LiteratureReview 34
4.Whatreasons.positiveandnegati ve,.as well as intended and unintended.were observed.
andhow didthevarious stakeholdersjudgetheworthand merit of the outcomes.and to what extent werethe needs of the targetpopulatio nmet (product information).(Medaus, et al .•1984.p.124)
Benefits .
According to Worthen andSanders (1987),the CIPP Model of evaluationwas:
1.comprehensive.
2.sensitive tothe infonnationneeds ofthoseinalead ership position, 3.systematicinits approach which satisfied the needs of administrators, 4.proc ess evaluation,inthat itwasongoing throughout, 5.gavedetailed informationforimplementation, 6. provided a wide range of information (p.152).
Limitations.
This model.as withallthe models of evaluations has itlimitations. They are:
I.it's emphasisisonorganizational efficiencyandproduction;
2. it makes assumptions of orderliness and predictabilityin the decision making process;
3.it can beexpe ns ive toadminister andmaintain;
4.it tends to focus onthe concernsandissuesofthe administrative stakeholding audience only(Worthenand Sanders,1987, p.152).
LiteratureReview35 EJ:wrthe-Or;en trdApproach
Ovel'View.
Theexpertise-oriented approach assumes the evaluatorisarecognizedexpertin thearea tobeevaluated.For example,theworthofaprogramwouldbe assessedbycurriculum or subject-matter experts who wouldobservethecurricul uminaction. examine its contentand underlyinglearnin gtheory or,insome other way,glean sufficient information to render a consideredjudgementabout its value (WorthenandSanders, 1987,p.98).
Prop onen ts.
The experti se-o rien tedapproachto evalua tio n hasbeenpractised ever sincecredentials have beenbestowed onothers.Educational institutions havebeengiven degreesto students for centuriesbyexperts orteachersincertain disciplincs.Themostwidelyusedapproac hinthis groupisthe Connoisseurship model.proposedby Elliot W.Eisner.Eisner proposed"eval uators, like other criticsinthe arts,bringtbcirexpertise,andtacit knowledge,tobearinevaluating the quality ofan educational experieece orprogram(KennedyandKerr,199 5,p.6-2).
Model
TheConnoisseurmodelof evaluationisradical ly diffen:nt fromallother mode ls,inthat the evaluatorusesnosetstandards orstandardizedapproach.The expe rt evaluatoruseshis own internal judge menttoassesstheworthof thepro grambeing evalua ted. Theapproachrequires the evaluator to be arecognized expert inthe areato be evaluated.lfthe evalua tionisto have meritwithinthe discipline,thenthe credentials of the evalua tor mustbeimpeccable.
LiteratureReview 36
Thistypeofevaluation would haveto be qualitativeby its very nature.The results depend ontheinmnctsoftbe evaluator. Patton(1990) points out that theapproachis explicitly andpurposefullya qualitativeODe.
Chancteris tics.
TheConnoisseur modelhastwo essential characteristics:
I.The evaluator must have a professional expertise in the areaevaluated;
2.The evaluatorwould havetoberecognized as an expert in the area beingevaluated (Kennedy andKerr.1995.p.6-6).
~
Theexpertise-oriented evaluationapproachbas certain strengths.Thestrengthsof this approach are asfollows:
1.empbasizestbe quality aspect ofeducationaIprograms;
2.theexpert looksfor indicationsofquality.not simplyeffective processes or satisfactory outcomes;
3.theapproach draws attention totheuse ofstandards, whether external totheevaluator or internal to the evaluatorasinthe connoisseur approach «(WorthenandSanders,1987,p.
110).
~
Theexpertise-oriented approachto evaluationhascertainlimi tati o ns andcriticisms that must be addressed.Theselimitationsandcriticisms include:
LiterarureReview37
I.the connoisseur modelisbasedODtheideathatthe expertisevaluatingbasedoninternal standards.Thesestandards.while internal, are oftenbiased.aDdthereforeprone to criticism;
2. thesecond criticismisbasedon the definition of expc:nisc:,andwhatconstitutesenough tovalidat ethe evaluator'sresults ;
3. the expertsusingthis modeloftendoDOtpossessexpertise.or evenabackgroundinthe tools of evaluation;
4. there is publicsuspicion regardingtheapproach (KennedyandKen,1995, p.6-13).
Adven ary-orie nted Appro acb Onniew.
Theadversary-oriented approach to evaluationtakeson the methodology and procedures oftbecourtroom.Therationaleof the approachisbasedonthepremise oCabalanced examinationofallsidesoCtbe program being evaluated.
Proponents.
One ofthe major proponents of thisapproachis RobertL WoIC(197S)whoarguedthat there wasmore tothisapproachthanjustargumenLHebelievedthatanevaluationshouldalso serveasan educationalfunction. Theclients andallstakeholding audiences shouldlearn, through theadversary approac h, thevalueor non-valu eof thepro gram beingexamined.
M!!!!!L
Thisevaluation approach follows a four-stageapproach.lbescinclude:
LiteratureReview 38
I.issue generation.whichinclude
me
identificationanddevelopmentofissuesto be addressedintheevaluation;2 issueselection.whichinvolves el.im.inationoftheissues notin dispute.andselectionand fwther clarification ofissuestobedealtwithinthejudic iarybearing;
3.preparationofthcargument,in whichthe evaluatorscollectthedata.synthesizeitsothat argwnents fortheopposing
views
canbe developed;4.thebearing•discovery sessionstoreviewcasesandprocedures.andaetua.Ipresentation ofcases, evaluationof evidence,and paneldecision (KennedyandKerr, 1995,p.7-5).
Cha ra cteris tics.
The evaluationprocess isbroken down wheretwoteams,eachtaking opposing sidesto theevaluation, work independently.Onesideinthe evaluationis tryingtoprovethe projectis validandtheother team or evaluatoristrying to provetheopposite,thereby applyingan adversarial juxtaposition. Unlikemanyapproachestoevaluation,thisapproach advocatesthe use ofbiasbyitsevaluators, withtheidea thattheevaluator withthe best argument willwinthe evalua tion. Panon(1980)argued thatDOevaluator orteamof evaluatorscocld maintain objecti vity(p.250).
Ben efits.
The adversary -orie ntedapproac hhasbothstrengths and weaknesses. Listed belo w are someof its morepromin en t strengths:
I.Worthenand Sanders (1987) advocated that the approach would showbothpositive and negativeaspects ofthe approach(p.12 1);
Litera tureReview 39 2.Worthen and Sanders (1987) also advocated thatthedatacollectionprocesse s are
diversifiedandallstakeholdersinthe evaluation become involvedintheprocess;
3.the approach is pluralistic,inthat it has the capability of beingusedwith other approac hesandpoints ofview;
4.allsides of theissue havebeenexamin edandthe likelihood of acceptabilityisincreased;
5.the approach bas a built-in scrutiny component, a sort of meta-evaluation. whereall proceduresare open to scrutiny.(Kennedy and Kerr, 1995, p.7· 12).
Limi ta ti ons .
Theadversary-orientedapproach alsobaslimitations. as follows:
I.the cost of such an evaluation could becomeprohibitive;
2.thereisthedangerthatthereisonly afor and against stance,rather than multipleviewsto the samequestion;
3.the approach is best suited to summativeevaluations and notto formative;
4.the approachrelies heavily on the presentation of the arguments;
5.the approach fosters competitio nrather than cooperation;
6.the existing program could be damaged in the process;
7.the judgesare fallibleandthereare noappeals (patton,1980,p.250).
Partici pan t-O ri en ted Approach Overvi ew.
The participant-orientedor naturalisticapproach to evaluationwasdesignedchieflyby Robert Stake.The approachwastakento giveall thosewho haveaninterestinthe outcomeof
UteratureReview40
an evaluationan opportUnity to inputintotheproductandprocessoftheevaluation. Thenew orientation grew during the 1970s and 19805andwas"aimedatobserving and identifyingaU(or as many as possible) of thecceceras,issues.andcoasequences integral to educational enterprise (WorthenandSanders.1987.p.128).Theputicipatory -orien tedapproach.unlikeits predecessors. focuses ontheproces softheevaluatio nandnot theproduc t(Kennedyand Kerr . 1995).This approachisa formative approach but canbe usedsununativelyaswell.
Pro pon ents.
Thechief proponent inlhe participant-oriented approachwasRobertStake.Stakefirst developedhisCountenance model, which focused the evaluation activity on the portrayal andthe processing of judgements onbehalfof program participants.In1995 Stakeshiftedhisemphasis fromtheCountenanc emodel to a Responsiveapproach(Kennedy and Kerr. 1995).The responsive-orientedapproachusedatwelve-stepprocess.which hepresen ts ina clockfashionto represent its inherent flexibility.The steps canbefollowedinany orderandanyone canbe deleted or others added iftheevaluatordeems itnecessary.The method Stake chose is called responsive evaluation,becauseit responds to the needsanddesires ofaUthestakeholding audiences.
Chan eteristics.
According toKennedyandKerr(1995).theresponsiveevaluation model is an evaluatio n design with thefollowin gcba:racteri sti cs:
I. itfocuses on the concernsandissues ofallparticipantgroups;
2. itisemergent indesi gn;
Literature Review41
3. it uses qualitativeor naturalisticmethods;
4. it is sensitiveto the pluralistic values of participants and clients;
5. requires prolonged engagement inthe program setting (p. 8-2).
Othercharacteristics that are peculiar to the Responsive model are as follows:
I.the main feature isflexibility;
2. eachstakeholderinthe evaluation has equal input in the process and product;
3. the modelisbased onthe premise of consensus by allthe stakeho1dingaudiences;
4. the modelcan be both formative and summative,depending on the issues and concerns of thestakeholders.
Stakes modeltookthe shapeof a clock which he calledthe responsiveevaluat ion procedures clock (See Figure 2.1).Evaluators can start at any point on the clock and move about as the need arises. In addition,anyinstnun ent from anyone of the models can be used,or the evaluators can develop and use their0\\'11.
Litera tureReview 42
Responsive Evaluation Procedures Clock
While the proponents of other modelsdislikethe participant-oriented approach, proponents of thismodel feelthattheapproac hgives"genuine undemandingof the inner workingsandintricaci es of theprogram[beingevaluat ed )"(Worth en andSand ers,198 7, p.141).
TheResponsivemod el of evaluationhasthe foUowin g benefits: I.it isthemostflexibl e ofallthe evaluationmodels;
2.the model allowsfortheuseof anyinstrumentationtheevaluators feelisneeded;
Literature Review43 3.the modelcanbeeither formative or summative;
4.the modelallows for the formationanduseofcvaluation standards.
~
Thosewhoprofessthescientifi capproachwouldfecithefreedom oftheparticipant- oriented approachistoo subjective. More realistically,Stake'sapproachbasthefollowing limitations:
I. the modelislabour intensive-the evaluatorbasto immersebimlbersc:lf in the evaluation situation;
2. themodel canbecostly dependingon thedepth the evaluation takes;
3. the approachmaybelengthy,based00thein-depthnature ofthe approach.
Inaddition totheseapprolI(:bestoevaluation. evaluationswere performedonNCCP coursesinthepastusingother approaches.Two oftbcsc approaches arerepresentedbelow.
LiteratureReview44
Review of Relevant Research Studies Resea rch Study One: Ange la Galla nt Thes is
Onrview
The study condu ctedbyGallant(1993)was acompari son of two methods of course delivery forthe NCCP(NationalCoachingCertificationProgram) Theory Two course.The two methods were the classroom course and the home study or distance education course.The course evaluationwascompleted to fulfill the requirements for a masters degree in Physical Education.
Approach
Theresearcher used three groupsinthestudy:thefirst groupwas a controlgroupwhich consistedof peoplenot involved intheNCCP program; the secondgroupconsistedofthose coaches who were involved in the NCCP andtakingthe course offering in a classroom situation.
The thirdgroup consisted of a home study group. The score differences in the study were analyzed using an Analysis of Variance (ANOVA) fora completely randomized design. and a Tukey testto determine the significant difference inthe means (Gallant,199 3,P.ii).
ResultsSum ma ry
An analysis of the data concluded that there was no significant difference in the mean score of the classroom and the home study group.Inaddition.therewasno significant difference between the mean score of the classroom and the control group.Results did suggest, however, thai the mean score difference of the home study group and the control group were significantly different (Gallant.1993,p. 34). The author concluded that the homestudycourse is an effective
