The Unfair Card Game: A promising tool to assess externalizing behavior in preschoolers

(1)

Pratiquespsychologiques22(2016)61–73

ScienceDirect

Psychological

assessment

The

Unfair

Card

Game:

A

promising

tool

to

assess

externalizing

behavior

in

preschoolers

Le

jeu

de

cartes

truqué

:

un

instrument

prometteur

pour

évaluer

les

comportements

externalisés

chez

le

jeune

enfant

I. Roskam

∗

,

M. Stievenart

,

E. Brassart

,

M. Houssa

,

L. Loop

,

B. Mouton

,

A. Volckaert

,

N. Nader-Grosbois

,

M.-P.

Noël

,

M.-A.

Schelstraete

Psychologicalsciencesresearchinstitute,universitécatholiquedeLouvain,10,placeduCardinal-Mercier, 1348Louvain-la-Neuve,Belgium

Received17February2015;accepted11September2015

Abstract

Theassessmentofexternalizingbehavior(EB)inpreschoolersiscrucialindevelopmental psychopathol-ogy.Intheabsenceofanygoldstandardmeasure,newtoolscontributetoamulti-methodandmulti-informant approach.Theaimofthecurrentstudyistopresentandvalidateanewobservationalparadigm,theUnfair CardGame(UCG),intentionallystructuredto increasethe likelihoodthatnegativeaffect,agitationand inattentionwillemergeduringavideo-recordedtask.Itwasadministeredto268youngchildren,andthe resultswerevalidatedbymeansoffactorialanalysis,reliabilityanalyses,inter-rateragreement,test–retest, discriminantanalysesandexternalvalidationwiththeChildBehaviorChecklist(Achenbach&Rescorla, 2004).ThevalidityoftheUCGissupportedbythefindings.

Keywords: Observation;Psychometricproperties;Preschoolers;Externalizingbehavior

Résumé

L’évaluationdescomportementsexternaliséschezlesjeunesenfantsestunenjeucrucialen psychopatholo-giedudéveloppement.Enl’absenced’uninstrumentdemesureoptimal,lavalidationdenouveauxdispositifs

∗_{Corresponding}_author.

E-mailaddress:isabelle.roskam@uclouvain.be(I.Roskam).

http://dx.doi.org/10.1016/j.prps.2015.09.004

(2)

contribueàl’approchemulti-méthodesetmulti-informateurs.L’objectifdecetteétudeestdeprésenteretde validerunnouveauparadigmed’observation,lejeudecartetruqué(UCG).Cejeuaétéconc¸upourfavoriser l’apparitiondesaffectsnégatifs,del’agitationetdel’inattentionaucoursd’unetâchevidéofilmée.Ilaété administréà268jeunesenfantsetlesrésultatsontfaitl’objetd’unevalidationparanalysefactorielle,indices deconsistanceinterne,accordinter-codeur,fidélitétest–retest,analysesdiscriminantesetd’unevalidation externeavecleChildBehaviorChecklist(Achenbach&Rescorla,2004).Lavaliditédel’UCGestsoutenue parlesrésultats.

Motsclés: Observation;Propriétéspsychométriques;Comportementsexternalisés

1. Introduction

Behavior in young children can be considered as an extremely powerful indicator of

well-being and mental health. Behavioral assessment reveals the goodness-of-fit between

the child and his/her environment (Chess & Thomas, 1999; Churchill, 2003). In particular, it shows how the child with his/her own features, such as his/her intellectual and cognitive functioning, temperament, somatic and physiological functioning, copes with environmental requirements. And these requirements come from specific settings wherethe child interacts withcaregiversthemselvescharacterizedbyparticulareducationalandemotionalskills,values, expectations and dreams about the child (Super & Harkness, 1986). As such, behavioral

assessment can reveal if the child’s psychological balance between personal features and

environmentalrequirements isoverdueor broken. Orit caninformus thatthe childis going well.Forthisreason,itsvalidassessmentisneededinanypsychologicalorpsychotherapeutic approach.

From aclinical pointof view, somespecificbehaviorsare of primary interest.Thisis the caseforexternalizingbehavior(EB),whichischaracterizedbyagitation,opposition,aggression, provocation,andtransgressionofsocialnorms(Achenbach&Rescorla,2004).Theidentification ofEBisveryimportantsinceEBmayinterferewiththechild’spersonal,socialandacademic development(Calkins,Blandon,Williford,&Keane,2007;Owens&Shaw,2003).Theaimof thecurrentstudyistopresentandvalidateapromisingnewobservationalparadigmofchildren’s

EB,theUnfairCardGame(UCG).

As it will be exposed below,behavioral assessment can be obtained in preschoolerswith

two main methods, i.e. questionnaire and observational paradigm. Each method has both

advantages and shortcomings. On the one hand, questionnaires are few time-consuming and

they provide information about children’s behavior in various daily situations. However,

informant’sbiascaninfluencetheresultsofquestionnaire-basedassessment.Ontheotherhand, observationsproviderichqualitativeandobjectiveinformationbuttheyaretime-consumingand limited toaveryspecificcontext.In the absenceof a“gold standardmeasure”, it isstrongly recommendedthatamulti-methodandmulti-informantbehavioralassessmentbeconducted(De

Los Reyes&Kazdin, 2005;Kraemeret al.,2003;Roskam, Meunier,&Stievenart,2013).In

thisway,UCGispresentedhereasasupplementtoexistingquestionnairesandobservational paradigms.

(3)

1.1. AssessmentofEBwithquestionnaires

The most widely employedassessment of EB inpreschoolersconsists of parents’ reports viaquestionnairescontainingEBsubscalesorquestionnairesrelatedtoexternalizedsyndromes such as ADHD or conductdisorders.The Child Behavior Checklist (Achenbach&Rescorla, 2004)isoneofthemostfrequentlyusedinresearch.TheStrengthsandDifficulties Question-naire (Goodman,1997)as wellas thePreschool Socio-AffectiveProfile (LaFreniere,Dumas,

Capuano, & Dubeau, 1992) and its short form, the SCBE-30 (LaFreniere &Dumas, 1996),

are also commonly used. With regard to the syndrome-oriented questionnaires, among the

most widely employed are the Conners rating scales (Conners, 1999; Goyette, Conners, &

Ulrich, 1978; Kollins, Epstein, & Conners, 2004) and the SNAP-IV (Bussing et al., 2008;

Swanson et al., 2001). The importance of the observation of child behavior both within

and outside of the parent–child context has been stressed (Wakschlag et al., 2007). The questionnaires are therefore sometimesalso completed bypreschool teachers, whoalso have the opportunity to observe the children in various daily situations (Kerr, Lunkenheimer,

& Olson, 2007). However, unlike with school-aged children and adolescents, self- and

peer-reports can only be used with extreme caution with young children (Clarke-Stewart,

Allhusen, McDowell, Thelen, & Call, 2003). In the rare studies, which have considered

self-reports, the preschoolers were interviewed by a trained research assistant who reported their responses onLikert-type scales(Perren,VonWyl, Stadelmann, Bürgin,&von Klitzing,

2006).

1.2. AssessmentofEBthroughobservation

ThespecialimportanceofobservationforassessmentofEBduringthepreschoolperiodhas beendelineated(Wakschlagetal.,2005).Analternativetoquestionnairesisprovidedby obser-vationalparadigms.Thesebriefmethodsareadministeredinastandardizedmanner.Ratherthan life-time symptomatologyasinquestionnaires,theyassesscurrentbehavior(Wakschlagetal., 2005).Observationalparadigmsareintentionallystructuredtoincreasethelikelihoodthatarange ofclinicallyrelevantEBwillemerge.Suchparadigmsareveryhelpfulforcollectingdatafrom clin-icianswhosediagnosticobservationhasbeendemonstratedtoprovide“addedvalue”beyondthat whichcanbegleanedfromparentorteacherinformation(Wakschlagetal.,2007).Afirstexample istheDisruptiveBehaviorDiagnosticObservationSchedule(DB-DOS),consistingofastructured laboratoryobservation(Wakschlagetal.,2007,2005).Thechildisobservedinseveralcommon tasks,suchascompliance,clean-upandfrustrationtaskswiththeexaminerandwiththeparent

(Wakschlagetal.,2005).AsintheDB-DOS,theMother–ChildInteractionTask(MCIT)consists

ofastructuredparent–childinteraction,whichinvolvesaseriesofepisodesoffreeplay,clean-up andteachingtasks,aswellasaseparationandareunion(Crowell&Feldman,1988;Crowell,

Feldman, &Ginsberg, 1988).Thecodingsystemismade upofbothchildandmother scales

assessingforexampleirritabilityandaggression.Asathirdexample,theSnapGame(Hughes etal.,2002)isaspecificbriefcontextofpeerinteractionwiththechild.SincechildrenwithEBare knowntodisplayovertanxietyandfrustrationincompetitionwithafamiliarpeer(Underwood,

Hurley,Johanson,&Mosley,1999),thechildrenareaskedduringaschoolvisittopickoneof

theirclassmatestoplaywith.TheSnapGameconsistsofariggedcompetitivecardgamebetween twochildrendesignedtoexposethemtothethreatoflosing.Successor failureincompetitive playisimportantforchildren.TheSnapGamehasbeendesignedtoelicitspontaneousagitation, negativeaffectandaggressioninarealisticcontext.However,themostimportantlimitationofthis

(4)

observationalparadigmistheinfluenceoftheclassmateonthetargetchild’sEB(Meunieretal.,

2011).

1.3. CorrelationsbetweenmethodsofassessmentofEB

Limitedagreementbetweeninformantsandmethodshasbeenconsistentlyreportedinprevious literature(Achenbach,McConaughy,&Howell,1987;Collishaw,Goodman,Ford,Rabe-Hesketh,

& Pickles, 2009; Gross, Fogg, Garvey, & Julion, 2004; Roskam et al., 2013). For example,

correlationsmeasuredbetweentheDB-DOSandanimpairmentquestionnaireforchildrenfrom preschooltoadolescence,theChildren’sGlobalAssessmentScale(C-GAS)(Lavigneetal.,1998;

Setterberg,Bird,Gould,Schaffer,&Fisher,1992;Wakschlag&Keenan,2001)completedbythe

mothersrangedfrom–.12and–.21(Wakschlagetal.,2008).Also,thecorrelationsbetweenthe

SnapGameandtheCBCL(aggression,delinquencyandexternalizedbehaviorscales)ranged

from.09to.16whencompleted bymothersandfrom.16to.21when completedbyteachers

(Hughesetal.,2002).Similarly,thecorrelationbetweenthreeoftheMCITchildscales(positive

affect,angerandeffortfulcontrol)andthemothers’CBCL(externalizedbehaviorscale)ranged

from–.10to.02(Robinsonetal.,2009).

VariationsinEBaccordingtoinformantsandmethodsofassessmenthavesometimesbeen consideredasreflectinginformantbiasormethodologicalproblems.Actuallytheycanberelated toseveralotherexplanations(DeLosReyes,Henry,Tolan,&Wakschlag,2009;Wakschlagetal., 2005).First,theycanbeduetorealvariationsofEBacrosscontexts.Children’sbehaviorinthe familysettingisnotexactlythesameasintheschoolcontextforexample.Second,theymaybe explainedbythefocusoftheinstrumentsonEBindailylifeinthecaseofthequestionnairesand oncurrentEBinthecaseoftheobservations.Third,theyareduetoinformants’subjectivityand thequalityoftherelationshipswiththechildwhomtheyarecurrentlyassessing(Kinooetal.,

2009;Roskametal.,2010).Eachsourceofinformationbeingparents’orteachers’reportswith

questionnaires orclinicians’observations actuallyprovides uniqueinformation, indicatingthe importanceofhavingmorethanoneinformantandmethodwhengatheringdataabout

preschool-ers’EB(Roskametal.,2013).TheUnfairCardGameisthereforepresentedhereasasupplement

toexistinginstruments.

1.4. TheUnfairCardGame(UCG)

The UCG is inspired by an adult paradigm focusing on perspective-taking (Bukowski &

Samson,2012).Like theSnap Game,it isbased onacontextofpeerinteraction,except that

inthiscase,thechildfacesavirtualpeer.TheUCGcanthusbeplayedinarangeofcontexts, such as at home,atschool or ina labsession. The flexibilityin itsadministration is agreat advantageforbothresearchersandcliniciansovertheDB-DOSandtheMCIT,whichneedto beadministeredinalab,andovertheSnapGame,whichneedtobeadministeredintheschool context.

TheUCGisacomputerizedversionofamovingcardgame.Twocardsappearonthescreen, oneofwhichcontainstheimageofacandy.Thetwocardsflipoverandstartmoving.Whenthey stop,thechildhastopointtotheonewiththecandypicture.Thechildisinvitedtoplaywitha virtualpartner(achildofthesameagerange):eachtimethechildpointstothecorrectcard,a realcandyis‘given’tothevirtualpartnerand,similarly,eachtimethevirtualpartnerpointsto thecorrectcard,arealcandyisgiventothechild.Thechildstartsbyplayingfiveroundsofthe movingcardgame,andthevirtualpartnerthenplaysanotherfiverounds.Thetrickisthatthe

(5)

childwillalwaysbecorrect,thusallowingthevirtualpartnertowinfivecandies(winningphase) whereasthevirtualpartnerwillbeapoorplayerandallowthechildtoreceiveonecandyonly (losingphase).

1.5. Currentstudy

ThevalidationoftheUCGparadigmwasthemainobjectiveofthecurrentresearch.Validation wasbasedonclassicalfactorialanalysis,reliabilityanalyses,andinter-rateragreements. Com-parisonsbetweenthewinningandthelosingphasesofthegamewerealsoconductedinorderto demonstratetheabilityoftheUCGtoelicitfrustrationsoastoimprovethelikelihoodofEBinthe secondphase,whichisthecoreprincipleofsuchobservationassessmenttools.Furthermore,as observationassessmentoftenreliesonalimitedsampleofchildbehavior,whichisconsideredtobe agreatadvantagebothinresearchandclinicalcontexts,itisnecessarytovalidatesuchan assess-ment procedurewith test–retestreliabilityanalyses. Discriminant analyseswere computed in ordertotesttheUCG’ssensitivitytoage-andgender-relateddifferences.HigherEBwasexpected amongboysandamongyoungerparticipants(Bartelsetal.,2004;Bayeretal.,2012;Bongers,

Koot,vanderEnde,&Verhulst,2004;Ensor,Hart,Jacobs,&Hughes,2011).Externalvalidation

wasconductedwiththepreschoolversionoftheCBCL(Achenbach&Rescorla,2004).Inline withpreviousresearchontheagreementbetweeninformantsandmethods(e.g.Achenbachetal.,

1987;Roskametal.,2013),onlymodesttomoderateagreementbetweenUCGandtheCBCLwas

expected.

2. Method

2.1. Sample

ThisstudywaspartoftheHard-t(w)o-Manage(H2M)Childrenresearchprogramconducted attheuniversityofLouvaininBelgiumwhichreceivedtheapprovaloftheEthicsCommitteeof thePsychologicalSciencesResearchInstitute.Datawerecollectedfromacommunitysampleof 268children(59%girls)agedfrom45to75months(M=59.5,SD=7.09)intheFrench-speaking partofBelgium.

2.2. Procedureandanalysisstrategy

FourresearchassistantsinthePsychologicalSciencesResearchInstituteoftheuniversityof Louvain whohadbeenintensivelytrained insamplinganddatacollectionprocedures, under-took the data collection. Parents were informed about the research programthrough leaflets distributedby surroundingschools,postersandawebsiteandFacebookpage createdfor this study.TheUCGwasadministeredto139children(52%)duringalabsessionandto129(48%) duringaschoolvisit.Thefactorialanalysis,reliabilityanalysesanddiscriminantanalyseswere computed onthissample (n=268).The test–retestanalysis wasperformed withasubsample of 51 participants. Externalvalidation was conductedon asubsample of 201 childrenwhose mothers wereaskedtocompletethepreschoolversionof theCBCL(Achenbach&Rescorla,

(6)

2.3. Instruments

2.3.1. Instrumenttobevalidated

TheUnfairCardGame(UCG)isacomputerizedgamelastingapproximately10minutes.It hasbeendesignedtoelicitspontaneousagitation,inattention,negativeandpositiveaffectinthe contextofplayinteractionwithavirtualpeer.ThecompleteprogressoftheUCGispresentedin detailsinAppendixA.Itcontainsthecompleteinstructionsgivenbythevirtualexaminerwho havebeenpreviouslyrecorded,bythereal examinerwhoispresentwiththetargetchild,and bythevirtualpeerwhohavealsobeenpreviouslyrecorded.TheAppendixAfurthercontains thestandardizedcommentsgivenbytherealexamineraswellasthebehaviorswhicharecoded duringthedifferentroundsofthegame.

Thegamestartswithinstructionspresentedtothechildbyavirtualexaminer(apreviously video-recordedadult).Inthefirst phase,thechildplaysfiveroundsof thegame.Thegameis designedtoensurethatthechildissuccessfulineveryround,allowingthevirtualpartnertowin fivecandies.Accordingly,aftereachround,thechildputsacandyinabowlcontainingaphoto ofthevirtualpeer.Therealexaminer’sroleisstrictlystandardized. Attheendofeachround, he/sheverballyreinforcesthechildforhis/hergoodperformanceandcommentsonhowthisis goodnewsforthevirtualpeer,whohasreceivedthemaximumnumberofcandies.

In thefirstphaseof thegame followingthe instructions,the childwatchesthevirtualpeer playingfiveroundsoftheUCGgame.However,hisperformanceispoor,andthechild conse-quentlygainsonlyonecandy,whichthereal examinerputsintoabowlwiththechild’sname onit.Inthissecondphase,theexaminer’sroleisalsostrictlystandardized.Attheendofeach round,he/sheverballycommentsonthepoorperformanceofthevirtualpeerandthelownumber ofcandiesbeingwonforthechild.AttheendoftheUCG,thevirtualpeerapologizesforhisbad performanceandproposestosplitthegainsequallywiththreecandieseach.

TheadministrationoftheUCGisvideo-recordedandcodedfollowingstandardizedguidelines setoutinamanual(Brassartetal.,2012).Thechild’sbehavioriscodedaccordingtofourscales, i.e.positiveaffect,negativeaffect,agitationandinattention.Thecodingofthesescalesismade withregardtoboththefrequencyandtheintensityofchild’sbehavioron5-pointLikertscales rangingfrom1(neitherfrequentnorintense)to5(veryfrequentandintense).Thefourscalesare codedonceforalltheroundsduringthefirstphaseinwhichthechildissuccessful.Theyarethen codedforeachofthefourroundsinthesecondphaseoftheUCGinwhichthechildobservesthe virtualpeercompletingthetaskunsuccessfully(seeAppendixA).Forexample,positiveaffectis coded1,i.e.nopositiveaffect,ifthechildisneutralordisplaysnegativeaffectand5,i.e.intense positiveaffect,ifthechildsmilesallthetimeduringtheroundandlaughsonatleastoneoccasion. Negativeaffectiscoded2,i.e.lownegativeaffect,ifthechildexhibitsangrybehavioronceor clutcheshis/herheadonce,and4,i.e.highnegativeaffect,ifangrybehaviorlastsforatleasthalf theround.Agitationiscoded3,i.e.moderateagitation,ifthechildshowstwobehaviorssuchas wrigglinginhis/herchair,bitinghis/herlipsorgettingphysicallyclosertothescreen.Asafinal example,inattentioniscodedas3,i.e.moderateinattention,ifthechilddisplaysthreeepisodes ofdistractioninthesameround,suchasstaringintospaceorturningawayphysicallyfromthe task.

In sum,the UCGprovidesordinal scores rangingfrom1 to5 for positiveaffect, negative affect,agitationandinattentionduringthefirstphaseofthegame,i.e.thewinningroundsand duringeachofthefourlosingroundsofthesecondphaseofthegame.Thepsychometric prop-erties of the UCGare extensivelypresented andcommented oninthe results anddiscussion sections.

(7)

2.3.2. Externalvalidity

ThepreschoolversionoftheChildBehaviorChecklist(CBCL)(Achenbach&Rescorla,2004) wasusedasanotherassessmentofEB.Itwascompletedbythemothers(n=201).TheCBCL consistsinanoriginalsetof100itemsintheformofaffirmativesas“Destroysthingsbelongingto his/herfamilyorotherchildren”or“Tooshyortimid”forexample.TheCBCLprovides3-point Likertscales:notatallpresent,moderatelypresent,oroftenpresent.Itisorganizedaccording to seven first order scalesencompassing Emotionally Reactive, AnxiousDepressed, Somatic

Complaints, Withdrawn, Sleep problems, Attention problems, and Aggressive Behavior, and

two-secondorderscales,i.e.InternalizingBehaviors(EmotionallyReactive,AnxiousDepressed,

Somatic Complaints,and Withdrawn),and ExternalizingBehaviors (Attention problems, and

AggressiveBehavior).TheCBCLhasbeenwidelyusedinclinicalanddevelopmental studies

(Roskam et al., 2014) as well as for cross-informantand cross-cultural research(Achenbach

etal.,2008).FortheexternalvalidationoftheUCGandtoremainwithinthescopeofthisstudy, weparticularlyfocusedonthe“AttentionProblems”and“AggressiveBehavior”scalesincluded inthesecondorder“externalizingbehavior”scale.Inoursample,theinternalconsistencywas good,with␣=.88forthe“aggressivebehavior”scale,␣=.72forthe“attentionproblems”scale, and␣=.89forthe“externalizingbehavior”scale.

3. Results

3.1. Factorialandreliabilityanalyses

Sincethefourscalesarecodedonceforallinthefirstphaseofthegame,ExploratoryFactorial Analysis(EFA)wasperformedonlyinthesecondphase.EFAwascomputedwithprincipalaxis factoringandvarimaxrotationonthe16scoresprovidedbythecoders,i.e.positiveaffect,negative affect,agitationandinattentioninthefourlosingrounds.Fourfactorswereextractedthatfitthe fourscalesperfectly,explaining45.20%ofthevariance.Positiveaffectexplained13.92%ofthe variance,agitation11.04%,negativeaffect10.22%andinattention10.01%.Alltheroundsloaded ontheexpectedfactor(>.37)withnocrossloadings.Table1presentstheresultsoftheEFAas wellastheresultsofthereliabilityanalysis.Insum,thefourscales,i.e.positiveaffect,agitation, negativeaffectandinattention,areclearlydelineatedandappeartobeconsistent.

3.2. Inter-rateragreements

Tenpercentofthevideo-recordedUCGswerecodedseparatelybytwotrainedindependent coders.Theagreementbetweenthetwocoderswasfirstcomputedwithintraclasscorrelations. Forthewinningrounds,itwas.84forpositiveaffect,.89forinattentionand.52foragitation.Note thatinthewinningphaseofthegameforthesubsampleunderconsideration,negativeaffectwas consistentlycodedas1onthe5-pointLikertscaleinthesubsampleconsideredfortheinter-rater agreement.Theintraclasscorrelationwasthereforeimpossibletocalculate.Forthelosingrounds, itwas.94fornegativeaffect,.89forinattention,.84forpositiveaffectand.77foragitation.In additiontotheintraclasscorrelations,aweightedKappacoefficientwasalsocomputedforthe codingbythetwoindependentcodersofthefourobservationsofpositiveaffect,negativeaffect, agitation andinattentionduring thewinningphase andthe16 observationsof positiveaffect, negative affect,agitation andinattention duringthe fourlosingrounds. ThisweightedKappa coefficientwas.72.Notethatafour-hourtrainingsessionwasnecessarytoensurethatthecoders

(8)

Table1

ResultsoftheExploratoryFactorialAnalysis(EFA)andCronbach’s␣.

Factor1 Factor2 Factor3 Factor4

Positiveaffectround2 .83 Positiveaffectround3 .79 Positiveaffectround4 .71 Positiveaffectround1 .61

Agitationround2 .74

Agitationround3 .67

Agitationround1 .62

Agitationround4 .51

Negativeaffectround3 .73

Inattentionround1 .66 Inattentionround2 .66 Inattentionround3 .62 Inattentionround4 .49 Eigenvalues 3.53 2.28 1.90 1.63 Cronbach’s␣ .82 .71 .75 .71 Table2

DescriptivestatisticsforthewinningandthelosingphasesoftheUnfairCardGame(UCG),resultsoft-testsandeffect size.

Winningrounds Losingrounds t(264) Cohen’sd

Positiveaffect 1.74(.73) 1.49(.50) 6.36*** _.39

Agitation 1.61(.73) 1.83(.59) –5.03*** _.12

Negativeaffect 1.20(.56) 1.58(.50) –9.05*** _.71

Inattention 1.67(.84) 1.90(.66) –4.97*** _.30

***_p_<_.001.

werereliable.Insum,theresultsoftheinter-rateragreementanalysesshowthatthecodingsystem ensurethatallindependentcodersreportthesamebehaviorsavoidinginformantbias.

3.3. Comparisonsbetweenthewinningandlosingphasesofthegame

t-testcomparisonsbetweenthetwosubsequentphasesofthegamewereconductedto demon-strate theabilityof theUCGtoelicitfrustrationsoas toimprovethe likelihoodof EBinthe secondphase,whichisthecoreprincipleofthisobservationassessmenttool.Asexpected, posi-tiveaffectssignificantlydecreaseinthelosingroundsincomparisonwiththewinningones,and negativeaffect,agitationandinattentionsignificantlyincreasebetweenthefirstandthesecond phaseofthegame.TheresultsofthecomparisonaswellaseffectsizesarepresentedinTable2.In linewiththeseresultswhichconfirmthatUCGelicitsEBinthesecondphase,i.e.lowerpositive affectbuthighernegativeaffect,agitation,andinattention,onlythefourscoresobtainedinthe losingroundshavetobeconsideredasanassessmentofEB.Onlythesefourscoresweretherefore underconsiderationinthesubsequent analyses.Insum,thecomparisonsbetweenthewinning

(9)

Table3

DescriptivestatisticsfortheUnfairCardGame(UCG)scoresaccordingtoageandgender. Youngerchildren M(SD) (n=129) Olderchildren M(SD) (n=139) Girls M(SD) (n=159) Boys M(SD) (n=110) Losingrounds Positiveaffect 1.39(.48)a _1.58_(.50)a _1.44_(.47)a _1.56_(.53)a Agitation 1.80(.61) 1.86(.58) 1.81(.57) 1.86(.63) Negativeaffect 1.56(.48) 1.60(.52) 1.57(.49) 1.60(.51) Inattention 2.01(.66)a _1.80_(.64)a _1.91_(.64) _1.89_(.68)

a_Means_are_{significantly}_different.

andlosingphasesofthegameshowthattheUCGprovidesarelevantsituationforobservingEB inpreschoolers.

3.4. Test–retest

Test–retestreliabilityanalyseswereconductedwithasubsampleof51children.Theseanalyses wereperformedafteranintervalofeightweeks,asitdoesnotmakesensetoexposethechildto ariggedgametwiceinashortperiod.Eightweekscorrespondstothetimethesechildrenwere onawaitinglisttoparticipateinanotherpartoftheresearchprogram.Thecorrelationsthatwere displayedweremoderate,with.26,p=.06,forpositiveaffect,.42,p=.002,fornegativeaffect, .41, p=.003,for agitation,and.27,p=.05, for inattention.In conclusion, test–retestanalyses indicatethatEBwasmoderatelystableovertimewhenitisobservedintheUCGsetting.Inother words,children’sbehaviorscodedatbaselineare notexactlythesamethanthosecodedeight weekslater.Changemaybeduetothefamiliaritywithboththetaskandtheexaminerbutalsoto developmentalchangeinchildren’sbehavior(Meunieretal.,2011).

3.5. Discriminantanalyses

t-testcomparisonswerecomputedinordertotestwhetherage-andgender-relateddifferences couldbefoundfortheUCG.Thechildren’sageinmonthswasdichotomizedinordertocreatetwo groupscontaining50%ofthesampleeach.Thefirstgrouprangedfrom45to58monthsofage; thesecondgroupfrom59to75monthsofage.Inlinewiththegoodfactorialanalysis,thescoresin positiveandnegativeaffect,agitationandinattentionwereaveragedfromthefourlosingrounds.

Table3presentsthedescriptivedataaccordingtoageandgender.Inlinewiththeexpectation,some

EBsweremorepronouncedamongyoungerparticipantsandgirls:theresultsshowthatduringthe losingrounds,youngerchildrendisplayedlowerpositiveaffect,t(262)=–3.04,p<.01,buthigher inattention,t(262)=2.50,p<.05,thanolderones.Boysalsodisplayedhigherpositiveaffectthan girls, t(262)=–3.04, p<.01.In sum,the discriminantanalysesshow thatUCGis sensitiveto age-andgender-relateddifferencesinchildren’sbehaviorastheywereexpectedintheliterature review(Bartelsetal.,2004;Bayeretal.,2012;Bongersetal.,2004;Ensoretal.,2011).

3.6. Externalvalidation

CorrelationswerecalculatedbetweentheUCGandthemothers’CBCL.Thecorrelationswere inthesamerangeaspreviouslypublishedinterrelations.Aggressionwasnegativelyrelatedto

(10)

positiveaffect(r=–.19,p<.01),andpositivelyrelatedtoagitation(r=.16,p<.05);attentionwas relatedtonegativeaffect(r=.12,p<.10)andagitation(r=.15,p<.05);externalizingbehavior was negatively relatedtopositive affect(r=–.18, p<.01)andpositively toagitation (r=.16,

p<.05).Toconcludeforexternalvalidation,theseanalysesconfirmthatthecorrelationsarein theexpecteddirectionbetweenthetwobehavioralassessmentssuggestingacertainconceptual overlap.However,theyshowthat variationsinbehavioralassessmenthappenaccordingtothe methodemployedandtheinformantinvolved.Thissupportsthestrongrecommendationthatthe bestwaytoassesschildren’sEBisamulti-informantandmulti-methodstrategyassuggestedin theintroductionsection(Roskametal.,2010,2011).

4. Discussion

ThemainaimofthecurrentstudywastovalidatetheUCGasatooltoassessEBinpreschoolers. Therationaleforthedevelopmentofnewassessmentmethodsespeciallyforpreschoolershasbeen presentedintheintroductionsection.SomeevidenceinfavorofthevalidityoftheUCGhasbeen presentedintheresultssection.Thiswillbesummarizedanddiscussedbeforestressingthelimits ofthecurrentstudy.

Inlinewiththegeneralprincipleofobservationalparadigmswhichareintentionallystructured toincreasethelikelihoodthatarangeofrelevantbehaviorswillemerge,significantdifferences werefoundbetweenthewinningandthelosingphasesofthegame,withadecreaseinpositive affect andan increase in negativeaffect, agitation andinattention as expected.The factorial analysis was conducted on the codingof the fourscales, i.e. positive affect, negative affect, agitationandinattention,ineachofthefourlosingrounds(secondphase).Itclearlydemonstrated that suchcodingleadstoconsistentandreliablemean scores explainingasignificantamount ofvarianceinthechild’sbehavior.Furthermore,basedonvideo-recordingandguidelinesina manual,theUCGscoresfromthelosingaswellasfromthewinningphaseswerealsofoundto beclosetoeachotherwhentheywereprovidedbyindependenttrainedraters.Test–retestshowed thatoveraneight-weekinterval,whichwasconsideredasacceptablefortheadministrationofthe sameriggedgame,thescoresweremoderatelystable.Finally,thediscriminantanalysesshowed thatsomeoftheUCGscalesweresensitivetoage-relateddifferencesandtogender-relatedones. Differenceswereintheexpecteddirection,withEBdecreasingwithage(higherpositiveaffect andlowerinattention)andboysdisplayinghigherEB(lowerpositiveaffect)thangirls.

Withregardtotheexternalvalidationwiththequestionnaire,wefoundmodesttomoderate correlationsthatwereinthesamerangeasthosereportedinpreviousstudies(Achenbachetal.,

1987;Hughesetal.,2002;Roskametal.,2013;Wakschlag&Keenan,2001).Thecorrelations

were alsointhe expected direction,withpositiveaffect inthe UCGbeingnegatively related toaggressivebehaviorandexternalizingbehaviorintheCBCL,withagitationbeingpositively relatedtothethreeCBCLscales,i.e.attentionproblems,aggressivebehaviorandexternalizing behavior,andwithnegativeaffectbeingmarginallyrelatedtoattentionproblems.Unfortunately, theUCGinattentionscaledidnotcorrelatewithanyofthethreeCBCLsubscales.Itcouldbethat inattentionbehaviorascodedintheUCGwasmorerelatedtolackofpersistenceinthegame, withdrawalandnon-compliancethantoexecutivefunctioning.

Inconclusion,theUCGcanbeconsideredasavalidobservationmethodfortheassessment ofpreschoolers’EB,i.e.lowpositiveaffect,highnegativeaffect,agitationandinattentionwhen exposedtofrustrationinthelosingrounds.Itoffersanadditionalwaytoratethetargetbehavior thatcanbeusefulbothforcliniciansandresearchersthankstoitsflexibleadministration.Since thereisnogoldstandardmeasureofchildren’sEB,theUCGhasbeenpresentedasasupplementary

(11)

instrumentthatcanbeeasilycombinedwithassessmentbyparentsandteachersaswellaswith alternativemethodssuchasbehaviorchecklists.Itsapplicabilityshouldbetestedinfutureresearch withclinicalsamplesofchildrenreferredforEBbutalsoindifferentculturalcontexts.

Disclosureofinterest

Theauthorsdeclarethattheyhavenocompetinginterest.

AppendixA. Supplementarydata

Supplementary data associated with this article can be found, in the online version, at

http://dx.doi.org/10.1016/j.prps.2015.09.004.

References

Achenbach,T.M.,Becker,A.,Döpfner,M.,Heiervang,E.,Roessner,V.,Steinhausen,H.-C.,etal.(2008). Multicul-turalassessmentofchildandadolescentpsychopathologywithASEBAandSDQinstruments:Researchfindings, applications,andfuturedirections.JournalofChildPsychologyandPsychiatry,49,251–275.

Achenbach,T.M.,McConaughy,S.H.,&Howell,C.T.(1987).Child/adolescentbehavioralandemotionalproblems: Implicationsofcross-informantcorrelationsforsituationalspecificity.PsychologicalBulletin,101,213–232.

Achenbach,T.M.,&Rescorla,L.A.(2004).TheAchenbachSystemofEmpiricallyBasedAssessment(ASEBA)for ages1.5to18years.Theuseofpsychologicaltestingfortreatmentplanningandoutcomesassessment:Volume2: Instrumentsforchildrenandadolescents(3rded,pp.179–213).Mahwah,NJ,US:LawrenceErlbaumAssociates Publishers.

Bartels,M.,vandenOord,E.J.C.G.,Hudziak,J.J.,Rietveld,M.J.H.,vanBeijsterveldt,C.E.M.,&Boomsma,D.I. (2004).Geneticandenvironmentalmechanismsunderlyingstabilityandchangeinproblembehaviorsatages3,7, 10,and12.DevelopmentalPsychology,40,852–867.

Bayer,J.K.,Ukoumunne,O.C.,Mathers,M.,Wake,M.,Abdi,N.,&Hiscock,H.(2012).Developmentofchildren’s internalisingandexternalisingproblemsfrominfancytofiveyearsofage.AustralianandNewZealandJournalof Psychiatry,46,659–668.

Bongers,I.L.,Koot,H.M.,vanderEnde,J.,&Verhulst,F.C.(2004).Developmentaltrajectoriesofexternalizing behaviorsinchildhoodandadolescence.ChildDevelopment,75,1523–1537.

Brassart,E.,Houssa,M.,Mouton,B.,Volckaert,A.,Nader-Grosbois,N.,Noël,M.-P.,etal.(2012).Codingsystemofthe UnfairCardGame(UCG).Louvain-la-Neuve:UniversitécatholiquedeLouvain.

Bukowski,H.B.,&Samson,D.(2012).Effectsofone’semotionalstateonperspective-takingandprosocialbehavior.

PaperpresentedattheAnnualmeetingoftheSocietyforSocialNeuroscienceNewOrleans,USA.

Bussing,R.,Fernandez,M.,Harwood,M.,Hou,W.,Garvan,C.W.,Eyberg,S.M.,etal.(2008).Parentandteacher SNAP-IVratingsofattentiondeficithyperactivitydisordersymptoms:Psychometricpropertiesandnormativeratings fromaschooldistrictsample.Assessment,15,317–328.

Calkins,S.D.,Blandon,A.Y.,Williford,A.P.,&Keane,S.P.(2007).Biological,behavioral,andrelationallevels ofresilienceinthecontextofriskforearlychildhoodbehaviorproblems.DevelopmentandPsychopathology,19,

675–700.

Chess,S.,&Thomas,A.(1999).Goodnessoffit:Clinicalapplicationsfrominfancythroughadultlife.Adolescence,34,

800.

Churchill,S.L.(2003).Goodness-of-fitinearlychildhoodsettings.EarlyChildhoodEducationJournal,31,113–118.

Clarke-Stewart,K.A.,Allhusen,V.D.,McDowell,D.J.,Thelen,L.,&Call,J.D.(2003).Identifyingpsychological problemsinyoungchildren:Howdomotherscomparewithchildpsychiatrists?JournalofAppliedDevelopmental Psychology,23,589–624.

Collishaw,S.,Goodman,R.,Ford,T.,Rabe-Hesketh,S.,&Pickles,A.(2009).Howfarareassociationsbetweenchild, familyandcommunityfactorsandchildpsychopathologyinformant-specificandinformant-general?JournalofChild PsychologyandPsychiatry,50,571–580.

(12)

Crowell,J.,&Feldman,S.(1988).Mother’sinternalmodelsofrelationshipsandchildren’sbehavioralanddevelopmental statusinmother–childinteraction:Astudyofmother–childinteraction.ChildDevelopment,59,1275–1285.

Crowell,J.,Feldman,S.,&Ginsberg,N.(1988).Assessmentofmother–childinteractioninpreschoolerswithbehavior problems.JournaloftheAmericanAcademyofChild&AdolescentPsychiatry,27,303–311.

DeLosReyes,A.,Henry,D.B.,Tolan,P.H.,&Wakschlag,L.S.(2009).Linkinginformantdiscrepanciestoobserved variationsinyoungchildren’sdisruptivebehavior.JournalofAbnormalChildPsychology,37,637–652.

DeLosReyes,A.,&Kazdin,A.E.(2005).Informantdiscrepanciesintheassessmentofchildhoodpsychopathology:A criticalreview,theoreticalframework,andrecommendationsforfurtherstudy.PsychologicalBulletin,131,483–509.

Ensor,R.,Hart,M.,Jacobs,L.,&Hughes,C.(2011).Genderdifferencesinchildren’sproblembehavioursincompetitive playwithfriends.BritishJournalofDevelopmentalPsychology,29,176–187.

Goodman,R.(1997).Thestrengthsanddifficultiesquestionnaire:Aresearchnote.JournalofChildPsychologyand Psychiatry,38,581–586.

Goyette,C.H.,Conners,C.K.,&Ulrich,R.F.(1978).NormativedataonrevisedConnersparentandteacherrating scales.JournalofAbnormalChildPsychology,6,221–236.

Gross,D.,Fogg,L.,Garvey,C.,&Julion,W.(2004).Behaviorproblemsinyoungchildren:Ananalysisofcross-informant agreementsanddisagreements.ResearchinNursing&Health,27,413–425.

Hughes,C.,Oksanen,H.,Taylor,A.,Jackson,J.,Murray,L.,Caspi,A.,etal.(2002).“I’mgonnabeatyou!”SNAP!: Anobservationalparadigmforassessingyoungchildren’sdisruptivebehaviourincompetitiveplay.JournalofChild PsychologyandPsychiatry,43,507–516.

Kerr,D.C.R., Lunkenheimer,E. S.,& Olson,S.L.(2007).Assessment ofchildproblembehaviors bymultiple informants:Alongitudinalstudyfrompreschooltoschoolentry.JournalofChildPsychologyandPsychiatry,48,

967–975.

Kinoo,P., Meunier,J.C.,Stievenart,M.,vande Moortele,G.,Roskam,I.,Charlier,D.,etal.(2009).Variabilité desjugementsévaluatifsconcernantles troublesducomportementdujeuneenfant.Enfances/Adolescences,16,

33–44.

Kollins,S.H.,Epstein,J.N.,&Conners,C.K.(2004).Conners’RatingScales – Revised.Theuseofpsychological testingfortreatmentplanningandoutcomesassessment:Volume2:Instrumentsforchildrenandadolescents(3rded, pp.215–233).Mahwah,NJ,US:LawrenceErlbaumAssociatesPublishers.

Kraemer,H.C.,Measelle,J.R.,Ablow,J.C.,Essex,M.J.,Boyce,W.T.,&Kupfer,D.J.(2003).Anewapproachto integratingdatafrommultipleinformantsinpsychiatricassessmentandresearch:Mixingandmatchingcontextsand perspectives.TheAmericanJournalofPsychiatry,160,1566–1577.

LaFreniere,P.J.,&Dumas,J.E.(1996).Socialcompetenceandbehaviorevaluationinchildrenages3to6years:The shortform(SCBE-30).PsychologicalAssessment,8,369–377.

LaFreniere, P.J.,Dumas,J.E., Capuano,F.,& Dubeau,D.(1992).Developmentand validationof thepreschool socioaffectiveprofile.PsychologicalAssessment,4,442–450.

Lavigne,J.V.,Arend,R.,Rosenbaum,D.,Binns,H.J.,Christoffel,K.K.,&Gibbons,R.D.(1998).Psychiatricdisorders withonsetinthepreschoolyears:I.Stabilityofdiagnoses.JournaloftheAmericanAcademyofChildandAdolescent Psychiatry,37,1246–1254.

Meunier,J.C.,Roskam,I.,Stievenart,M.,vandeMoortele,G.,Browne,D.T.,&Kumar,A.(2011).Externalizingbehavior trajectories:Theroleofparenting,siblingrelationshipsandchildpersonality.JournalofAppliedDevelopmental Psychology,32,20–33.

Owens,E.B.,&Shaw,D.S.(2003).Predictinggrowthcurvesofexternalizingbehavioracrossthepreschoolyears.

JournalofAbnormalChildPsychology,31,575–590.

Perren,S.,VonWyl,A.,Stadelmann,S.,Bürgin,D.,&vonKlitzing,K.(2006).Associationsbetweenbehavioral/emotional difficultiesinkindergartenchildrenandthequalityoftheirpeerrelationships.JournaloftheAmericanAcademyof Child&AdolescentPsychiatry,45,867–876.

Robinson,L.R.,Morris,A.S.,Heller,S.S.,Scheeringa,M.S.,Boris,N.W.,&Smyke,A.T.(2009).Relationsbetween emotionregulation,parenting,andpsychopathologyinyoungmaltreatedchildreninoutofhomecare.Journalof ChildandFamilyStudies,18,421–434.

Roskam,I.,Meunier,J.C.,&Stievenart,M.(2013).Thecomparisonandcombinationofmulti-informantand multi-methoddataonpreschoolers’externalizingbehaviour.TheInternationalJournalofEducationalandPsychological Assessment,14,79–93.

Roskam,I.,Stievenart,M.,Meunier,J.C.,VandeMoortele,G.,Kinoo,P.,&Nassogne,M.C.(2010).Commentles parents,lesenseignantsetlescliniciensévaluentlestroublesducomportementexternalisédujeuneenfant?Étudede lavariabilitédesjugementsévaluatifsetdesonimpactsurledéveloppementdel’enfant.PratiquesPsychologiques, 16,389–401.

(13)

Roskam,I.,Stievenart,M.,Meunier,J.C.,VandeMoortele,G.,Kinoo,P.,&Nassogne,M.C.(2011).Lediagnostic précocedestroublesducomportementexternaliséest-ilfiable?Miseàl’épreuved’uneprocéduremulti-informateurs etmultiméthodes.PratiquesPsychologiques,17,189–200.

Roskam,I.,Stievenart,M.,Tessier,R.,Muntean,A.,Escobar,M.J.,Santelices,M.P.,etal.(2014).Anotherwayof thinkingaboutADHD:Thepredictiveroleofearlyattachmentdeprivationinadolescents’levelofsymptoms.Social PsychiatryandPsychiatricEpidemiology,49,133–144.

Setterberg,S.,Bird,H.,Gould,M.,Schaffer,D.,&Fisher,P.(1992).Parentandinterviewerversionsofthechildren’s globalassessmentscale.NewYork:ColumbiaUniversity.

Super,C.M.,&Harkness,S.(1986).Thedevelopmentalniche:Aconceptualizationattheinterfaceofchildandculture.

InternationalJournalofBehavioralDevelopment,9,545–569.

Swanson,J.M.,Kraemer,H.C.,Hinshaw,S.P.,Arnold,L.E.,Conners,C.K.,Abikoff,H.B.,etal.(2001).Clinical relevanceoftheprimaryfindingsoftheMTA:SuccessratesbasedonseverityofADHDandODDsymptomsatthe endoftreatment.JournaloftheAmericanAcademyofChild&AdolescentPsychiatry,40,168–179.

Underwood,M.K.,Hurley,J.C.,Johanson,C.A.,&Mosley,J.E.(1999).Anexperimental,observational investiga-tionofchildren’sresponsestopeerprovocation:Developmentalandgenderdifferencesinmiddlechildhood.Child Development,70,1428–1446.

Wakschlag,L.S.,Briggs-Gowan,M.J.,Carter,A.S.,Hill,C.,Danis,B.,Keenan,K.,etal.(2007).Adevelopmental frameworkfordistinguishingdisruptivebehaviorfromnormativemisbehaviorinpreschoolchildren.JournalofChild PsychologyandPsychiatry,48,976–987.

Wakschlag,L.S.,Briggs-Gowan,M.J.,Hill,C.,Danis,B.,Leventhal,B.L.,Keenan,K.,etal.(2008).Observational assessmentofpreschooldisruptivebehavior,PartII:ValidityoftheDisruptiveBehaviorDiagnosticObservation Schedule(DB-DOS).JournaloftheAmericanAcademyofChild&AdolescentPsychiatry,47,632–641.

Wakschlag,L.S.,&Keenan,K.(2001).Clinicalsignificanceandcorrelatesofdisruptivebehaviorinenvironmentally at-riskpreschoolers.JournalofClinicalChildPsychology,30,262–275.

Wakschlag,L.S.,Leventhal,B.L.,Briggs-Gowan,M.J.,Danis,B.,Keenan,K.,Hill,C.,etal.(2005).Definingthe “disruptive”inpreschoolbehavior:Whatdiagnosticobservationcanteachus.ClinicalChildandFamilyPsychology Review,8,183–201.