Putting theory oriented evaluation into practice: A logic model approach for evaluating SIMGAME

(1)

HAL Id: hal-00703991

https://hal.archives-ouvertes.fr/hal-00703991

Submitted on 4 Jun 2012

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Putting theory oriented evaluation into practice: A logic model approach for evaluating SIMGAME

Jan Hense, Willy Christian Kriz, Joseph Wolfe

To cite this version:

Jan Hense, Willy Christian Kriz, Joseph Wolfe. Putting theory oriented evaluation into practice: A logic model approach for evaluating SIMGAME. Simulation and Gaming, SAGE Publications, 2009, 40(1), pp.110-133. �hal-00703991�

(2)

SIMGAME

! "#$ !% %! ! & $" ' $ ( ) *++,

& "# -

! " ! $ " ! $ !%- ! $ $ & "# $" #! # !%

. ! $$! " !-

! $ . !/ . ! ! ! "# !% 0! 1 %"! !% (! %2

$ ( % ! %" ( ! (! "# - ! ! !% ! %! ! %0 !%

" ! "# !% 0! 3 #! " %

22 (- (! "#- $2 ! 24+25255+- #

% 6 5+-557725+489795+7:+9+79 ;

! ! ! ! $ "

! !1 <-1 = >1 ?- -1 ' ? !1 <- @*++,A- " ( ! . ! !% !0 " ! (

$ %! !0 " ( & -Simulation & Gaming140@5A1 55+B5::-

(3)

= 3 8

" ( ! . ! !% !0 " ! ( $ %!

!0 " (Simgame

< ! !

"%3 ( / $ 0! .1 ! % - 5:1 B9+9+* " 1 ! $ . C4, @9,A *59+B:*D71 E C4, @9,A *59+B,,B:*D7

F - ! !G .- $"-%!

? . = >

0! . !% & ! ! #! (1 " - 51 B89D+ # 1 "

C4: @DD7*A 7,* :*591 E C4: @DD7*A 7,* ,D+, 3 .-H >G 0-

< ! ? !

/ ! ! %0! " ! 1 D++ ?- . & !! -1 &! ! ? ,955, &

@*+8A 9*, 9+*9

<3 !95*DG - $

CCCCCCCCCCCCCCCCCCC

< ! !

"%3 ( / $ 0! .1 " 1 ! $ .

? . = >

0! . !% & ! ! #! (1 "

< ! ? ! / ! ! %0! " ! 1 &! !1 &

Evaluations of gaming simulations and business games as teaching devices are typically end state driven. This emphasis fails to detect how the simulation being evaluated does or does not bring about its desired consequences. This paper advances the use of a logic model approach which possesses a holistic perspective that aims at including all elements associated with the situation created by a game. The use of the logic model approach is illustrated as applied to Simgame, a board game created for secondary school level business education in six European Union countries.

= ? & evaluation methods; logic model; business games; secondary education

! H # H 0! . ( . ( $ ( $" % ! " ! !%"

" ! - ! . ! "%. % 0! ! ! ! 0! ! % ! ! . ! (

(4)

#" ! !%" " ! ( @? ! ' H 1 5,,9A % #!!

$ !% #. (! 3 #!! ! !% 0 % !0 " ! % $! % ( !

@ ! ! ' 1 *++*A- % ( .1 !I"! I"! ! ( $ ( $" ! " !

! ! %" % ! " !0 " 0 ! - #!! ("!% !0 "

0! #!! # !% ! % !0 %! ! @ !! # 1 5,9,A1 H $! % ( ( @ !$" 1 5,95J 1 5,9,A1 ! " ! " ! 0 % " $! ! @? !1 5,,+J %! ' 3 1 5,,*J ! ! ' 1 *++*A1 "#F! !% ! " ! . $! . " 0!

$! " !$! ! I"! @ ! ' ? #" 1 *++4J & ! 1 *+++J & ! 1 ! ! 1 & " !

' $ H1 *++5A-

0! 3 ! ! " ! !% ! ! %! !% $! % ( . " %1 %

!0 " $ ( #! $! ! - ! ! ( 0! ! " !% 3 %! . 0 . (

! 0 "! $ " ! B$!% !% %!0 ! ! ( #" ! "#F! - #!

!/!$ !% #. !0 !3 %" !% #. ! " @*++5A1 ! ' ? #" @*++4A1 =!.

' ? ! @5,,+A1 ? ! @5,,7A1 !! 3 ' ?.$ 1 @5,7:A- "( ! 0! " ! "((!

( $ ( $" ! ! 0! ! ( 1 ! ! " ! ! !0 "% ! ! $ #! (

$# (" " % ! ! % .-

&" ! ! (! ! " ! " % ! $!% ( $ . !/ 1 % ! !

!/! $ "! ! 3 $ H! ! ! 0! !/ ! ! ! ( ! 0 $! -

! 1 " $" %! ( ! % ! 3 " ! !$1 3 " % #! 0 "! !

$ ! # " ! ! "! ! - 3!0! 1 ! $ "% ! ! ! 3 ( 3 ! !

$" ! ! ! 0! $ !% 3 ! 0! ! ( $! % @? !1 5,,7A1 !.

% 0" ( ( ! ! % % $" #! $! $ H! $"

! ! 0! ! ( - " 1 ! # !$ H 3 (whether !%" $! % "

$" ( $ ( 3 H ! !0 . 3! H 3how 3 H - ! ! ! %! # !

! 0! #!! ! !0 " ( ! $" ( $ ( $! %1 ! !

#!! % (! ! ( H 3 !%(! # " ! $! $ " %! ! ! ! 0! ! ( "

" -

H ( ! ! . !0 " ! . (! ! @ 1 !.

' !!$ 1 *++4J & % 1 H ' !0 1 5,,5A- ?! # ! 0! ( 3 ( 3 ! ! 0! !

!! %! %! $! ! " $!B " !% "% ! ! . $ !% 0 "! !0! .% .

!- # % ! ("! ! K 0! .# %. !!$ H !0 "

! %! ( ! ! 0 ! " %K @ # ! -1 5,9+1 - 44A- ! ! $ $

%!0! $! ! ! % (! ! ( $ !0 " 1 3 H ! ! ! !

! 1 3 ! !$! (! ! ! .B ! !% !0 " ! - ! $$ .

(5)

!% 3 ! ! ! ! !$ ! " ! ( $ %! ! !

"%. ( % !/ ( 3 ( $ ! ( $! % %" ! # ! %

" ! " $! -

! L 3 ( ! %" ! ! ! .B ! !% !0 " -

! ! 1 % " ! $! !( % !0 " ( ( $ ( $" 1 %

3 ! $! % ! $! ! ! # !$ ! $ ! % !0 " !

0! % $ !% ! ! %L 3 H 3 #! ( 0! - &"# !I"! .1 ! $ !$! ! .B

! !% !0 " 3 #! %!$ !% % !% ! ! Simgame1 #" ! !%"

$" ( $!-

? ! %% ! ( ! .B ! !% ! !0 " 1 $ $ H! 3

- ! ! $ ( 1 0 ! . #! " !

" !- $ ( " #! ! K ! .B% 0! !0 " K @ ! ' 1 5,9:J ! 1 5,,+A1 K ! .B# !% !0 " K @?! 1 5,,7A1 K ( $ ! . !0 " K @ (! 1 ! 1 "!# ! ' 1 *+++A1 % K ! .B ! !% !0 " K @& $!1 *++4A- ? ! M !/ 1 !

! K ! .B ! !% !0 " K 3 #! !$ .!%1 !0! "( ! #! !% # 0! 0!

#!! " !% ! (! # . ! " !-

! % ! ! ! $ K ! .K- ! ! % ! ! ! ! .

!0 " - ! %1 ! ! ! ! ! "$ " %! ! ! ( $ !0 " !%

%2 !0 " %! ( - ! $ !I"! . " !% ! (! # . 3 K ( $ ! .K ! K ! . (!K1 K ( $ $ %! K1 K ( $ %! K- & ! ! ! $ K ! .K ! .

$ " %! %1 % ! % #! ( !/ 1 " ! ! !% ! $ ! K ( $

$ %! K K ( $ %! K-

! ! ! ! ! .B ! !% !0 " " % #! # !% (

$ %! 3 !/ 3 ! ! ( $1 1 % ! 0 $!

!/ ! !% ! ! ( $M %! !% " $! - ! ! ( $ ( $" ! (

1 ! ( $ %! 0! #! # !% ! ! "$ % !$ !0 %! !

%! 0!% $ ! ( . (.1 !%" 1 % $" ( $ ( ! .- ! ! ( $ %!

" % %! #! 3 ! $" M ! " ! 1 ! ! ! 1 % !/ %

! (! ! ! ! %!% ! ( " $! -

( $ %! . . !0! 0 # ! ! ! !0 ! !/ !

!0 " !% ( $1 3! ( ! $" " ! - " .1 ! !! H %

(6)

0 # ! ! !% ( $ %! - ! ! ! ! !%! 0 # ! @K " KA1 0 # ! ! !%

( $ 0 ! @K ! KA1 % 0 # ! ! !% ( $ ! ! @K " $!KA- ! 1

( $ %! ! ! ! ! !% ( . 3 # /! % 3 %! ( 0 # ! % !

$" " " ! 1 3 #! !/!$ !% #. ! ( $ %! & $( $! 3 3! 3

! ! ! ! ! @ - (" ! :A-

! @5,,+J *++DA % ! ! ! 3 # ! ( $ ! .-(ormative theories

action models(" %! ! %" ( ! %!0! $! % $ !$! ( $- & !

!. ! $ . H! ( !% $ !0 " !/ ! ! !1 $ 0! ! ! ! ! .

$ . !% % " 0! #! ! " !% % ! . $ ! 0 !3 ( ! F! M

% !/ $ ( ! $ ! !. " !%- $ %! . . H! $

! 3 !!% #! !% " " ! ( $ !- ! @*++DA %! !% !$ K

. !$ ( ( 1 ! " ! 1 ! ( 1 % " ( > %! !

(! " % %! 0! ! 0! ! 0 ! K @ - *:A- % ( .1 $ %! !

!/ . " !% !0! 3 "$$ >! 3 ! " I" ! %!% #!

$!% ( ! ( $M ( - 0 " ( ( $M $ %! $ !%

#. ! ( " ! 3 ! ( $ " " !- " ! !0 " %! 0!

$ $$!% ! . $ 0 ( ! ( $M $ !$! -

Causative theories change models1 ! ! %1 ! " ! . %! 0! " !- !

" !/ why ( 0! ( $ # ( ! ! " %! ! % - ? !

!. ! !!%!% !0! .% . !1 !. ! ! ! %! . ( 3! H ! ! !

! ( $ " " ! $ 0!$! - 0! ! $ !/ . $ ( $ %

! !/ $! $ 1 (! $ %! ! ! ! !/ $ %! ! ! ! (

! !0! 3 ! % ! ( $M %! !% (! - !/ 3 . ! " $! ! "

$ ( $1 $ . 0! #! %! !% ! $! $!- & (! $ %! !

! ! ! (! ! "$$ . ! !/ 1 " 1 % ! 3 ! !/ ! !% !/!

"! ! ! " $! ( $- 0 " ( " 0! ! ! !I" ! .> ( !

" ! %! ! %! ! #! 3!! ( $ ! !$! -

? ! ! # ! ! 3 !% %" ! .B ! !% !0 " N !. %

@5,,9A %!0! !% $ ! ! 0! ! $ . ( $ !0 " -

# ! 3 #! " !% #! 3- " ! ! " (" %! % !/ $ ! " ( % " (

(7)

( $ %! !0 " #! " % ! @5,,+A1 ?-=- =! (( " % @*++4 J

*++4#A1 % % ! @*++:A1 % $ H . 1 ! $ 1 ! %!> % = @*++4A-

!. % @5,,9A !% !0! H!. ! %" ( ! .B# !% !0 " - !

3 ( ! ! ! !% !0 " ( $" ( $!

5- ! ! ! % !$ H 3 !%(! $ ( $ ( $" ! " !1 ! ( ! .1 %

!%" ! . ( $ %! 3 3 ! " ! ! ( $ ( $" 1 ! !

! 1 % !/ % ! (! ! ! ! %! !% ! ( " $! 1 3!

# ! %!B! ! - . " ! 0 0!% ! ( $!M %!0! $! 1 %! ! (

%!0! ! M ! ( $! M $! $ ! !0 ! (-

*- ! " ! ! ! ! ! ( $ ( $" !( % ! ( $ %! M

" $! 1 -!- $ . ! ( ! " -

:- ! % $!% ( % H!. # H( " % 1 !-(- ! ! M !0 "

H 3 !%(!1 ! ! 1 $" ! " ! 1 % ( $ ( ! -

4- $ ! $ ! ! ! ( " % . ! !0 "#( " -

D- $ ! ! ! %! ! !%1 ! ! ( $ %! M " $! $ !/ !$- ! !

! %! ! !% %" " . " %! % ! H ! ! -

8- ! ! ! ! % ( ! (! ! > ! !0 " ! " !

!/ --

7- %! . $ 0! " ! % ( ( $ $ 0!$! -

! $ " % $! ! 1 3 !!% $! " ! ! 1 ! %!0! $! !

! ! ( $ %! - ! ! 0 . %! ! %! " !0 " % ! ( $ (

$" #! !0 " !%- ? ! $! ! ! ! ! %. $ . !/ 3! B%! ( !%

! ! # 3 ! 0!% ! # ! $" ( $!M %!0! $! 1 ! ! $ !

!/ ! $! % " 0! . ! " >!%- . !1 %! ! $ ( ( $ %! .

!I" ! !0 !3 ( !$ % ! ! H 3 !%(! ( $ ( $" 1 #" $ ! 0 0! ! 3 ! ( $!M %!0! ! %2 " ! - ? ! " ! " ! I"!

" ( ! % ! 0 ( # ! # !$ ! %! #!% ?! @*+++A1 !

@5,,+A1 !!"3 @*++:A1 $ H . ! - @*++4A1 ! ! % ! 3 %!$ !

3 ! ! 3! ! !% " !0 " -

(8)

? . % 3 " ! " % ( $ %! ! !0 " ( $ ( $" #!

" !%N ! 3 ( " ! #! ! 0! #!! !% ! ! " ! @ (! ! - -1

*+++J ! 1 *+++A- ! ! !% % ( ! !0 " ! M H!. (! -

Choice of focus ( $ %! " (" % ( " %" ( ! !0 " ! M

! . (! - 3! B%! ( !% ( $ %! 1 3 %! 3 ! ( !/ ! !% ! 1

! %! . ! ! !0 I"! % 0 # ! 3 ! %! ( ( ! !0 " "%.-

Vague and distant outcomes ( $ ! $ " $! ! % " $! " ! %

! . !/ ! !% (B ! $ ! ! - $ ( $" ! !!H ! (! ! H

" ! 0 . . !$ H (- & ! ! ( $ %! 3 ! !%! % "

(B ! $ " $! 1 ! % ! ! . " % % " ! ( " $! -

" 1 !0! % " $! " ! ! ! . ! " $! 1 # ! % 3 3! B

"# !% " !( % ( ! !/ ! ! #. H ( ! ! !%! -

Stakeholder participation 0 " ! $ . %! ! %! %!( !! ! H! %! M

%!0! ( ( $M ( $ %! - ! ! ( $ ( $"

"%! ( $! %! ( ! 1 ! 1 1 % # . %$ 0! - ! 0! %!B

! ! #! ! 1 " $" %! ( ! ! 1 !!% $ H! !

"$ # " ! 3 H ( % ! ! ! ( $ ( $" !/ @ "!# ! 1 *+++A-

! $ 0! ! $" !0! #! ! ! !0 " ! H! ! #! " ! $

!/ ! 0! #! $ %! !/ 1 % O0! % $ !/ ! ! " I"! -

Implementation monitoring & ! ( $ ( $" M ( $ %! !

! %! !% ! !0 $ H! 3 H1 ! $ %! ! 3 ! $ ( ! !

3 ! !. ! $ !$! !% %" ( !/! " - !/ $ !1 ! ( $ %! " ! !

%" ( 0! " .! M ! ( " $! 1 % ! ! !0 "

$! " ! ! ! ! ! . ! ! ! $" -

Interpretation of results ! !0 " 0! ! $" ( $! #! ! ! 0!1 ! (

$ %! ! ! ! " !/ ! ! ! @ H1 $ ' ! % 1 5,,*A-

"( !0! # ! K 0!K ( $ %! @?! 1 5,,7A1 #! %! !% 0 %

3 % ! $! - ! $! ! $ $ ! " !% #. ! (

$ %! !/ 1 % ! 0! $ %! " % !/ 3 ! ! ! %" !% % ! ! . 0!

#!! " !% " #. ! ! %! @?! 1 5,,7J 0 % 1 *+++A-

Understanding failure and success ! F" " ! " ( $ ! ! #! .>!%

0 ( $ %! - # ! % !/ ! 3 ! ( $ ( $" % not

" !!%- & $! $! ( $ %! # ! $! 1 #" !0! %! !% ( - (

(9)

$ %! ! % 0! ! ! ! 3 ! %%! ! ! % " . $ $ H ( " " !

$ 0!$! ! $" -

& $! 0! 1 ! ! ! 1 0! #! $! !%- 1 %!0! ( ( $ %! %!$ %

%! # ! ! H 3 !%(! ! !0 " !% #F! M % $ @ ! 1 5,,+A- ? ! $ !

% $! %B ! ! !% ! !0 " ! ! $ ! ! ( "

! 1 %!0! ( ( $ %! 0 0! ! ! ! ! %! - ? !

% $ ( $ ( $" 1 $ !0 " ! !% " #. $" !/ ! 1 ! !

" % #! H H 3 !%(! ! ( $" ! .- !/ ! ( 3

$" 1 %% !/ ! ! ! % $ !%" % ! ( ! . #! !!

! ! ! !I" ! %!0! ( ( $ %! -

! 0! ! " ! %!% " $! - !0! ( ( $ %! " ! . $ !

( $M 0 !3 $ H! ! !0 " " # ! %! ! !( 0! 3! %! .

0! " $! % %!B! ! - 0! ! ! . ! !( % ( $ (

$" - "! ! ! " ! ! ! ( ! 0 $! ! ! ( ! ! !

! ! !% # ! 0! ! ( - & 0! M @5,7:A !% ( B !!

!0 " % ! %0 # ! !0 " H!! ! !.! " " !% !%

" $! % ! ! !0 " %! . $ - !I"! !1 3!

$" $" ! " . %! ! ( "! !% ! ( $M %!0! $! % (

$ ! ! % !$ " ! 3 ! " ( ( $ %! @ 0 % 1 *+++A-

! 3 ( ! " ! 3 ! .B ! !% !0 " #! "

!- ! ! 3 #! !% #. ! ! ( ! ( $ %! # !% !0 " !

#" ! ( $!Simgame-

! #" ! $" ( $!Simgame3 ! " F! !% *++: % *++4 #.

! " ! M ! % % ( $ @ 22333- $( $!- (2 %!/- $ A- !

F! M 0! ( 3! ! @5A $ 0! ! I" . #" ! !%" 1 " . !

% $ ! ! ! !" H (1 % @*A ! ! "%! M !$ . # . % $ ! 0! !

! " ! # $ H! -Simgame3 %!0! !% #. ( $ ( $" !/ ! %

"# !I"! . ! !% :+ ! 0! " ! " ! " 1 >! ! "# 1

! $ .1 . % & 0 H -

Simgame # %B# !% ( $! 3 ! ! ! !% ! $ $" ! $ .M

%" % $ H! ( %! . ! @ (" ! 5A- (! ( " ! "%! ! $

(10)

# ! " ! ! % . !%" % !! #" ! ! - 3 0! !

( $! 3! ! %!0! !%BB K 0! K 3 ! ! ! $B$!$#! ! 3 ! ! 3

! 3 ! $ % K%. $ 0! K 3 ! ! ! $ $ ! ! ( ! ! $$

$ H! - ! ! !% 3 ! !Simgame!0 " 0! (

0! - ! $ !$! ! 0! 3 !% " ! B.!

*++:2*++4- P! ! $ !$! ( ! ( $! ! 1 B !B ! 3 H 3 ( >!%

! ! ! F! ! ! %!I" ! .- 9+ ! ! ! ! ! 3 ! "%!% !

3 ( . ! ! %!% ! 3 H -

BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

! (" ! 5 # " ! !

$! . $ " 3! 0! " 1 %! ! % ( ! ( $! 0! 1 !0 " !/ ! ! !

! % 1 %1 " !1 $! 0 # ! . ( ! ( $!- ! ! 3! !

! " (!% . (Simgame3 %!# ! ( !- " ( %!# ! (1 ! ! !0 !3

3 ! !% %" ( ! ( $! % ! ! ! ! ( !/ ! ! ! @ ! ! ' ! 1 *++4A-

% $ ( Simgame F! 3! ! ! !% !! ! !0! - !

! ! M !0! 1Simgame3 %! ( !% ! ! # !$B ! !% % !/ ! ! ! (

! 0 $! ! # !% "%! ( H 3 !%(! % H ! !% #" ! %

$ (!$! - ! $ !0! 1Simgame3 $! #" ! (! $ !

% ! ( " " ! 0! ! %" " " ! ! ! 0!

%" # ! H 3 !%(!- ! ! !1 % !0! 1Simgame 3 ! %!%

$ ! ! I" #. ! ! ! 0 0! % % H ! !!%!% ! >!

!3 ! ( % " % ($-

0 " % I" . " ! 3 $ !$! !% # !% ! ( $M !! !0! - !

!0 " Simgame’s!%" ! 1 3 "%!% ! "%. ! !% ! !1 3

$ !$! !% %0 !% (! ( $ %!0! $! 1 3 ! . ! ! ( $! 3

! %. !/ ! !- ! !/ ! 0! !0 " 0 ! 1 ! "%. 3 $!%

$ . 3 ( Simgame " !!%!% # ( ( %! !% ! ( " $! 1 3!

%! . ( ( $! $ 0!$! ! ! ! " % ! (- % ( .1 ! .B

# !% 3 ! . "%. ! ! ( ! ! Simgame1 #" !/

! $! $ $ H! ! ( $! " ! " ! ( !/ ! ! !- 3!0! 1

( 0! ! %0 !% ! ( $ ( ! 1 3 ! F . 3 H " ( $

! . 3 ( $ 3 " % !! ( $ %!0! $! % !0 " ! $! $!-

(11)

! 3 ( ! 3 % " 3 ! ! . !0 " (Simgame3 %!0! !%

" -

% ( ! !%" ! !% #. !. % @5,,9A1 ! ! ! . # !%

!0 " %!0! ( ( $ %! - ! !0 " "%. ! !% ! ! 3 % ! !%

$ . $ 0 ( ! ( $!1 " ! 0 ( $ !$! - !I"! .1 !

!/ "%.1 ! " 0! ! . change model Simgame3 $ F ! ! -

! (" ! * # " ! !

! @5,,+A % ! ! $ /!% !%" !1 3 $" ! " . H!

" % $ ! ! ! ! ! % H! %! M 0 !3 @ "!# ! 1 *+++A- & !

! ! 3 $ " ! ( $ ! . Simgame ! %. 0 # !1 ! ! ! %! 3 " !%

%!0! 0! 3 " % #! " !% "# !I"! . % " 3

H! %! @ - (" ! *A- !! " ! 3! ! " !% ( %! 0 ( (! $ %! M

! ! ! % !$ !0 %! !

5- $ !learning psychology ! " ! ! ! !0 % 0 %" ! (

! ! 3! ! % 3 - " ! ! # !% ( 0! % $ %! !

" 0 0 !3 ! ( 3! ! ! #! " ! !. !$ >! " ! . %

# !$B 0 ( ! H!. ! ( " ! @P 3 1 ' "(" %1 5,9,J

! !1 % 1 ' Q ! 1 *++5A- %% .1 = #M @5,94A !/ ! ! ! ( . !

0 %!% 0 " # ! ( %"! ! " 0! ! ( "

@= >1 *++:A-

*- %% $ %! #!% 3 ! ( H! ! ( $ ( % $"

! 0 $! 3 # !% $ !gaming simulation ! " !- ! $ %! !% #.

1 ! ' H! @*++*A 3 ! " 0 %!% # " B ! B " $!

( $ !% ! ! ! " . !% 3 ( $ ( $" - $

% ( ! !0 ! ( 3 #" ! $" 3! ! %! 0!% $

0! 0 !3 ! 1 " @*++5A1 % ? ! @5,,7A1 3! $ ! ! ! ! !

@ % !1 *++*A- !. $ . 0 %!% ( ! ! ( $ ( ! $ ! "

! ! $M $ H!" % " 0 0!$! -

(12)

:- % " !1 !instructional quality ! " ! 3 " !% ! ! # %!

$ ! ( !/ ! !% #.Simgame@P . ' %1 5,98A- M

@*++*A $ %! % " I" . 3! ! % !% !/ ! (! !

!% 3 ! ( !/ - $ $ ! 1 $!3 H

$ %! ! ( 0 " !$ #. !% 1 ' ! ! @*++5A 3! ! % !%

! ! $ %! !$ >! ! ! ! 3 ! (. ! ( ! ! -

%% .1 ! ! H! %! M ! ! 0!1 ! ! . !

( $M $ ! 3 B0 % !% #. % " ( !$ 3 - ! ( %" (

0 % ! 3! ! 3 %- ! ( 3 ! " ( ! ! ! . %! 0!%

$ %! ( !!% 3 ! H! %! M !/ ! ! ! % !/ ! ! # " ! ( $- ! ! %

( 3 %F" ( ! $ %! M ( ! !$! (" !! ! ! ! ! " $! !

( $ %! !%- $ ( ! !% $ (! "#B ! !

$ %! - 3 " ! ! ! ( ( ! "%! M !0! -

! ( $ ! . Simgame1 3 !$! (!% $ ! ! #!! " !%

# 0!1 ! ! !% (" ! :- # . " B ! B " $! $ %! - !outcome element $ ! ! F! ( % % 0 %!% B ! $ % (B ! $ " $! - 3

!/ ! !% ! ! ( ! "%! !0! @ ! ( % ! !A % ! ! !0!

@ ! !A #" ! $ ( ! F! M 0! ( - !input and process

elements ! $ %! $ ! 3! ! !/ ! !% #" ! ( . !

$! ! B ! $ " $! - " ! ! 3 ! ! ! # 0! $! !% " !%

! ! H 3 !%(! 0! #!! " !% ( %! . ! % ! ! $ ! !

$ %! -

! (" ! : # " ! !

!input % $ ! $ %! $ ! ! ! ! % ! ( ! ( $!-

!students’ %!1 $ . ( 0!1 $ 0 1 % %!$ ( ! - !

$ ! previous knowledge %previous experience !/!$ !% H M @5,74A

$ %! "$" 0! ! (1 % #!! $!% ! ! !% . !$ "% ! ! !

$ 3! " !% ! ( " ! @ ! $H! ' ?! ! 1 5,,7A- ! 0! ! %

(13)

! ( " motivation %expectancies 0! ( #!! !( ! !% ! ! 1 .! !.

0! 0! !/! "# "! ! ! ( ! ! % " $! @ !H " '

& ! ! !1 5,,8A- & "%! Mage #!! "%!% ! $ %! ! $ ! 0 !3

%!0! $! . (. #! !/ ! !% ! ! ( . 3 ! (- .1

gender #!! "%!% H! " ! (! %! B ! !% ! (

! ! ! !% ! ( ! 0 $! -

% ( P $ @5,78A1 ! ( ! ! " ! ! ! ! % "

! - ! !1 %% "%! 1 ! ! ! ( ! 0 $! 3!

! ! ! 0! #!! "%!% ! " %! ! $ %! - ! ( !learning

environment1 ! ! "((! ! ! ! 3 $ I" . ! ( $ ( $"

@ ! ! ' 1 *++*J ! ' ? #" 1 *++4A-Content quality ! ! ! $"

$ %! 3 ! ( $! 3 #! 0 % ! ! ! ! . ! # ! "%!

I" ! # ! H 3 !%(!-Game/simulation quality ! ! % ! ! ! ! " ! 3 " ! $" ! ( ! 0 $! - "%! ! " ! " ! ( $!

$ ! 1 "%! $ " . ( ! ( $!1 % ! $ " ( ! ( $!-

P! %! ! ! ! ! % ! ! ( ! 0 $! 1teacher#! 0 %

$ " !/ ( " ! " ! ( " $! @P . ' %1 5,98J ? ( 1

' & %! 1 5,,7A- ! $ %! !% ! training experience with gaming simulations1 ! $ . 3 !3 " $! % #! !/ ! !% #! "

( > ( % $ ( ( ! ! 0! ! ( !/ ! ! !-Motivation %expectancies ! !

$ ! 0! % "% ! ! ! #! 0 1 3 ! $! preparation

! 3 Simgame #! 0 $ ! 3 % ! ! ! 3 ! "( $! !

! ( ! ( 3 $ !/ % ! . !3 $! % @P! ! 1 5,94J '

1 5,,,A-

! $ %! M process $ ! 0 # ! !! !0! 3! ! !% % 0 %"

! (1 ! (1 % ! ! ! 3 ! ! ( ! 0 $! - ! !

0 # ! "%!% ! ! ! !0! 3! ! $ . %! 0!% $ ! ! ! ! ! 0!

! ( % ! (1 % ! $" % ( $ ( ! " !-Intensity of involvement %

! % 0 %" ! ! M %!( !! ! ( $!- 3 "%!% ! ! 0!

! ( ! ( 1 % 0 %" ! ( (! ( % !!B %! #! 0 @& $ '

#! 1 5,9,A- ! % % 0 %" ! ( ! !!% $ #! 3!! H

% " . % ! ! M # ! @P . ' %1 5,98A- ! 3 0! !% #. !over

/under challenged0 # ! 1 3 ! ! $ %! 1 3! ! !/ ! !%

!/! !( 0! "! ! "%! " $! - ? ! K ! 3 ! ( $!K $ ! 1

(14)

3 ! & 0 M @5,,4A 3! BH 3 R B$ %! 3! ! "%!% @time on task % appropriatenessA1 (! ! 3 debriefing- !# ! ( ! ! ! ! ! . ( ! ( $!1 3 ! ! ! ! " ! 3 ! !% %" ( ! ( $!1 ! ! ! ! ( !/ ! ! ! %

" ! !/ ! !% ( %!! ! " %! % ( ! ! ( ! @ ' !1 *++*J

! ! ' ! 1 *++4J ( F" 1 5,,*A-

.1 !Simgame ( . ! 0! ! ( ! 0 $! 1 $ !

% ! 0! ! ( 3! ! %! !% @ ! 1 5,,4J & 0 1 5,,+A-Student student interaction3 % 0 %!% I" 0! ! !$! @intensityA1 3 ! ! ! !! $ "

! % 0 %" 1 % I" 0! ! !$! @qualityA1 3 ! ! ! ! %!( !!

3 ! 3 %" 0! % HB% ! !%- & $ .1 !teacher student interaction

!% I" 0! % I" 0! $ ! instruction ! $ %! $! ! $ "

%% " " ! ! %! 0! !% %" ( ! ( $!-Support$ 3 ! !

" 3 %!I" ! ! ( ! "%! %" ( ! ! ( ! -

!$! ! $ %! M output % $ 3! ! %! 0!% $ . $ F! $ ! %

H! %! " - !short term outcomes $ !% $$!% ! ( 0!1 %

$ 0 ! ( ! ! 1 3! "%! M % ! ! M ! ! ! ( $!- ? ! !

short term " $! ! ! !% SimgameM ! ( !long term outcomes ! ! ! !%

%!0! $! ( ! % ! . # ! %" ( ! F! " $!-

! ( ! " H !% ! $ %! 1 #! !% !. ! ! 0! .

" ! ! ! ! (! %!0! $! - ! $ %! . $ ! (! ! " "! !

$ " ! % $ ! " $! 1 3 " " ! ! . ( 3 ! !$!

! !0! !/! "# "! ! 3 ! !$! ! !/ - 1 ! ! ! $" "

! %! ! %! ! $ %! !% 3 ! !0! 1 " #! 3!! "%! M ! . 0 0!$!

% ! ! " ! ! !0! - "( $ ! %! !% $ %! 3 " % 0! #!!

%! # !1 !-(- ! " ! $ %! ! ! ! ! @*++DA1 " !

! $ %! "$ 3! ! !$ !% %"! ! 3 ( ! ! 1

$! % ( % %! -

! ! ! ! ! . % ! ! ! ( 3 ( $ ( % $" 1

#! !0!% ! ! ! !0 %! ! 3 " % ! % " . $! " H 0!

! - ! ! % ( $ ( % $" I" ! ! ! (! ! " 3 $ !% ! (

! 0 $! % ! %! # . $ ! ! - %! ( ! "! !/ ! 0 % .

! I"! # ! ( " " % ! #! ! !% " (!%

! ( $ ( $" - ? ! ! ! ! 0! # %! !% ! $ ! (! ! ! %

! ( ! ! 1 $" #! ! ( >!% ! ( $$ . %! !% !/ !$! .

(15)

$ !/ "$ 0 . "! !% #. "$! " % ! ! !0! @& 0 1 5,,4A-

! ! ! % " 1 ! ! ! ( ! 0 $! % ! !

$ ( ! ! !! ! !$! 0! #! %! !% ! $! $!- ! ! " ! ? #! (M

%" 0 . $ %! @? (1 ! ! ' ? #! (1 5,,:A "%! ! $! ! @ #

' & 31 5,77A ! ! $ !/ . #. " ( $" 0! ! #! 3!!

! ! ! % ! ! ( ! 0 $! -

$ H! ! # !$ !0! $ ! ! ( ( " !1 $ ( $ ( $"

! # . $ !/ ! ( ! 0 $! - !. ! . ! 0 . ! $! $ . !$B

%. $ % ! - ? ! ! %0 (! ! ( $" ( %

" ! ! ( !/ ! ! ! $ H! ! " ! !0! ( $! ( . " !% # !-

.1 $ ! ! ! ! % $! % ( # !$ 1 % ( ! !0 " M

( 1 ! $ " ! $ %! 3 ! # ! . !$ "%. ! ! !0

! ( 3 Simgame- 3 ! ! $! (" % ( ! %! ( ! ( $!1

$ %! ( ! F! M $ !$! ! @ ! 1 *++DA-

H ( %! ! ! ("$! 1 ! !I"! ! !( ! ( # . $

H !!$!% $ ! !0! ! ! !I"! ! " ( ! %! !% $ %! - ! % ( .1

$ ! ! $ %! $ ! ! !/ . . . ! ! (-

! % ! $ ( !0 " "%.BB %! . " # ! !0! (!

$ 0!$! Simgame % $ $ !$! -

!

! ( ! !0 " 1 3 I"! % #! H!% @ !. % 1 5,,9A

5- ! ! ! . $ ! ! ! ( $! ! $ ! B ! $ " $!

!% ! (! $ %! N

*- ? ! !$! ! " % ! ! $ %! #" ! "# .

!% ( ! " $! SimgameN

? !. @5,7,A1 ! ! " ! ! ! $ K ( $ %! K ! 3 !0 " 1

% ! ! !% !logic $ %! $ !evaluable $ %! ( $- ? ! ! ( $ %!

%! #! ! ! ! ( $1 ! !0 " # ! $ %! ! !% ! ! ! ( $

3 # ! ! " %! ! % ! ( 0! !0 " !/ - ! !0 "

Simgame3 "#F! !0! " % $! % ( 1 3 $ !%

(16)

% ( ! ( (! $ %! @ (" ! :A % ! " !% evaluable change model

Simgame- ! 3 ( $ % 3! ! !% " ! ! !/ ! -

!long term outcomes3! ! !/ "%!% $ ! ! . $ ! !0 " # ! $ %! - ! % 0 %"

" $! !0! @!-(- 0 " ! A #! $! ! !0 . .! ! ( !

( $!1 !. 3! ! #!. % ! !$ ! " "%.- ! 0! 1 "( Simgame3

!/ ! !% #" ! ! ! % 0 %" (B ! $ ( 1 ! . ! 0! .

! 0! H!Simgame . #! !/ ! !% !/! 0! . $ "! ! !$1

! 3 " % !!% 0! . (! $ ! >! %!$ ! " $ ! ! -

? !input student $ ! 1 ! age3 ! $ !% $ ! $ %! ! ! .

& "%! 3! ! ! ! 57 59 .! %1 3 ! " !% 0! . 3 0 !1 " ! %! (

(! " " # ! ! . # ! !% ! ( " $! - ! $!

! !% ! students’ previous experience1 ! . "%! % .!% !%" ( $ ( $" #! !Simgame1 % !input gaming simulation

$ ! - ! ! % #! !/ "%!%1 ! .> ( ! (" ( $ ( 1 !

I" . ! % ( $! $ ! ! 3 ( . 0 ! 1 " !-⁵

.> ( ! ! ! !I" ! $ ( % ! ! ( $! 3 % ! ! I" . ! " ! -

"

0 " % 3 ! !% ( ! >! ! "# 1 & 0 H 1

" 1 . % ! $ .1 3 ! " !% $ ! *D ! 3 :7 ! ! % 489

"%! - $$!% ! . #! ! % ! . ( ! ( $! H 3 !%(! ! @ "%! A %

I"! ! @ "%! % ! ! A 3! ! %$ ! !%- "! ! ( > %

(!B ! F! 1 ! ! ! 3! ! ! # ! %" ! % ! 1

3 ! ! !0 " !B B ! ( % %! !% 3 ! " -

! H 3 !%(! ! 0! (SimgameM ! %!% ( 0! ! ( % $ 3 %!0! !% #.

#" ! !%" ! !/ ! - !% *+ $" ! ! I"! 1 3 "

3! ( ! 0! ! - " ! ! " ! 0! ! I"! 1 $ $"$ ! %

$ / $"$ " 3! ! ! 1 ! 3 % 1 ! I"! % >! !! 3 ( 3!

@% A % ! ! ! 3! -

5 ! I" . ! Simgame 0! #!! !0 " !% #. ! ( $ ( $" % ! !/ !

$ ( "%.-

(17)

!! % ! ! ! 3! ! %! 0!% $ ! H 3 !%(! !B % ! "$#! 3! ( !%

! 3! @ ! ! (! + *+A1 "$#! 3! ( !% ! 3! @ ! !

(! + *+A % ! 3 !I" !% ! % ! ! ! ! $ " ! 3!

@ ! ! (! B*+ *+A- ?! ( ( 3! 3 ! $!% ! BI"! # 1

$ ! ! ! % ! ! "$#! % ! I"! -

? ! ! ! ! " 3! ! " !% ! >! ! ! !$! previous knowledge % cognitive learning ! $ %! 1 ! $ ! 3! ! ! >!% #. !$ % ! !

%! 0!% $ ! "%! % ! ! I"! ! - ! " 0 # ! 3 ! !%

#! ! % ! % ! . ( ! ( $!- # ! 5 3 0! 0 !3 !$ % !

" !% ! . -

! # ! 5 # " ! !

#

0 " R"! 5 ! !% ! $ B ! $ ! ! Simgame-

# ! * 3 ! ! " ! ! H 3 !%(! ! - ( ! ! ! 3!

3 ! ! >! ! . 52* % % %!0 @dMS+-48A 3 %! ! !%- & $ .1 ! ! 3

( ! ! ! ! . ! $! ! ! >! @dMS+-4*A- 1 $ (

%! ! ! ! 3! 3 # ! 0 # !- ! 3 !1 ! ! ! " % ! Simgame

! !% "%! M ! ! H 3 !%(! % ! ! 3 . ( -

! # ! * # " ! !

%% ! ! #F! 0! ! ! " 1 ! ! ( ! ! ! ( 0!1 %

$ 0 % $ 3! ! $! " !% #. "#F! 0! ( "%! % ! ! DB

H! ! @ # ! :A- "( . ! ! ( !I" 0 #! " ! " !

! ! ! ! 1 ! 3 ( ! ! ! !0 %!

5- & "%! M % ! ! M ( ! 0 # ! !( % ( ! !! ! ( % $ -

*- ! ! ( ! ! ! !% #. # ( " 3 # " ! $! ! ( % $ -

:- ! ! ! 3! ! $ ! $ # " ! ( ! " 3! ! ! "%! -

(18)

! # ! : # " ! !

# ! : ! ! "%! M % ! ! M ( $! ! ! ( - P ( ! $ !

0 # ! ! ( $! ! "#F! 0! ( ! ( ! ! - & "%! 1 ( 1 3! ! !

0 ( ! ! ! % 3! ! ! " $ ! @SDS5-*:A-

0! " !0 " R"! 51Simgame " !!%!% ! ( "#

! ( ! ! - 1 "%! M % ! ! . ! ! M ! ! ( ! ( $! % !%

! "% ( Simgame3! ! " ! " . $! -

0 " R"! * H!% 3 ! $ %! M " % ! #" !% ! $

! !% " $! !% 3 Simgame- & !cognitive learning 3 ! .

" $! 0 # ! $! " !% #. #F! 0! ! 1 % ! #! %! !% #! ( ! ( !

$ $ "%! !0! " $!1 ! 3 ( . ! ! ! !/ (

- ! ! #! 3!! ( 0! ! ( % ! !% 3 Simgame3

0! ( !% 0 !( ! . -

$ / $ >! ! ! ! . 1 . ! 3 " $ ( % ! 0 # ! " !%

!( ! 3! ! "%!% @ B3 ! %! ! A1 3 ! " !% !%" !% $ ! >!

(S*+7- 0 # ! %! 0!% $ (B ! 3! ! >B % % >!% ! . 1 !

! " 3 ! 3 # ! 4- 0! 1 ! $# !% ! $ %! M " %

! $ ! !/ !% D8-+T ! 0 ! "%! M ! ! @Rcorr2 S -D8J

F@5815,+A S 57-*D1pU -+5A-

! # ! 4 # " ! !

H ( !student $ ! ! " B !0! ! $ %! 1 "%! Mprevious knowledge

! $ $ "%! ! ( @βS-D:A- ! !/ ! $ %! M !

1gender .!% $ ! ! . - !$ ! "%! !% $!3 #! !

$ ! "%! @βS-5DA-Expectancies and motivation1 ! ! %1 % % . ( ! ! . ! ! 3 " % 0! !/ ! !% $ ! $ %! M "$ -

" ( teacher " 1 ! ! " "((! ! training experience@βS-4*A % motivation@βS-:+A #" !% ( . !% ( "%! ! (-Expected problems

! !% #. !$ !/! !% $ !( 0! "! ! ! !% @βSB-58A- ? ! ! !

% ( ! % ! 3 ! !/ ! !% ! $ %! 1 ! ! M $! !

(19)

preparation3 ( . ! !% 3 "%! ! ( 3 ! !/ $" !

! . -

? ! $ %! M process 1 ! % 0 %" ! ( !0! .intensity of involvement .!% $ #" !( 0! ! !% ( "%! " $! @βSB-54A- !

#! ( ! (!% $" !1 ! ! %1 3! ! ( . ! !% 3

! (-

!interaction with the game $ ! 1 ! game appropriateness 3 (

"! ! ! !% "%! M ! ( @βS-D4A- #! ! ! !% $!

"%! ! !% $ ! ! ! ! ! ( $! 3 3! " !% ! "%! M ! (

! % -Time on task %debriefing ! ! % % ( #"

. -

! ( ! ! ! !0! ! (1 .interaction quality $ (

"%! @βS-54A 3 ! !0 !% ( "%! ! (- ! $ ! "%! ! !% 3

! "%! %" ( ( $! . ! ( ! ! ! 3 ! ( ! ! !%- 1

teacher instruction %teacher support % % 3 ( #" -

&"$$ > ( ! ! " !0 " R"! *1 ! . ! !% "%! M

( 0! ! ( 3!%1 ! ( !0 " H 3 !%(!1 ! $ $

3! ! " % ! ! ! !0! - ! ( !/ ! ! ! % $ 0 3! ! ! $ $

" - ! ! ! ( $!1 ! ! . ! ( $! "%! M ! (

! !I" ! 3 !0! $ ! "# !/ ( ! ( ! " - ! $

! !0 ! 3! ! !/ ! !% # !$ #. ! ! ! % ! I" .1 # 3 3! !

% ! 3 ! $ %! M "$ - ! % ( (! %! % ! . 0 0!$!

! ( $!1 3!0! 1 ! " !/ ! !% % %! ! 0! " ! % " -

! %! 3 "%!% ! $ %! (! ! . $ %!$ ( ! (

! ! - !0 " ! ! #" ! $" !%" ! ( 1 3!0! 1

"((! !% ( "! ! % ! - (! %! ! ! ( $! #!! 3 !

!/ $ " ! ( $! @P " ! 1 P! ! ' !.1 5,,9J 1 *++DA- ! !( 0!

#" ! . 0 0!$! 3 ! % " !/ ! !% % (- !( 0! ! "

$ ( % ! "%! M %! 0 ( %!! . SimgameM %! " ! !$ ! (

! 0! " !- ! 0! !/ $ ( #! #! ( # ! 0!% #! " !

3 ! $ ( "%! " . !!% $ !B%! !% 0 0!$! 3 ! ( $!M $ ! -

1 " $ . " !$ $ ! ( $! . ! (B ! !0 ! ! -

! $ 0!$! ! $ !$! Simgame1 ! % ( "((! ! !%

!$ " % #! " ! ( ! ! Simgame- !. ! ! %. !/ ! ! !%

(20)

( ( $ ( $" 1 $ ! . $! " ! 3 " % ! #! ! ! . %

#! ! - 1 ! % ( ! ! $ 0 ! $ - ! #! H! ! !

! ! $" ( $! 3 ! ! % 3 ( ! 3 !3 ! (

$! %1 %% ! ! ! . ! ! ! $ 0 ! ( $!-

3 $ ! % ( ( ( ! $ ! ! " ! ! 3 Simgame !/ -

$ "%! $!! ! ! % " ! " ! ( $!- !

!( 0! !% !/ ! !% # !$ #. ! ! % ! !$ 1 1

( > % " " ! % ( $! . !!% #! $! $ / $ >!

Simgame-

P. " ( $!3 H (! $ %! ! . ! (Simgame1 !0! 3! !

%! !% 3 " % #! 3 #! $ $ !/ ( ! !0 ! ( " $! -

P! " ! ! 0! . $! % !0 " ( ( $ ( $" 1 "%.M ! "

" % #! %! !% #! !/ . " ! % $ . #! (! ! >!% ! ( $! -

P !% "%.M !/ ! ! ! 3 ! ! .B# !% !0! I"! !!% #!

%% ! !%-

! % ( .1 ! $" ! " . $ . ! "! ! !!% %! # !

(! $ ! >! - ! !/ (! " ! 0 F! 1 3 # ! !

! ! ! . % ! ! $! % . - ! !0 " !/ $ ( #!

$" %! $ - ! $! % ( # !$ $ % ( ! !%

$" ( $! $ . ! . ! !%- ! .1 " % 3 " % #! #! "#F! !%

$" !0! . 1 3 " " ! . !0! (! %!$ % !0! 3 $ ! >!

@ /1 5,,DA- $ H! !$ " ! # ! 3 $ . !0 " !/ - " 1 ! .B# !%

!0 " ! ! !% 3 $! % ( . %!$ % ( !%" ! @ ! ! . '

!! #! (1 5,,,A1 !$ #! !! 3 ! ! #! $ !$! !% " ! $"

( $ ( ! ! -

! ! .1 ! I"! ! !0 %! ! $ ( ! ( " $!

$" ( $! #! $ !% ( ! $ %! - ! $ ! (! ! ! % $

! (1 ! "% ! 3! ! !!%!% 0! !$ . ! (! ! > # ! (

$ @? (1 ! ! ' ? #! (1 5,,:A- &" ! !$ #! " %! H! !

! % ! ( 3 ( $ ( $" - ! ! "%. . ! $ !

% ! -

(21)

$ % ( $ ( $" ! ! 0!1 ! ! .B# !% $ ! #! !

" %! % ( learning within games- $ ! !%! % ! % #!

.>!%1 "%. $!% " !- 3!0! 1 3 ! !0 " ! " ( $!1

0 # ! 0! ( ! ! ! % 0!features of the game1 " ( $! %

! I" .1 0! ! (- " .1 ! ! ! " ! #! "#F! !% . !$

$ " 3 ! ( ! !0 " "%.1 3 3 !/ ! $! " %

# . ! ( ! ! %! !% " ! ! #. ! ! .B# !% !0 " -

P! ( ! ! ! $ %1 ! ! .B# !% ! "%. %

( %0 (! - 0! # HB# / !0 " 1 !0! H!.

" ! $ 0!$! % $ !$! Simgame " % #! %! !%- $ !%" B

. (. ! ! 0!1 $ 0! ( ! ! ! !% 3 ! ! 0!

$" ( $! . % - " ! "% ! 3 ! ( $! $ . 3 H %

# ! #" ! (! ! ! . ( $ ( $" ! (-

0! I"! ! %% ! !% 3 L3 ! ( 3 . %" !0 "

( $ ( $" NM P. ( I"! $ !% ! ! ( ! K ( K 3 .- !

1 %! ( ( !0 " "%. ! 0 . %! ! %! $ . !/ " 1 % # 0!

! ! %!% !0 " "%.M " ! @ ! $ H. ' & % 1 5,,7J & " !#! $1 % " '

=! ( 1 *+++A- ! 1 ! !0 " ( $ ( $" 1 3! ! !0 "

! !%" !%" ! ( $ 1 ! ! .B ! !% ! ! $ .

! #! ! - ! ! $! % ! !0 " M !

$ ! " %! % ( ! 3 % 3 . ! ( 3 ( $ ( $" ! (

! 0 $! -

#

%! 1 - - ' 3 1 - @5,,*A- " 0!. $! % " !% !0 " ( "%!

! $ ! #" ! $" -Simulation & Gaming1231 4,+B4,9-

H 1 <-?- @5,74A- 0 %! ! $ ! ! " ! $ ! % "$" 0!

!0!$! - <-?- H ' <- . @ % A-Achievement and performance@ - :9,B45+A-

? ( 1 ? -

P! ! 1 - @5,94A- ! B " ( !0 !3 ! ! ! (- - % @ %-A1

Using what we know about teaching- !/ % 1 - & -

P $1 P- &- @5,78A-Human Characteristics and School Learning- !3 H 3B -

(22)

P .1 <- - ' %1 - - @5,98A- ! ! #! 0 % "%! !0!$! - - - ? H

@ %-A1 %# H ! ! ! ( @: % !%-1 - :*9B:7DA- !3 H $ -

P 3 1 <- &-1 1 - ' "(" %1 - @5,9,A- & " !% ( % ! " " ! ! (- Educational Researcher1181 :*B4*-

P " ! 1 -1 P! ! 1 - % !.1 - @5,,9A- ( $! % ! ( %! !- <- ! ' - <! H 1 @ % -A1From Barbie to Mortal combat: gender and computer games1 7*B9,-

" ! ! -

1 - @*++DA- !/ 1 ( $ ( ! " ! 1 % (! %! !% ! ! ! ! -Simulation & Gaming1361 484B49*-

! $ H.1 - ' & % 1 ?- - @ % -A @5,,7A-Evaluation for the 21st century. A handbook.

" % H & (!-

! 1 - - @5,,+A-Theory driven evaluations- !3#" . H & (!-

! 1 - - @*++DA-Practical program evaluation- " % H & (!-

! 1 - - ' 1 - - @5,9:A- 0 " ( 3 ! ! ! ! .B% 0! -Evaluation

Review171 *9:B:+*-

! 1 - - @5,,4A- ! " " ( ! $ % %" 0! $ ( " -Review of Educational Research1641 5B:D-

# 1 - <-1 $# 1 &- -1 #" 1 &- -1 ! 1 - -1 H1 - -1 1 - -1

? H! 1 - - ' ?! ! 1 &- &- @5,9+A-Toward reform of program evaluation.&

< !.BP -

# 1 - ' & 31 - @5,77A-Aptitudes and instructional methods: A handbook for research on interactions- !3 H 0 ( -

0 % 1 - <- @*+++A- ! ( " . ! .B# !% !0 " -(ew Directions for Evaluation1871 DB5:-

1 - @*++*A- ! I" Q - = >! ! 1 $! % ! V#! !(" (! " % ! !H 0!

WR" . $ ! ( % ! (- ! 1 $! % ( %! %

! ! 0! X-Unterrichtswissenschaft1 5,7B*5*-

% 1 &- - ' ! 1 - - @*++:A- ! .B% 0! !0 " ! $ Y*+

$ !3 %! ? H % ! 0!-Evaluation and Program Planning1261 :DDB:88- 1 - &- @5,9,A- & $" ( $! ! $ ! ! !% ( ( ! -Teaching Sociology1 171 5B59-

1 - <- @*++5A- ! ( ( " ! #" ! $" 2( $ ( ! ! # ! .- Simulation & Gaming1321 ,7B55+-

(23)

! ! 1 - - ' 1 - - @*++*A- " $" !0 " -Simulation &

Gaming1331 4*DB44+-

!% 1 - -1 1 - ' ! !1 - ?- @*++5A- $!3 H %! ( ( % !0 " (

0 " !$ -European Journal of Education1361 5D7B574-

1 -1 ! 1 - ' H! 1 <- - @*++*A- $! 1 $ 0 1 % ! ( ! ! %

! $ %! -Simulation & Gaming1331 445B487-

! 1 <- ' ? #" 1 <- @*++4A- !0 !3 ! ( !/ ! ! ! (

! ! 0! ! -Simulation & Gaming135 @*A1 *7+B*,:-

!! # 1 - &- @5,9,A- / ! % ( ! (! !/ ! ! !- - H ' - & " %! @ % -A1 Communication and simulation: From two fields to one theme@ - *8,B*9:A- !0! % 1 ( %

" (" ! -

!! 31 -&-1 ' ?.$ 1 - - @5,7:A- ! ! ( ! ! 0! ! ( $! !( ! #" !

" ! -Simulation & Games, 4 @:A *D,B*,4-

! $H!1 - ' ?! ! 1 - - @5,,7A- P!% (" ( H ! " ! ! " (! W ! ! $

! $ !X- Enzyklopädie der Psychologie. Band D, Praxisgebiete, Serie 1, Pädagogi sche Psychologie [Encyclopedia of psychology, Vol. educational psychology]@ - 75B578A-

Z (! 1 ! $ . ( ! !-

! ! .1 - ' !! #! (1 <- @5,,,A- P ( ( (! ! %! ( ! 0! ! !

" ( " " !I" $ %! (-American Journal of Evaluation1201 475B49+-

! !1 <-1 % 1 - ' Q ! 1 - @*++5A- # !$ ! ! ! ! ! - ? "$ %! ! $

!"! !% ! $! ! $" ! ! $ !"! !% ! W # !$ ! !% ! (-

? . ! ( 3 !3 $!% #! $ ! " ( 3 !3 $!% X-Computer und

Unterricht111 @4A1 88B55-

1 <- - ' !1 - - @*++*A- !# ! ( ! -Simulation & Gaming133 @4A1 4,+BD+:-

% !1 =- @*++*A- ( " %!% ! . ! ( ! ! ! !" " ( $" ( $! - Simulation & Gaming1331 *:8B*45-

/1 <- @5,,DA-Applied Multilevel Analysis- $ ! % $ B "# H ! -

"!# ! 1 - - @*+++A- ! .B# !% !0 " ( !% " %! % ( #! 3!!

% !0 " -(ew Directions for Evaluation1871 7,B9,-

=!. 1 <-P- ' ? !1 <- @5,,+A- ! ! $ (!$! ( $! % $" !%" %

! ! -Yearly Review of Management, 16 @*A1 :+7B::8-

= #1 - - @5,94A-Experiential learning: Experience as the source of learning and development- ( !3 % 1 < ! ! -

(24)

= >1 ?- - @*++:A- ! ( ! ! 0! ! 0! ! ( ! 0 $! "( ( $ (

$" %! ( -Simulation & Gaming1341 557B5:4-

!!"31 - - @*++:A- ! " ( ( $ ! ! - ! % 0 # ! % # !$ #!

0!%-American Journal of Evaluation1241 DB*+-

H1 - -1 $ 1 - - ' ! % 1 - &- @5,,*A- ! ( ! ! ! .B% 0!

!0 " @ ! A $ %! ( - - - ! ' - - @ % -A1Using theory

to improve program and policy evaluations@ - 75B94A- !3 H !! 3 % ! -

!H " 1 - ' & ! ! !1 - @5,,8A- $ B " % $ 0 . ( ! P!% (" (! %!

! ! " (! W $ % $ 0 %! ! $ ! ( ! $ !X- - - ?! B

! @ %-A1Psychologie des Lernens und der Instruktion@ - 5D:B59+A- Z (! 1 ! $ . ( ! !-

! ! 1 - - - ' ! 1 - - - @*++4A- $ ! $ %! %!# ! (

$" ( $! -Simulation & Gaming135 @5A1 7+B94-

! 1 - @*+++A- ? ! ! % 3 .N ! ! #! ! "% ( ( $ ! .

!0 " $! B . -(ew directions for evaluation1871 D,B8,-

1 <- - ' - - @5,,,A- ? " H 3 !%(! ! ! !%" N <- - ' - - @ % -A1Advances in Teacher Education (Vol. 5)- & %1 # !/-

!$" 1 ?- - @5,95A- / ! $! %! ( .> ( % ( $! 1 !0! ! #!

$! % % ! ! ( % !/ ! $! -Simulation and Games1121 :B54-

!. % 1 - <- @5,,9A- $ . ( $ !0 " $! % ! ( ! ( "

! ! !-American Journal of Evaluation1191 *+:B**5-

(! 1 - <-1 ! 1 -1 "!# ! 1 - - ' 1 - - @*+++A- ( $ ! . !0 "

!1 $ !1 % # !$ -(ew directions for evaluation1871 DB5:-

1 -1 !.1 - ?- ' !!$ 1 - @*++4A-Evaluation. A systematic approach. " % H & (!-

& $ 1 - ' #! 1 - @5,9,A- ? ! ! $ % " ! 3 . !. "( - International Journal of Educational Research1131 9,B,,-

& 0! 1 - @5,7:A- B !! !0 " - - - " ! @ %-A1School evaluation: The politics and process@ - :5,B:*9A- P! H! !.1 " -

& % 1 ?- -1 H1 - - ' !0 1 - - @5,,5A-Foundations of program evaluation.

Theories of practice. !3#" . H & (!-

& 0 1 - - @5,,+A-Cooperative learning: Theory, research, and practice- ( !3 % 1

! !B -

& 0 1 - - @5,,4A-Educational Psychology. Theory and Practice- P . % P -

(25)

& ! 1 <- - @*+++A- &. !$ %. $ % ! 0! ! ( ! 0 $! ! ! !%

% $ ! " " !-Simulation & Gaming1311 D*9BD:D-

& ! 1 - <-1 ! ! 1 - -1 & " !1 - - ' $ H1 - @*++5A- %! %

$" ! ( $ !/ % $ ( " % ( $ ! $! %

!0 " -Computers in Human Behaviour1141 D57BD4D-

& $!1 - @*++4A- ! .B# !% !0 " % . ! $ !/ .-Evaluation1101 D9B78-

& " !#! $1 - -1 % " - - ' =! ( - @ % -A @*+++A-Evaluation models viewpoints on educational and human services evaluation.P = "3! -

( F" 1 &- @5,,*A- ( ( $! %!# ! (-Simulation & Gaming123@*A1 585B57:-

?- =- =! (( " % - @*++4 A-W.K. Kellogg Foundation evaluation handbook.P ! !!H1

?-=- =! (( " % -

?- =- =! (( " % - @*++4#A-W.K. Kellogg Foundation logic model development guide.

P ! !!H1 ?-=- =! (( " % -

? (1 - -1 ! ! 1 - -1 ' ? #! (1 - <- @5,,:A- 3 % H 3 !%(! # ! ! (-Review of Educational Research1631 *4,[*,4-

?! 1 - - @5,,7A- ! .B# !% !0 " 1 ! ! 1 % " " !-(ew Directions for Evaluation1761 45BD8-

?! 1 - - @*+++A- ? H 3 ! ! 3! !0 " !N(ew directions for evaluation1871 :DB4D-

? !.1 <-1 &- @5,7,A-Evaluation. Promise and performance.? ( ! # " !-

? !1 <- @5,,+A- ! !0 " $ " ! B# !% #" ! ( $ (- <- ?- ! . @ %-A1Guide to business gaming and experiential learning@ - *7,B:++A- % = ( (!-

? !1 <- @5,,7A- ! ! ! 0! ! #" ! ( $! !( $ (!$! " ! 3 H- Simulation & Gaming1281 :8+B:78-

? !1 <-1 ' H 1 - @5,,9A- !0! ( ! H 3 !%(! $" 2( $ (- Simulation & Gaming1291 7B5,-

? ( 1 &- -1 1 &- - ' & %! 1 ?- - @5,,7A- ! ! % $ !/ ! ! "%!

!0!$! $ ! ! !0 " -Journal of Personnel Evaluation in Education1 111 D7B87-

$ H . 1 &-1 ! $ 1 - -1 ! %!>1 - ' = 1 - @*++4A- ( ! $ (

%!0! ( $ %! % " ! ( $ ! . ! !/ $ !-American Journal of Evaluation1251 5,5B*+7-

(26)

Jan Hense, PhD (University of Munich, Germany), is assistant professor of education and psychology at the Ludwig Maximilians University Munich (Germany). His research interests include evaluation theory and research, entrepreneurship education, and innovative learning methods with new media. He is an active member of the Swiss Austrian German Simulation and Gaming Association.

Willy Christian Kriz, PhD (University of Vienna, Austria), Professor of Human Resources Management and Gaming Simulation Methods at the University of Applied Sciences Vorarlberg (Austria), Guest Professor at Universities in Austria, Germany, Italy, Peru and India. Chairman of the Swiss Austrian German Simulation and Gaming Association, Past President of the International Simulation and Gaming Association.

Joseph Wolfe received his PhD from (ew York University’s Stern School of Business. He is a professor emeritus at the University of Tulsa, and is president of Experiential

Adventures LLC as well as a fellow and a past president of the Association for Business Simulation and Experiential Learning (ABSEL).

Address: JH: Ludwig Maximilians University, Department Psychology, Leopoldstr. 13, D 80802 Munich, Germany;

tel.: +49 (89) 2180 3257; fax: +49 (89) 2180 99 3257;

e mail: [email protected] muenchen.de

Address: WCK: University of Applied Sciences Vorarlberg, Hochschulstrasse 1, 6850 Dornbirn, Austria;

tel.: +43 5572 792 3218; fax: +43 5572 792 9509;

e mail: [email protected]

Address: JW: Experiential Adventures LLC,

500 West Roy Street W208, Seattle, WA 98119, USA;

telephone: +1 206 829 8028;

e mail: [email protected]

(27)

+++++++++++++++++++++++++++++++++++++++

(" ! 5

Simgame . ( # % @ ! $ (" (! 0! A

(" ! *

& " ! Simgame ( $ ! .

(28)

(29)

(" ! :

Simgame ( $ ! . @ (! $ %! A

!0 " H 3 !%(!

!0 " !/ ! ! !

$ 0 2

!/ ! !

(!

(! %!

( !/ ! ! !

3 ( $ (

$"

$ 0

!/ ! !

!

$

! I" . ( $!2 $"

I" .

% ^&

"%! B "%!

! I" .

"%! B ! !

"

% !

$!B B H

! !

%!# ! (

%

! . 0 0!$!

0! B ! (!%

" %! B ! (!%

&

( 0!

@H 3 !%(!A

$ 0

%

" ! 0

" !

'

$ !$!

! ( $!

% !$

! $! %

!0 " H 3 !%(!

!0 " !/ ! ! !

$ 0 2

!/ ! !

(!

(! %!

( !/ ! ! !

3 ( $ (

$"

$ 0

!/ ! !

!

$

! I" . ( $!2 $"

I" .

% ^&

"%! B "%!

! I" .

"%! B ! !

"

% !

$!B B H

! !

%!# ! (

%

! . 0 0!$!

0! B ! (!%

" %! B ! (!%

&

( 0!

@H 3 !%(!A

$ 0

%

" ! 0

" !

'

$ !$!

! ( $!

% !$

! $! %

(30)

# ! 5

0! 0 !3 " % ! !

Sourcek α exemplary item / (operationalization) Input – students

!0 " H 3 !%(! & B 5 4+ - - @H 3 !%(! ! ! ! A

0 2 / ! ! & B 5 4 -78 K !/ ! ! $. ! ! #" ! "% ! K

! %! & B 5 5 - - @5S$ !1 *S !$ !A

Input – teachers

( !/ ! ! ! B 5 * -94 @ "$#! $" ( $! .!%2 !%A

! ! $ 0 B 5 D -9+ K $ H ( 3 % . ( & $( $! K

/ ! !% # !$ B 5 4 -9* K !/ ! # !$ ( > ( ! K

! B 5 5 - - @ " ! ! ( & $( $!A

Process – individual learning

! . 0 0!$! & B * : -74 K " ( ( $! . H !% ! ! ! ( $!K

0! B ! (!% & B * 5 - - K . ( & $( $! !/ ! !% $" $ $!K

%! B ! (!% & B * 5 - - K . ( & $( $! !/ ! !% ! $ $!K Process interaction w/game

$!B B H B * 5 - - @ " ! . ( ! ( $!A

$! ! ! B * 4 -79 K ! $! " !% (! ( " K

!# ! ( B * 5 - - @ " ! 3 %!# ! (A

Process – social learning

! I" . & B * 4 -7, K 3 H!% ! ! 0! . 3 ! "%! K

! ! " & B * 4 -9: K . ! ! !% & $( $! 3! K

! ! " & B * * -7* K . ! ! !/ !% %" ( ! ( $!K

Outcome – student learning

( 0! ! ( & B * 4+ - - @H 3 !%(! ! ! A

( 0! ! ( & B * 5* -,5 K& $( $! ! !% $. H 3 !%(! ! ( (" ! K

( 0! ! ( B * 5* -97 K& $( $! ! !% "%! M H 3 !%(! ! ( (" ! K

& ! ( & B * : -7: K& $( $! ! !% $. ! $3 H H K

& ! ( B * : -9+ K& $( $! ! !% "%! M ! $3 H H K

0 ! ( & B * : -9: K& $( $! !% $. ! ! ! $ K

0 ! ( B * : -9+ K& $( $! !% "%! M ! ! ! $ K

Outcome acceptance

! ! "%! & B * 4 -79 K 3 " % H! . & $( $! ( K

! ! ! ! B * D -77 K 3 ! $$! % & $( $! ! ! ! K

(otes &S& "%! I"! !J S ! ! I"! !J 5S#! ! . (J *S ! . (kS "$#!

!$ JαS # M @ # !A

(31)

# ! *

( 0! ! ( ! ! @ S4+,A

x SD T p d’

! ! ! 3! 9-,7 *-:5 8-D* -++ +-48

! ! 3! ,-9D *-99

! ! ! 3! 4-D* 5-:D B5-,* -+: B+-5:

! ! 3! 4-:, 5-:9

! ! ! 4-44 *-99 8-++ -++ +-4*

! ! D-48 :-D8

(otes !B !% B ! %! ! %! $ ! J % S4+9Jd’S ! M d %! ! %! $ !

# ! :

&"#F! 0! ! ( ! ! % ! ! !% #. "%! % ! !

( x SD

Student ratings

( 0! ! ( 4*+ :-D5 -77

& ! ( 4+8 :-D4 -,4

0 ! ( 4*5 :-*7 -,8

! ! :,D :-74 5-*:

Teacher ratings

( 0! ! ( :D :-9, -8+

& ! ( :4 4-+, -D7

0 ! ( :: 4-+* -9+

! ! *4 4-44 -D:

(otes ! ! (! x 5S ! ( ! ! J DS (! ! ( ! !

(32)

# ! 4

!( ! . !% ( 0! ! ( @ S*+7A

B SE B β p

@ ! ! A -,7 -,5 -*,

Input factors – students

!0 " H 3 !%(! -D, -+8 -D:\\ -++

/ ! ! 2 0 B-55 -** B-+: -85

! %! 5-5+ -:8 -5D\\ -++

Input factors – teachers

( !/ ! ! ! 5-D+ -*+ -4*\\ -++

/ ! !% # !$ B-8, -*8 B-58\ -+5

! ! $ 0 5-58 -*, -:+\\ -++

! B-+: -+4 B-+D -44

Process factors – individual learning

! . 0 0!$! B-85 -*4 B-54\ -+5

0! B ! (!% B-5D -59 B-+4 -45

%! B ! (!% -5D -57 -+4 -4+

Process factors – interaction with game

$!B B H -+4 -+: -+9 -5:

$! ! ! 5-,+ -:7 -D4\\ -++

!# ! ( -45 -*4 -+, -+9

Process factors – social learning

! I" . -8+ -*: -54\ -+5

! ! " -** -** -+D -:*

! ! " B-5+ -*+ B-+: -8*

(ote Rcorr2 S -D8JF@5815,+A S 57-*D1pU -+5J \ U-+D1 \\ U-+5