Optimal Strategies in Turn-Based Stochastic Tail Games

HAL Id: hal-00370046

https://hal.archives-ouvertes.fr/hal-00370046

Preprint submitted on 23 Mar 2009

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

Optimal Strategies in Turn-Based Stochastic Tail Games

Florian Horn

To cite this version:

Turn-Based Sto hasti Tail Games

⋆

Florian Horn f.horn wi.nl

CentrumWiskunde&Informati a Amsterdam,TheNetherlands

Abstra t. Innitesto hasti gamesareanaturalmodelforopen rea -tive pro esses: one playerrepresentsthe ontroller,and the other rep-resentsahostile environment.Theevolution ofthe system dependson the de isionsof the players, supplementedby a randomfun tion. The problemsonsu hgames anbesortedintwo ategories:thequalitative analysisponderswhetheraplayer anwinwithprobabilityone(or arbi-trarily losetoone),while thequantitativeanalysisis on ernedabout themaximal(orsupremal)valueaplayer ana hieve.

Inthis paper,weestablishthe existen eofoptimal strategiesingames whosethewinning onditiondoesnotdependonniteprexes.Wealso presentageneralpro eduretoderivequantitativeresultsfromqualitative algorithms.It also follows from the orre tness of this pro edure that optimalstrategiesarenomore omplexthanalmost-surestrategies.

1 Introdu tion

There is a long tradition of using innite games to model open rea tive pro esses [BL69 ,PR89 ℄. The system is represented as a game arena, i.e. a graph whose verti es belong either to Eve ( ontroller), Adam (non-deterministi environment), orRandom(sto hasti evolution). Thegame isplayedbymovingatokenonthearena:whenitisinoneofEve'sverti es, she hooses its next lo ation among thesu essors ofthe urrent vertex; when it is in one of Adam's verti es, he hooses its next lo ation; when it is in a random vertex, its next lo ation is de ided by a xed random fun tion.Playing thegamefor

ω

movesresults ina playofthegame,i.e. aninnitepathofthegraph.Thespe i ationofthesystemisrepresented bya(Borel) subsetofthe possibleplays,thewinning ondition. Evewins a playifitbelongs to thewinning ondition, andAdamwins otherwise.

In this paper, we fo us on tail onditions, where the winner of a play doesnot depend on nite prexes. Our mainmotivationis thattail

⋆

This work was arried out during the tenure of an ERCIM Alain Bensoussan FellowshipProgramme,andwasalsopartiallysupportedbythefren hANRAveriss

regular games. From a veri ation perspe tive, tail onditions also or-respond to ases where lo al glit hes are tolerated inthe beginning of a run, aslong asthe spe i ationis metinthelimit,e.g. inself-stabilising proto ols. Finally, one of the most popular payo fun tions in e onomi games,themean-payofun tion,isatail ondition.Duetola kofspa e, many proofs are sket hed or omitted. Complete proofs an be found in the third hapterof [Hor08℄.

Outline of the paper. Se tion2 re allsthe lassi alnotions about sim-plesto hasti games.In Se tion 3,we showthat thedierent qualitative riteriaareequivalentinniteturn-basedsto hasti tailgames,anddene a new notion of qualitative determina y. Se tion 4 takes on the quanti-tative problems, and shows how a qualitative algorithm an be used to ompute thevalues of a nite turn-based sto hasti tail game. The exis-ten eofoptimalstrategies forboth playersinniteturn-basedsto hasti tail games also follows from the proofs, as well asthe fa t that optimal strategies arenomore omplex than almost-sure strategies.

2 Denitions

Were allhereseveral lassi alnotionsaboutsimplesto hasti games,and refer thereaderto [GTW02℄ and [dA97 ℄for moredetails.

Arenas and plays A simple sto hasti arena

A

is a dire ted graph

(Q, T )

without deadlo ks, whose verti es are partitioned between Eve's verti es (

Q

E

, represented as

#

's), Adam's verti es (

Q

A

, represented as

2

's), and random verti es (

Q

R

, represented as

△

's), and supplemented by a fun tion

δ

: Q

R

→ D(Q)

, whi h is the random law dire ting the hoi e of su essors inthe random verti es: so

δ(r)(q) > 0 ⇔ (r, q) ∈ T

. Asub-arena

A

|B

of

A

istherestri tionof

A

to asubset

B

of

Q

su hthat ea h ontrolledvertexof

B

hasasu essor in

B

,andallthesu essorsof random verti es in

B

belongto

B

.A play

ρ

of

A

isan (possiblyinnite) path inthe graph

(Q, T )

.The set of innite plays is denoted by

Ω

, and the set ofinniteplays starting inthe vertex

q

is denotedby

Ω

q

.

Strategies and measures A pure strategy

σ

for Eve is a deterministi wayofextendingnite playsendinginavertexof Eve:

σ

: Q

∗

_Q

E

→ Q

is su hthat

(q, σ(wq)) ∈ T

.Strategies analso bedened asstrategies with memory. Given a (possibly innite) set of memory states

M

, a strategy

σ

n

: (Q

E

×M ) → Q

andamemory-update fun tion

σ

u

: (Q

E

×M ) → M

. Noti e that any strategy an be represented as a strategy with memory

Q

∗

.A play

ρ

is onsistent witha strategy

σ

if and only if

∀i, ρ

i

∈ Q

E

⇒

ρ

i+1

= σ(ρ

0

, . . . , ρ

i

)

.Thesetofplays onsistent with

σ

isdenotedby

Ω

σ

. On e an initial vertex

q

and two strategies

σ

and

τ

have been xed,

Ω

q

σ,τ

an naturally be made into a measurable spa e

(Ω

σ,τ

q

,

O)

, where

O

isthe

σ

-eldgeneratedbythe ones

{O

w

| w ∈ Q

∗

_}

:

ρ

∈ O

w

ifandonlyif

w

isa prexof

ρ

.The probability measure

P

σ,τ

q

isre ursively dened by:

∀r ∈ Q, P

σ,τ

q

(O

r

) =

 1

if

r

= q ,

0

if

r

6= q ;

∀w ∈ Q

∗

,

(r, s) ∈ Q

2

, P

σ,τ

q

(O

wrs

) =







P

σ,τ

q

(O

wr

) · 1

σ(wr)=s

if

r

∈ Q

E

,

P

σ,τ

q

(O

wr

) · 1

τ(wr)=s

if

r

∈ Q

A

,

P

σ,τ

q

(O

wr

) · δ(r)(s)

if

r

∈ Q

R

.

Winning onditions and values A winning ondition

Φ

is a Borel set of

(Ω

σ,τ

q

,

O)

. An innite play is winning for Eve if it belongs to

Φ

, and winning for Adam otherwise. Finite plays arenot winning for either player. A winning ondition

Φ

is a tail ondition if thewinner of a play doesnot dependon niteprexes:

∀w ∈ Q

∗

_,

_{∀ρ ∈ Q}

ω

_{, ρ}

_{∈ Φ ⇔ wρ ∈ Φ}

. The value of

q

∈ Q

with respe t to the strategies

σ

and

τ

for Eve and Adam (or

{σ, τ }

-value) is dened by:

v

σ,τ

(q) = P

σ,τ

q

(Φ)

. The value of

q

with respe t to a strategy

σ

for Eve (or

σ

-value) is the inmum of its

{σ, τ }

-values:

v

σ

(q) = inf

τ

v

σ,τ

(q)

. Symmetri ally, the value of

q

with respe ttoastrategy

τ

forAdam(or

τ

-value)isthesupremumofits

{σ, τ }

-values:

v

τ

(q) = sup

σ

v

σ,τ

(q)

.Bythequantitativedetermina yofBla kwell games [Mar98 ℄, the supremum of the

σ

-values is equal to the inmum of the

τ

-values. This ommon valueis alledthevalue of

q

.

Winning riteria A strategy

σ

for Eve is almost-surely winning (or almost-sure) from a vertex

q

if and only if the

σ

-value of

q

is one. It is positively winning (or positive) from

q

if and only iffor any strategy

τ

, the

{σ, τ }

-valueof

q

ispositive(noti ethatthe

σ

-valueof

q

maybezero). The almost-sure region of Eve (resp. positive region of Eve) isthe set of verti es fromwhi h Evehasan almost-sure(resp. positive) strategy.The limit-sureregionofEve(resp.boundedregionsofEve)isthesetofverti es with value one (resp. withpositive value).In general, the limit-sure and almost-sure riteriaaredierent,asarethepositiveandbounded riteria.

Intailgames,itisalwayspossibleforbothplayerstodisregardthehistory of a play, and onsider that the urrent vertex isthe initial one. We an thususestrategytranslationstoderivethevalueofavertexfromitsowner and the valueofits su essors:

∀q ∈ Q

E

, v(q) = max{v(s) | (q, s) ∈ T }

∀q ∈ Q

A

, v(q) = min{v(s) | (q, s) ∈ T }

(1)

∀q ∈ Q

R

, v(q) =

X

(q,s)∈T

δ(q)(s) · v(s)

Thisisverysimilarto the aseof rea habilitygames, where su h sys-tems an dire tly be usedto ompute the values.Yet, there are two im-portantdieren es:ingeneral,tailgames donot featureatargetvertex, whosevalueisknowntobeone;andthereisnonotionofstoppinggames, where (1)hasauniquesolution.Inordertoestablishourresults,weneed thus to onsiderthemore omplex notionof

σ

-valueof anite play:

Denition 1. The

σ

-value of a nite play

w

onsistent with

σ

is the inmum of the

{σ, τ }

-values under the assumption that

w

is a prex of the play:

v

σ

(w) = inf

τ

P

σ,τ

w

0

(Φ | ρ

0

= w

0

, ρ

1

= w

1

, . . .) .

Using the

σ

-values of the prexes, we an observe how theprospe ts of the players evolve during a play. In parti ular, for any positive real number

η

,wedenetheevent

L

σ

η

, orrespondingtotheplayswhereEve's han esof winninghave dropped below

η

at some point:

L

σ

_η

= {∃i, v

σ

(ρ

0

. . . ρ

i

) ≤ η} .

This event has two interesting hara teristi s: rst, if the

σ

-value of the initial vertexis greaterthan

η

,the probability that theensuing play belongs to

L

σ

η

is bounded away from one (Proposition 2); se ond, the probabilitythat Adamwins iszerooutsideof

L

σ

η

(Proposition 3).

Proposition 2. Let

q

be avertex of

Q

,

σ

and

τ

bestrategies forEveand Adam, and

η < ν

≤ v

σ

(q)

be two positive real numbers. We have:

P

σ,τ

q

(L

σ

η

) ≤

1 − ν

1 − η

.

Proposition 3. Let

q

be a vertex of

Q

,

τ

be a strategy for Adam, and

η

be a positive real number. We have:

P

σ,τ

q

(Φ | ¬L

σ

η

) = 1 .

Thesetworesultssuggestawaytoimprove

σ

withareset pro edure with respe t to a given real number

η

.Assume that Eve plays

σ

and,at some point,the

σ

-valueof theprex drops below

η

,while the

σ

-value of the urrent vertexis greaterthan

η

.She an improve her han es to win byforgettingthepast, and restartplaying

σ

asiftheplayjuststarted. Denition 4. The strategy

σ

resetwithrespe t to

η

, denoted by

σ

↓η

, isa strategy withmemory, whose memory statesare the plays of

A

onsistent with

σ

.Its memory-update andnext-move fun tionare denedas follows:

σ

n

↓η

(w, q) =

 σ(q)

if

v

σ

(wq) ≤ η ∧ v

σ

(q) > η

σ(wq)

otherwise

σ

u

↓η

(w, q) =

 q

if

v

σ

(wq) ≤ η ∧ v

σ

(q) > η

wq

otherwise

If Eve plays a ording to

σ

↓η

, it follows from Proposition 2 that the number ofresets inthe ensuing playis nite withprobabilityone. Thus, byProposition 3,eitherthe

σ

-value ofthe visitedverti es is onsistently below

η

,or Evewinswithprobabilityone. We an usereset strategiesto exposeseveral links between thedierent notions ofwinning regions.

Theorem 5. In any nite turn-based sto hasti tail game, Eve has an almost-sure winningstrategy in the region withvalue one, and Adam has an almost-sure strategy inthe region withvalue zero.

Sket h of proof.Letus prove theresultfor Eve.Asthearenaisnite,we an hoose

η

and

ν

su hthat

0 < η < ν < 1

,and anyvertexwhose value islessthan oneisalsolessthan

η

.Now, if

σ

hasvalue

ν

fromtheverti es withvalueone,

σ

↓η

is almost-sure forEvefrom theseverti es.



Theorem 5states that thelimit and almost-sure winning riteria are equivalent in nite turn-based sto hasti tail games. It follows dire tly that the positive and bounded winning riteria are alsoequivalent. Using the standard redu tionto paritygames, these results an beextendedto nite simple sto hasti

ω

-regular games. However, Theorem 5 does not holdforgameswith ontext-free onditions,innitearenas,or on urrent moves: in ea h of the three games of Figure 1, the value of the initial vertexis one,yetEve hasno almost-sure strategy.

a

b b

W = a

n

_b

n

⊚

(a)Context-free ondition

1

|1

0

_|0

0

|1

1

|0

W = Reach ⊚

(b)Con urrentmoves

· · ·

· · ·

.2 .8 .2 .8 .2 .8

W = ¬ Reach ⊗

( )Innitearena

Fig.1.Limit-sureisnotalmost-sure

We an also use reset strategies dire tly to derive a positive-almost propertyforniteturn-basedsto hasti tailgames,extendingChatterjee's bounded-limitpropertyfor nite on urrent tail games [Cha07a℄

1 :

Theorem 6 (Positive-almostproperty).Inanyniteturn-based sto has-ti tail game, if Eve has a positive strategy from every vertex, she has an almost-sure strategy fromevery vertex. If she has a positive strategy from at least one vertex, she has an almost-sure strategy fromat least one ver-tex. The same holdsfor Adam.

Sket h ofproof.Intheproofoftheuniversal partofTheorem6 forEve, the tri k is to hoose

η

and

ν

between zero and the lowest value for a vertexinthegame (byTheorem5,all verti eshavepositivevalue).On e again,if

σ

hasvalue

ν

ontheverti eswithvalueone,

σ

↓η

isanalmost-sure strategy for Eve.The otherstatementsfollowbyduality.



1

Itis alled a positive-limit propertyin thepaper, butrelies onthe existen eof a vertexwithpositivevalue:aboundedvertex,a ordingto[dAH00℄'staxonomy.

this proof still holds in themore general ase of nite on urrent games whose winning ondition is sux- losed. However, Theorem 6 itself does not: Figure 1 again provides ounter-examples. But we ould derive an existentialpositive-limitpropertyforEveandauniversalbounded-almost property forAdaminthese games:

Claim 7. Inanynite on urrent gamewithasux- losedwinning on-dition,if Evehasa positive strategyfromatleast onevertex,thenthere is at least one vertex with value one. If no vertex has value one, Adam has an almost-sure strategy from every vertex.

Last,but not leastof ourtripty h isTheorem8,whi h extends quan-titative determina y inprex-independent games:

Theorem 8 (Qualitativedetermina y).Inanyniteturn-based sto has-ti tail game, from any vertex, either Eve has an almost-sure strategy or Adam has a positive strategy, and vi e versa.

Proof. Theorem 8 follows dire tly from Theorem 5 and the quantitative

determina y ofBla kwell games [Mar98 ℄.

⊓

⊔

By ontrast with Theorem 5, we did not nd any ounter-example for natural extensions of Theorem 8. In parti ular, the three games of Figure1 arequalitatively determined.

4 Values and optimal strategies

The algorithms omputing thevalues of simple sto hasti tail games are oftenadaptations ofalgorithmsfor rea habilitygames whi huse qualita-tivealgorithmsasora les.Forexample,one anguessasolutionto(1)and useaqualitativealgorithmto he kne essaryandsu ient onditionson thevalueregions:see[CdAH05℄forRabinandStreettgames,[Cha07b℄for Mullergames,and[CHH08 ℄fornitarygames.Itisalsopossibletoadapt thestrategyimprovementalgorithmof[HK66℄whenoneoftheplayershas positionalstrategies: see[CJH04 ℄forparity,and[CH06 ℄forRabingames. Finally,inone-player sto hasti tail games (MarkovDe ision Pro esses), one an ompute rst thealmost-sure region,and thenthe values of the rea habilitygame to thisregion [Cha07a ℄.

Inthisse tion,weshowhowour permutationalgorithmsfrom[GH08℄ an be automati ally modied to solve any nite turn-based sto hasti

proofs). The main idea isthat ifAdamdoesnot make obvious mistakes, Eve an only hope to win by rea hing her almost-sure region. This an onlybedonethroughrandomverti es:thereisnovertexofEveleadingto it (it wouldbelong to thealmost-sure region),and she annothope that Adamwillenteritvoluntarily(thatwouldbeanobviousmistake).We an thus onsider the winning ondition asa tool for Eve to ensure thatthe tokenrea hesthebestpossiblerandomvertex:ifAdamrefusesto omply, heloses.Thebehaviourofbothplayersisthendeterminedbytheir prefer-en esovertherandomverti es.Furthermore,itissu ientto onsiderthe ases whereEve andAdamshare thesame estimationovertherespe tive qualityofrandomverti es,i.e.,whentheirpreferen esareopposed.These preferen es arerepresentedbypermutations overtherandom verti es.In theremainderofthepaper,apermutation

π

designatesapermutationover the

k

random verti es, su h that

{π

1

, . . . , π

k

} = Q

R

. We often onsider thesinkandtargetverti esasrandomverti esinpermutation-based on- epts, with the impli it assumption that they arerespe tively thelowest and greatestverti es:

π

0

= ⊗

and

π

k+1

= ⊚

.

For simpli ity (and e ien y), we rst normalise the games we on-sider:we omputethealmost-sureregionsofbothplayers,andmergeea h of them into a singlesink vertex (

⊗

for Adam,

⊚

for Eve). The winning ondition is modied a ordingly: a play that rea hes

⊗

is winning for Adamand aplaythat rea hes

⊚

is winningfor Eve.

Adam'salmost-sureregion

Eve'salmost-sureregion

(a)Originalgame (b)Normalisedgame

Fig.2.Gamenormalisation

In ea h of our algorithms, the atomi loop onsiders a permutation

for ea h vertex,thebest(with respe tto

π

) randomvertexthat Eve an ensure to rea h. We do so with the help of a qualitative ora le, whi h omputesembedded almost-sure regions forEve:

Denition 9. Let

G = (A, Φ)

be anormalisedniteturn-based sto hasti tail game, and

π

be a permutation over the

k

random verti es of

A

. The

π

-regions of

G

are dened asfollows:



W

π

[k + 1] = {⊚}

;

 for any

1 ≤ i ≤ k

,

W

π

[i]

is the almost-sure region of Eve in

A

with respe t to the obje tive

Φ

∪ Reach(∪

j≥i

{π

j

})

, minus

∪

j>i

W

π

[j]

; 

W

π

[0] = {⊗}

.

Noti e that a random vertex

π

i

may belong to a region

W

π

[j]

with

i < j

(but not

i > j

). In this ase, theregion

W

π

[i]

is empty. On e the

π

-regionshave been omputed,we derivefromthema MarkovChain

G

π

, with

k

+ 2

verti es numbered

0 . . . k + 1

: for any

i, j

∈ [0, k + 1]

, the probabilityofgoingfrom

π

i

to

π

j

isequaltotheprobabilityofgoingfrom

π

i

to

W

π

[j]

in

G

.We denoteby

v

π

[i]

thevalueof

π

i

in

G

π

.

Weusetwologi alrelationstode ideaboutthepertinen eofa permu-tation.Self- onsisten y isamost natural ondition,asitsimplyexpresses the adequation between a priori preferen es,and resulting values:

Denition 10. Let

G

be a nite turn-based sto hasti tail game with

k

random verti es. A permutation

π

over

Q

R

is self- onsistent if for any

1 ≤ i ≤ j ≤ k

,

v

π

[i] ≤ v

π

[j]

.

Itiseasytoderiveasolutionto(1)fromaself- onsistentpermutation. However, ingeneral,thereismorethanone solutiontothissystem,sowe need another property,that we dubliveness:

Denition 11. Let

G

be a nite turn-based sto hasti tail game with

k

random verti es. A permutation

π

over

Q

R

is live if for any

1 ≤ i ≤ k

,

δ(π

i

)(∪

j>i

W

π

[j]) > 0

.

Itmayseemthatthisnotionisalready apturedbyself- onsisten y,as itis abad idea for Eve to sendthe token to a randomvertexthat does not verify the internal property. However, the hoi e of thepermutation alsoee tsAdam'sbehaviour[Mur07 ℄,andtheremaybespuriousnon-live self- onsistent permutations.

Livenessandself- onsisten yareusedinastraightforwardway.Inany nite turn-based sto hasti tail game, there is a live and self- onsistent permutation.Moreover,ifapermutation

π

isliveandself- onsistent,then

for anyvertex

q

in

Q

,

q

∈ W

π

[i] ⇒ v(q) = v

π

[i]

.It istheneasy to derive algorithms omputingthevalues froma qualitative algorithm:

Theorem 12. Let

C

be a lass of nite turn-based sto hasti tail games. If the almost-sure region of Eve in a

C

-game

G

an be omputed in time

t(|G|)

, then the values of any

C

-game

G

an be omputed in time

|Q

R

+

1|! · t(|G|)

.

Sket h of proof. For any given permutation

π

,the

π

-regions an be om-puted in time

|Q

R

| · t(|G|)

. We an then de ide whether

π

is live and self- onsistent inlineartime. We mayneed to do soforea h of the

|Q

R

|!

permutation, leading toa worse-time omplexityof

|Q

R

+ 1|! · t(|G|))

.



Theorem 13. Let

C

be a lass of nite turn-based sto hasti tail games. If theproblem of omputingthealmost-sure regionsof

C

-gamesbelongs to the omplexity lass

K

, then thequantitative problems of

C

-gamesbelongs to the lasses NP

K

and o-NP

K

.

Sket h of proof. Instead of sear hing exhaustively for a live and self- onsistent permutation, we an guess itnon-deterministi ally, and he k thatit is orre tinlineartime with

|Q

R

|

allsto a

K

-ora le.



Aninterestingby-produ toftheproofisthatthe

π

-strategiesderived froma live and self- onsistent permutation areoptimal:

Theorem 14. In any nite turn-based sto hasti tailgame, both players have optimal strategies.

As the parity a eptan e ondition, whi h an be used to represent any

ω

-regular language, is a tail ondition, Theorem 14 also yields an alternative proof of the existen e of optimal strategies in nite simple sto hasti

ω

-regular games [dAH00℄.

It analsobenotedthatEve'sstrategy isdened asaspatial ompo-sitionofresiduallyalmost-sure strategies,anddoesnot usemorememory than its omponents:

Theorem 15. Let

C

be a lass of nite turn-based sto hasti tail games. IfEvehasalmost-surestrategies withmemory

Υ

in

C

-games,thenshe also has optimal strategies with memory

Υ

in

C

-games.

Note that Theorem 15 does not hold when the winning ondition is not a tail ondition: seetheweak paritygame ofFigure 3.

2

3 .5

.5

Fig.3.Optimalstrategiesrequirememoryinweakparitygames

Inthisgame,thevalueoftheinitialvertexis

1

2

.Indeed,ifEvesendsthe token on eto theleft and thenalways to theright, thelowest o urring olour has equal han es to be

1

or

2

. However, this value annot be a hieved bymeans of apositionalstrategy:

 ifthereisapositiveprobabilitytosendthetokentotheleft,thelowest o urring olour is almostsurely

1

;

 otherwise, thelowesto urring olour issurely

3

.

Both players have positional almost-sure strategies in weak parity games [GZ05 ℄. Optimal strategies for weak parity games with

d

olours mayrequire up to

d

− 1

memorystates.

5 Con lusion

We provedthe existen eofoptimal strategiesforboth playersinallnite turn-based sto hasti tail games. This also yields an alternative proof forthe existen eofoptimalstrategiesinnitesimplesto hasti

ω

-regular games[dAH00℄.Furthermore,wepresentedasinglepro edureto ompute the values of nite turn-based sto hasti tail games, provided that we already have a qualitative algorithm. The ost ofthis pro edure iseither a

|Q

R

|!

fa tor,oranon-deterministi guess,generalisingseveralresultson the omplexityofquantitativeproblems.On eagain, theseresults anbe used to ompute thevalues of

ω

-regular games, although it is ne essary to rst redu ethem to equivalent paritygames.

The existen e of optimal strategies is very sensitive to ea h of our hypotheses, as demonstrated by Figure 1. However, the qualitative de-termina y mayholdinmore generalsettingsIt wouldalsobeinteresting to lookfor a pro edure to ompute thevalues of simplesto hasti games witharbitrarywinning onditions,and/or innitearenas.

[BL69℄ J. R. Bü hi and L. H. Landweber. Solving Sequential Conditions by Finite-StateStrategies.Transa tionsoftheAmeri anMathemati alSo iety, 138:295311, 1969.

[CdAH05℄ K. Chatterjee, L. de Alfaro, and T. A. Henzinger. The Complexity of Sto hasti RabinandStreettGames. InPro eedings ofICALP'05,volume 3580 ofLe ture NotesinComputer S ien e,pages878890.Springer,2005. [CH06℄ K. Chatterjeeand T. A.Henzinger. StrategyImprovementfor Sto hasti Rabin andStreettGames. InPro eedings ofCONCUR'06,volume4137 of Le ture NotesinComputer S ien e,pages375389.Springer,2006. [Cha07a℄ K.Chatterjee. Con urrentGameswithTail Obje tives. Theoreti al

Com-puter S ien e,388(12):181198, 2007.

[Cha07b℄ K.Chatterjee. Sto hasti MüllerGamesarePSPACE-Complete. In Pro eed-ingsof FSTTCS'07,volume4855ofLNCS,pages436448.Springer,2007. [CHH08℄ K.Chatterjee, T.A.Henzinger, andF.Horn. Sto hasti FinitaryGames.

Te hni alReport2008002,LIAFA,CNRSUMR7089,2008.

[CJH04℄ K.Chatterjee,M. Jurdzi«ski,andT. A.Henzinger. Quantitative Sto has-ti Parity Games. In Pro eedings of SODA'04, pages 121130. So ietyfor IndustrialandAppliedMathemati s,2004.

[dA97℄ L. de Alfaro. Formal Veri ation of Probabilisti Systems. PhD thesis, StanfordUniversity,1997.

[dAH00℄ L. deAlfaro and Thomas A. Henzinger. Con urrent

ω

-regularGames. In Pro eedingsofLICS'00,pages141154.IEEEComputerSo iety,2000. [GH08℄ H. Gimbert and F. Horn. Simple Sto hasti Games with Few Random

Verti esare EasytoSolve. InPro eedings ofFoSSaCS'08, volume4962 of LNCS,pages519.Springer,2008.

[GTW02℄ E.Grädel,W.Thomas,andT.Wilke,editors. Automata,Logi s,and In-niteGames:AGuidetoCurrentResear h[out omeofaDagstuhlseminar, February 2001℄,volume2500ofLNCS.Springer,2002.

[GZ05℄ H.GimbertandW.Zielonka.GamesWhereYouCanPlayOptimally With-out AnyMemory. InPro eedings of CONCUR'05,volume 3653 of LNCS, pages428442.Springer,2005.

[HK66℄ A. J. Homan and R. M. Karp. On Nonterminating Sto hasti Games. ManagementS ien e,12(5):359370,1966.

[Hor08℄ F. Horn. Random Games. PhD thesis, Université Paris 7 and RWTH Aa hen,2008.

[Mar98℄ D. A.Martin. TheDetermina yofBla kwellGames. Journal ofSymboli Logi ,63(4):15651581, 1998.

[Mur07℄ R.Muroe. Ee tanEe t. XKCD326,2007.

[PR89℄ A. Pnueli and R. Rosner. On the Synthesis of a Rea tive Module. In Pro eedingsofPOPL'89,pages179190,1989.