Modélisation et création d'un joueur virtuel de billard intelligent à l'aide de techniques d'optimisation et de planification

MODELISATION ET CREATION D'UN JOUEUR VIRTUEL

DE BILLARD INTELLIGENT A L'AIDE DE TECHNIQUES

D'OPTIMISATION ET DE PLANIFICATION

par

Jean-Frangois Landry

Memoire presente au Departement d'informatique

en vue de l'obtention de la maitrise en informatique

FACULTE DES SCIENCES

H SHERBROOKE

1*1

Archives Canada

Published Heritage

Branch

Archives Canada

Direction du

Patrimoine de I'edition

Your file Votre reference ISBN: 978-0-494-49523-0 Our file Notre reference ISBN: 978-0-494-49523-0

NOTICE:

The author has granted a

non-exclusive license allowing Library

and Archives Canada to reproduce,

publish, archive, preserve, conserve,

communicate to the public by

telecommunication or on the Internet,

loan, distribute and sell theses

worldwide, for commercial or

non-commercial purposes, in microform,

paper, electronic and/or any other

formats.

AVIS:

L'auteur a accorde une licence non exclusive

permettant a la Bibliotheque et Archives

Canada de reproduire, publier, archiver,

sauvegarder, conserver, transmettre au public

par telecommunication ou par Plntemet, prefer,

distribuer et vendre des theses partout dans

le monde, a des fins commerciales ou autres,

sur support microforme, papier, electronique

et/ou autres formats.

The author retains copyright

ownership and moral rights in

this thesis. Neither the thesis

nor substantial extracts from it

may be printed or otherwise

reproduced without the author's

permission.

L'auteur conserve la propriete du droit d'auteur

et des droits moraux qui protege cette these.

Ni la these ni des extraits substantiels de

celle-ci ne doivent etre imprimes ou autrement

reproduits sans son autorisation.

In compliance with the Canadian

Privacy Act some supporting

forms may have been removed

from this thesis.

Conformement a la loi canadienne

sur la protection de la vie privee,

quelques formulaires secondaires

ont ete enleves de cette these.

While these forms may be included

in the document page count,

their removal does not represent

any loss of content from the

thesis.

Canada

Bien que ces formulaires

aient inclus dans la pagination,

il n'y aura aucun contenu manquant.

L e i 4 Janvier 2008

lejury a accepte le memoire de M. Jean-Frangois Landry dans sa version finale.

Membres dujury

M. Jean-Pierre Dussault

Directeur

Departement d'informatique

M. Richard Egli

Codirecteur

Departement d'informatique

M. Abdelhamid Benchakroun

Membre

Departement d'informatique

M. Froduald Kabanza

President-rapporteur

Departement d'informatique

Dans ce memoire, nous proposons une approche pour la creation d'un joueur de billard virtuel intelligent. Les elements physiques du jeu de billard sont tout d'abord decrits afin d'illustrer la complexite du probleme et un modele est introduit permettant l'utilisation de techniques d'optimisations pour arriver a une solution.

L'aspect technique du billard est tout d'abord observe et une approche pour creer un joueur capable de se repositionner n'importe ou sur la table est proposee. L'aspect tactique du jeu est ensuite aborde a l'aide de differentes heuristiques pour determiner la difficulte d'un coup et la selection de la cible ou se repositionner. Des resultats sont demontres pour supporter les heuristiques proposees.

Finalement, plusieurs modifications sont effectuees sur le modele original pour arriver a un joueur final beaucoup plus performant. Le maximum de connaissances sont extraites de la table de billard pour arriver a effectuer une recherche intelligente dans les coups possibles. Ces modifications permettent de doubler la vitesse de l'optimisation en utilisant des points de departs beaucoup plus performants. Plusieurs tests sont generes pour demontrer les gains de performance avec ces nouveaux ajouts.

Je desire tout d'abord remercier Jean-Pierre Dussault, directeur de ces travaux sans qui la creation de ce memoire n'aurait pas ete possible. Non seulement m'a-t-il transmis sa passion pour la complexite du jeu qu'est le billard, mais plus important encore son amour pour la recherche et le developpement.

Merci a mes parents, qui ont toujours ete presents et n'ont jamais cesses de me supporter au cours de mes longues annees d'etudes.

Salutations a Mike Smith, Will Leckie et Marc Godard pour la bonne compagnie en Italie et Taiwan, ainsi que la motivation a toujours produire un joueur plus performant arm de les affronter en competition.

Remerciements a Micheal Greenspan pour l'organisation des deux premiers tournois de billard virtuel ainsi que pour son hospitalite lors de notre court sejour a Kingston.

1 Introduction 1 2 Optimisation d'un joueur de billard virtuel - techniques 3

2.1 Cue ball positioning 8 2.1.1 Aiming the cue ball at a target 8

2.1.2 Aiming the cue ball after a collision 10

2.2 Shot parameters 15 2.3 Solving the optimization models 16

2.3.1 Optimization algorithms 17

2.4 Extensions 17 2.4.1 Breaking clusters 17

2.4.2 Using the rail for positioning 18 2.4.3 Using other balls for positioning 18

3 Optimization d'un joueur de billard virtuel - tactiques 21

3.1 Context of computer pool 26

3.2 Shot difficulty 27 3.2.1 Direct shots 27 3.2.2 Kick/Bank Shots 29

3.2.3 Combinations 30

3.3 Global table difficulty 30 3.4 Best and worst spots on the table 32

3.5 Optimization with ball speed considerations 36 3.5.1 Sinking object ball with minimum speed 36

TABLE DES MATIERES

3.5.2 Cue ball scratching 37 3.5.3 Breaking clusters 37 3.6 Improved PoolMaster 38

3.6.1 Shot difficulty 39 3.6.2 Choice of the global table coefficient 39

3.6.3 PoolMaster Old vs PoolMaster — Improved vs Simple Player . . . . 42

4 Optimisation d'un joueur de billard virtuel complet 45

4.1 Billiard table physics 50 4.1.1 Shot parameters 52 4.2 Objective function 53 4.3 Shot difficulty 55

4.3.1 Direct shot difficulty coefficient 56 4.3.2 Indirect shot difficulty for shots ranking 57

4.3.3 Repositioning target selection 57

4.4 Noise consideration 59 4.5 Pre-computed starting points 60

4.5.1 Pre-computed tables for normal shots 60 4.5.2 Pre-computed tables to break clusters and improve table state . . . . 61

4.5.3 Choosing the right cut angle 64

4.6 Complete algorithm 65

4.7 Results 67 5 Conclusion 73

2.1 Two cases where the target is attained 10 2.2 Two cases where the target is not reachable 11

2.3 Attainable regions 13 2.4 Two unreachable targets 14 2.5 The effect of collision induced throw 16

3.1 Difficulty coefficient estimated with cut angle and distance 29

3.2 Difficulty coefficient represented by a, 29 3.3 Problems with the exact difficulty coefficient 29

3.4 Kick shot using north rail 31 3.5 Outline of the best regions using sum measure 33

3.6 Outline of the best regions using average measure 34 3.7 Outline of the best regions using maximum measure 35 3.8 Shot success percentage using added noise (harder shots) 40 3.9 Shot success percentage using added noise (easier shots) 41 4.1 Difficulty coefficient estimated with cut angle and distance 56

4.2 Kick shot using north rail 58 4.3 Outline of the best regions for cue ball positioning 59

4.4 Possible repositionning targets for side shot 62 4.5 Possible repositionning targets for corner shot 63

4.6 Lower cut angle 65 4.7 Upper cut angle 65

3.1 Optim vs 2d zone scan 42 3.2 Zones Max vs Optim Max (small noise) 42

3.3 PoolMaster Old vs PoolMaster Improved 43 3.4 Simple Player vs PoolMaster Improved 43 4.1 Normal shot statistics (13 balls randomly placed on table) 69

4.2 Safety shot statistics (13 balls randomly placed on table) 69 4.3 Normal shot statistics (9 balls randomly placed on table) 69 4.4 Safety shot statistics (9 balls randomly placed on table) 69 4.5 Normal shot statistics (5 balls randomly placed on table) 70 4.6 Safety shot statistics (5 balls randomly placed on table) 70

Introduction

Le billard est un sport tres complexe. Un bon joueur doit non seulement etre capable d'empocher la boule choisie avec une tres grande precision, mais il doit aussi pouvoir se repositionner autant bien que possible pour faire son prochain coup. II doit tenir compte de la friction sur la table, de la transmission du spin entre les boules et de plusieurs autres effets de la physique du billard.

Bien des travaux ont ete effectues au cours des dernieres annees pour decrire de fagon concise l'aspect mecanique du billard. Cependant, ce n'est que recemment que des recherches ont ete orientees en direction de la creation d'une intelligence artificielle pour faire jouer un robot sur une vraie table de billard. Le jeu de billard ressemble en plusieurs aspects a celui des echecs. On doit judicieusement choisir le coup a faire tout en prenant bien soin d'analyser egalement le jeu de l'adversaire. Cependant, aux echecs, on possede un nombre fini (meme si tres grand) de possibilites de coups alors qu'au billard, ce nombre est infini. Cet aspect de continuite est ce qui promet un tres grand potentiel pour l'utilisation de techniques d'optimisations a la resolution du probleme.

Dans un premier temps, nous affrontons l'aspect technique du jeu, c'est a dire la capacite de definir les parametres necessaires decrivant la fagon dont le joueur va frapper la blanche

avec la queue de billard p o u r empocher la boule visee et bien se repositionner. D a n s u n

deuxieme temps, nous attaquons l'aspect technique du jeu, ce qui consiste en la definition des heuristiques dictant la fagon dont le joueur selectionne ses coups. Finalement nous allons faire un rappel sur les principes utilises et proposer quelques ameliorations pour obtenir une meilleure performance.

C H A P I T R E 1. I N T R O D U C T I O N

Organisation du memoire

Ce travail consiste en la presentation de trois articles sur la creation d'un joueur de billard virtuel intelligent. Une courte introduction precede chaque article et fait la mise en scene. Tout d'abord le premier article propose l'approche de l'optimisation pour resoudre l'aspect technique du billard. Ensuite le second article poursuit cette idee en ajoutant quelques mo-difications au modele et en completant les informations manquantes comme la selection des coups et des cibles a viser. Finalement, le troisieme article fait un rappel complet sur les techniques proposees et adresse certains problemes comme la selection de bons points de departs pour l'optimisation. Une courte conclusion fait enfin une synthese du probleme et de Pefficacite de la solution proposee, ainsi que de possibles travaux futurs. La bibliographie de chaque article a ete jointe en une plus grande bibliographie commune a la fin du memoire afin de minimiser la redondance.

Optimisation d'un joueur de billard

virtuel - aspects techniques

R e s u m e

Cet article represente la preuve de concept que Poptimisation peut en effet etre utilisee avec succes pour la creation d'un joueur de billard virtuel intelligent. On observe tout d'abord les elements qui contribuent a la complexity de ce probleme et en font un tres bon candidat pour la resolution par minimisation de fonction. On y explique les equations decrivant le mouvement d'une boule de billard sur une table, pour ensuite construire une fonction objectif qui sera minimisee. Cette fonction correspond a la position finale de la blanche apres avoir empoche une boule visee sous certaines contraintes de vitesse et de spin pour rester dans le domaine de la realite. Quelques graphes sont generes afin de demontrer l'efficacite de la methode.

C o m m e n t a i r e s

Dans le cadre de ce travail, j'ai contribue au developpement du modele utilise pour l'optimisation. Essentiellement je me suis charge d'appliquer les principes d'opti-misation au probleme pour obtenir des resultats prouvant l'efficacite de la methode en creant un joueur de billard virtuel complet. N'ayant pas de librairie prefabriquee pour la simulation physique d'une table de billard, il m'a fallu en creer une partielle

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R D E BILLARD VIRTUEL - T E C H N I Q U E S

a l'aide de Scilab afin de pouvoir obtenir quelques tests. J'ai genere l'ensemble des graphiques dans Particle pour illustrer les resultats et contribue a ecrire et reviser certaines parties du texte en proche collaboration avec l'auteur principal Jean-Pierre Dussault. L'article fut accepte et presente dans le cadre de la conference ICGA 2005, Taiwan pour etre ensuite publie dans "Lecture Notes in Computer Science" [7]. Les principes de cet article furent mis a l'epreuve dans le cadre de la competition lO*'1 Computer Olympiads pour s'emparer de la 2e position.

Jean-Pierre Dussault and Jean-Frangois Landry

Sherbrooke, Quebec, Canada J1K 2R1

[Jean-Pierre.DussaultI Jean-Francois.Landry2]SUSherbrooke.CA Keywords: Computer pool, optimization.

Abstract

We develop optimization principles to produce a computer pool player. A good player has technical and planning abilities. Technically, he sinks balls with preci-sion, and controls the position of the cue ball after the shot. He uses his technical abilities to devise a game plan, sinking the balls in a winning order.

We propose to use optimization techniques to simulate an excellent player. In this paper, we focus on the technical abilities. We provide optimization models to compute the shots to not only sink a given ball, but bring the cue ball at a specified target. Some hints on planning optimization strategies are given.

This paper presents a key computation to develop an optimal billiard player, namely the skill to sink a ball while achieving good position for the next one. This is a very basic skill. Actually, in the game sometimes named carambol (Billard Frangais, three cushions), the only aim is to hit the two balls with the cue ball, thus the cue ball positioning is most important in this game. For this paper, we limit ourselves to optimizing one shot to sink an object ball by hitting it with the cue ball, and reach and stay on a given target point on the table. Although the problem has to be addressed to make an artificial player, to our knowledge, no reference describe any solution.

In order to develop our optimization model, we need to understand deeply the behavior of balls rolling on a cloth stretched on a table, the effect of the collision between two balls, and also the effect of hitting the cue ball with the leather tip of the cue stick. The optimization model we develop will benefit from any improvement on the physical model. As a physical model, we use a sufficiently complete model to illustrate the potential of our optimization modeling approach. We base our physical model on many sources.

We construct a quite general model by first exposing simplified situations made progres-sively more and more realistic. We first address the simple situation to prescribe the initial speed and spin to the cue ball in order that it reaches a given target, given a shoot direction. This allows to introduce a model of ball movement largely inspired by [22], but those ideas are common, and explained for example in [23, 17, 1, 11].

Next, we consider the situation in which the cue ball is targeted to a collision point, hit an object ball in a prescribed direction, and rebounds toward a target. This introduces simple elastic collisions without ball-ball friction, whose treatment comes from [11].

In a third step, we consider adding ball-ball friction. This time, since the ball-ball friction modifies the trajectory of the object ball, the collision target is no longer known in advance, but must be optimized. Our treatment of ball-ball friction comes from [23], but similar

developments are t o b e found in [1].

Finally (for this paper!), we address the problem of optimizing an actual shot, identifying the speed, direction of the cue stick and the contact point on the cue ball. This is simply a change of variables since angular speed of the cue ball is expressed in terms of the shot parameters. For this part, we assume that the ball is struck with a level cue stick (i.e. no

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R D E BILLARD VIRTUEL - T E C H N I Q U E S

masse) which imposes a natural limit on the magnitude of the spin with respect to the speed. We will discuss aspects of the solution of our models, and point out many stimulating perspectives.

2.1 Cue ball positioning

Two aspects of accuracy are important for the pool player: accurately sink the object ball, and accurately position the cue ball. Computing the target ensuring to sink the object ball is trivial (but nevertheless difficult to perform for an amateur player), so we will concentrate on the computation of the shot in order to position the cue ball on a specified target, or at least as close as possible to the target.

After hitting the object ball, the cue ball is left with some linear (vo) and angular (u>o) velocity. Throughout the paper, vectors will be denoted by boldface letters (like v0)), their norm by italic letters (vo = ||v0||) or regular symbols (u0 = ||u>o||) and finally the normalized

vectors will inherit a hat (vo = —).

We will first present an optimization formulation to compute suitable velocities in order for the cue ball to reach, and stop on a given target. Since the cue ball was shot by the player, we will then present another optimization problem to compute the angular and linear velocities required to have the cue ball hit the object ball suitably to sink it, and to reach and stop at the target. Finally, we will translate the initial angular and linear velocities required to a specification of the cue tip contact point on the cue ball, and the speed of the cue tip.

2.1.1 Aiming t h e cue ball at a target

In this section, we simply compute the initial velocities required to bring the cue ball from some point (the origin) to a specified target.

T h e motion of a ball is known t o b e first of a sliding n a t u r e , a n d after some point, become

of a pure rolling nature.

Let the table be in the plane xy, and the vector e3 = (0 0 1)* point upward the table. The sliding movement is characterized by a non vanishing relative velocity of the contact point of the ball with the cloth, i.e. 0 ^ vr(£) = v(£) + Re3 x cj(t). Then, we may assume

that the frictional force is constant and given by the relation f = —mg/j,svr(Q). We denote

by / the inertia tensor for the sphere, / = 2m^ , m is the ball's mass, R its radius, and g

is the gravitational constant. Also, /xs is the sliding friction coefficient, ~ 0.2. We may now deduce the equation for the linear and angular velocities

v(i) = v(0) - giisvr(0)t (2-1)

and

w(t) = w(0) - ^ j ^ t , (2.2) Similarly, the linear and angular positions are given by

p(i) = p(0) + v(0)i - ^ t 2 (2.3)

and

a(i) = a(0) + w(0)t - "~->'—Re3 x ft\ (2.4) 2

Using those developments, we may deduce that the relative speed will become null at a time

TV = ^-vr(0), (2.5)

after which the ball will be in a rolling motion. In rolling motion, the force is f = —mgiJ,rv

and the friction coefficient //r ~ 0.01. Since the side spin does not influence trajectories, we may consider the two first components of u?, so that the angular quantities depend on the linear ones, u(t) = | e3 x v(i); (2.6) v(i) = v ( rr) - 5//rvr(f - rr) , (2.7) p(*) = P(rr) + v ( rr) ( t - rr) - ^ ( t - rrf (2.8) and a(t) = a(rr) + ^ e3 x (v(rr)(t - rr) - ^ ( t - rr)2. (2.9) Moreover, we deduce that the ball will stop at a time rf = ^ ^ .

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - T E C H N I Q U E S

Figure 2.1: Two cases where the target is attained, using strong spin in the left figure. The curve represents the trajectory while the red lines are a sampling of the spin velocity u>. Back spin is showed by the lines on the right of the curve while natural roll is on the left.

in order for the cue ball to reach and stop at the target s. We assume that v(0) = a d0 where do is an imposed initial direction of movement. The decision variables for our optimization problem are then a and UJI,U2 and U3. Let x = (a u\ u>2 W3)*. The developments above allow to define a function F(x) which computes P ( T / ) , and we simply solve

m i n - | | F ( x ) — s|

x 2

As defined, the solution always yields exactly the target. Of course, the initial velocities (particularly angular) are not realistic, but adding lower (I) and upper (u) bound constraints of the form I < x < u helps ruling out unrealistic solutions; then, the target is not always reachable, for instance if situated backward (with respect to do) and far, but then, the solution brings the cue ball as close as possible given the bounds on x.

2.1.2 Aiming t h e cue ball after a collision

In this section, we set up an optimization problem to take into account that the cue ball must reach a target collision point c (collision target), then obey the collision laws, and finally try to reach and stop at s (stop target).

/ / /

mmm

mmm

w

V\\y

_{^\ \ y}

\y

Figure 2.2: Two cases where the target is not reachable because of spin restrictions.

The collision calculations are simplified by the fact that the object ball is at rest when the player executes his shot. We will first develop the model assuming no friction between the balls, and then generalize the model to the realistic case.

Frictionless and elastic ball-ball collisions.

For frictionless ball-ball collisions, the target point c is easy to compute. Suppose the object ball is at o, and the target in the pocket at b . Then, the desired direction for the object ball is 8 = b — o, and the target c = o — 2RS. Since the angular speed may affect the trajectory, we now leave the vector do (previously imposed) free to the optimization process; we choose to use the vector vo as the optimization variable. From the developments above, there must be a time value t (depending on vo and u?o) f°r which p(i) = c. From this point, we define a second trajectory denoted with barred quantities: we set v0 = <&(v(t)),

t h e resulting direction for t h e cue ball after t h e collision, p0 = p(i), &o = tt,(*)i **o = aW

-We then compute, as in section 2.1.1, p(f/). Our decision variables are now v0, u>0 and the

unknown time t to reach the impact point. Since ado = vo> we thus solve

min ±(||p(t) - c||a + ||p(7» - s||a) v0,u>0,t L

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R D E BILLARD VIRTUEL - T E C H N I Q U E S

In the above formulation, one should be aware that the first term, \\p(t) — c||2 is actually a

constraint, i.e. must vanish for the computation to make any sense. If the solution is such

that p(f) is not close enough to c, its weight in the objective function should be increased. To compute the velocity of the cue ball after the impact, we may define the function $ as follows: v(i)x6v - v(t)y5x V°x = S^TTl °V (2-U> ux ~ "y V ( * ) A ~ v(t)ySx s /9 1o N v

°* = sf^l *- (

-

)

L i n e a r s h o t s . In most shots, the cue stick is held horizontally, and thus does not impart curvilinear motion to the cue ball. In such cases, ct>o -L do and we know in advance the direction do; the decision variable then reduces to the scalar a. This simplified setting allows to actually compute the value t, so that the optimization problem is reduced in dimension, as well as in difficulty.

Since we have p(i) as our target, we can transform the position equations (2.3 and 2.8) to get a quadratic equation of the form ax2 + b x + c :

In sliding friction: 0 = - p ( t ) + p(0) + v(0)t - ^ e (2.14) where: In rolling friction: c = - p ( * ) + p(0) (2.15) b = v(0) (2.16) a = - ^ (2.17) 0 = -p(<) + p ( rr) + v(rP)(t - rr) - 9-^{t - rr)2 (2.18) where: c = - p ( t ) + p ( rr) (2.19) b = v ( rr) (2.20)

-150 -100 -50 \ J E J- + i _ J.+HH++*4-" 50 v_p ,+TOO 1)50 _{.;r +} T v J -+ -I . 250 300 + -H++|: +

Figure 2.3: We illustrate the attainable region (sampled by pluses), the target reached (the black diamond) by the cue ball (red broken line) using the optimization, and the path of the object ball (yellow line); on the left, close to a 45° cut aimed at the top left pocket and on the right, an almost direct shot to the side. The blue rectangle represents the table.

which are then solved with:

g/j,r\ * = 2

- b ± Vb

We then select the smallest of the two results for our time (or the positive one if a value is jO). Of course, the sliding/rolling state has to be determined first by calculating rr and

applying the corresponding equation.

Ball-ball collision w i t h friction.

As it happens, there exists a small friction coefficient between two balls, imparting to the velocities of the balls after the collision an effect named collision-induced throw. As explained

in [23], t h i s effect is slight, changing t h e trajectories of 2-4 degrees, b u t m u s t b e t a k e n care

of, in particular for long shots since a 4 degree angle over 8 feet results in a deviation of 6.7 inches, more than enough to miss the shot.

In order to take this effect into account, we must add a new component in the objective function. Let p be the trajectory of the object ball, and b the target position in the pocket.

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R D E BILLARD VIRTUEL - T E C H N I Q U E S -150 -100

*4"f***T

Figure 2.4: Two unreachable targets: the optimization does it's best, and get as close as possible to the target.

At some still unspecified time i, one must have p(i) RS b . Moreover, the cue ball is no longer aimed at a specific target, but must impact the object ball in such a way that it reaches the pocket, c now represents the center of the object ball and p(t) — c is no more constant, but subject to optimization. The objective function becomes

min_ - ( | | | | p ( t ) - c||2 - 4 * RY + \\p(ff) ~ s||2 + ||p(f) - b |

vo,u)o,c,t,t ^ 2)

(2.23)

Here again, the first term, slightly more complicated than before, is a constraint. It expresses the fact that the cue ball will actually touch (hit) the object ball at time t. Since the so called 90° rule is not exactly obeyed to under realistic friction and elasticity assumptions, the exact contact point is no longer known in advance, but depends on the actual speed and spin of the cue ball at the impact. In practice, the contact point will be close to the contact point as used before, and should be used as an initial guess to the optimization algorithm.

The last term \\p(t) — b | |2 also represents a constraint, and could be modeled differently. As a term to minimize, one seeks to reach as close as possible the center of the pocket. One could also specify bounds, required to be satisfied in order to sink the object ball.

given by the formulae

vox = v ( * )x- vt e (2.24)

v02/ = v(t)y - voj, (2.25)

(2.26) As mentioned above, we must now take into account the collision-induced throw. As it happens, all we have to do is to add a (small) correction to the velocity vectors v0 and vo. The correction is a component in the direction v0 and obeys the law Vby = cos(j)/j,bbVo, where

Hbb is the ball-ball friction, a is the cut angle and

(**») +

* # )

cos(7) = * ^ - T 7 , . (2.27)

(sin(a) + M ) +(c o s ( a) « ) )

1/2-We then correct the after collision velocities by adding ^ v to v0 and subtracting it from

vo-Side spin The side spin of the cue ball affects the throw. The simple rolling-sliding model

presented in section 2.1.1 does not propagate side spin in its form since its equations only depend on the force components in the z = 0 plane. On the other hand, the UJZ component

may well persist after the cue ball is stopped in extreme situations, as one observes on real tables. We must add a damping to simulate the attenuation of the side spin.

2.2 Shot parameters

We consider a parameter transformation from v0 the velocity of the cue stick and h0, the hit

point on t h e cue ball t o v0,o>0- We assume t h a t t h e cue stick is horizontal. In t h i s context,

there is only a combination of back—top spin and side spin.

Following [28], let h0 = (0 hy hz) represent the point hit on the cue ball. After a suitable

change of variables so that x = v0, y = e3x x and z = e3, the angular velocity vector will be orthogonal to hyy + hzz and its magnitude will be | ^ . Moreover, in order to avoid miscue,

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - T E C H N I Q U E S I ^ ^ ^ H ^ - ^ ™ 250 + 300 + Tfr x+ x + x + 1 ^ : + 200 "ScT x$"f x+

Figure 2.5: The effect of collision induced throw: on the left, we see the throw trajectory (blue line) of the object ball without correction of the target, on the right the modified attainable region when correcting the target. The effect of throw is slight and has a negligeable effect on the repositioning of the cue ball (sampled by crosses), but is important for the sinking of the object ball in the pocket.

we must impose a constraint ||h0|| < f.

2.3 Solving t h e optimization models

We first observe that the optimization models we developed are mostly unconstrained prob-lems. We simply include bounds on the parameters, which reflect reality: there is an actual limit on the speed a pool ball can reach! Similarly, we just saw that the cue stick has to hit the ball within half of its radius in order to safely avoid miscue. We model the point hit by the cue using an angle (the spin direction) and a scalar (the amount of spin), the amount being bounded by ^.

In order to solve those optimization efficiently, we will benefit from some knowledge of the game to provide a good initial approximation, and bounds on the parameters to be satisfied. It is possible to compute in advance a minimum speed required by the cue ball in order to reach the object ball, and in order that the object ball actually reaches the pocket.

Also, we will first solve the simpler model neglecting the ball-ball friction, and only after solve the general model, taking as initial guess the simpler solution, to refine the solution

to take into account the collision-induced throw, and eventually the corrections due to an inelastic ball-ball collision.

In all the cases where the cue stick is horizontal, we know that the cue ball trajectory is linear, so we may express Vo = ado f°r a value of do either constant (no ball-ball friction), or expressed in function of other parameters (c if the cue ball is at the origin). Moreover, since in the case of no ball-ball friction, the side spin has no effect at all, we may reduce the number of decision variables to only 3, namely a, i, and hy the amount of back—top spin.

When adding the ball-ball friction, the collision target c, the side spin hz and the time i are

added to the decision variables.

2.3.1 Optimization algorithms

The resulting optimization models may be solved using many optimization methods. The objective functions are smooth, but their derivatives are difficult to express. We produced the graphics using Scilab [21], which provides a function optim to optimize a function subject to bound constraints. The method it uses is a limited memory quasi-Newton iterative algorithm, and we used finite differences to compute the gradient values.

As mentioned above, some terms in the objective function are actually penalization of constraints. It could be preferable to use some constrained optimization method, but the simple penalty approach we described seems to work fine in practice.

We will incorporate those computations within a player. We then use the open source O P T + + [18] optimization software. Again, we chose a quasi-Newton iterative method to treat the minimization subject to bound constraints.

2.4 Extensions

We have presented the basics of an optimization methodology to achieve good position play. We discuss now extensions to make the methodology really powerful.

2.4.1 Breaking clusters

Often in actual play, the position play consists in breaking clusters of balls. Our optimization models adapt easily by specifying a target position (the cluster) together with a speed range;

C H A P I T R E 2. O P T I M I S A T I O N D ' U N J O U E U R D E BILLARD VIRTUEL - T E C H N I Q U E S

above, the speed range was null, i.e. we required the cue ball to stop on the target.

2.4.2 Using t h e rail for positioning

As it was apparent in the figures, the reachable region extends largely outside the table. In such cases, a simplified model assuming perfect collisions between the ball and the rail would allow to simply aim the cue ball at the mirror target. However, the energy loss in the collision with the cushion is as large as 70%, so that an accurate position play must take into account the friction and inelasticity of the collision.

Building on the above model, the cue ball's trajectory after the collision with the object ball will be separated into n + 1 parts; the first part reaches a first cushion, the second one a second one, and so on. The presented model treats the special case n — 0.

2.4.3 Using o t h e r balls for positioning

Instead of considering the other balls on the table as added difficulties, we may well use them to achieve positioning by having the cue ball bouncing on them. Again, it consists in adding pieces to the path of the cue ball.

Conclusion

We have presented the fundamentals of an optimization methodology to give a virtual pool player good sinking and positioning abilities. This constitutes the basic skill on which a good player builds his game.

Still discussing technical skills, we will have to add to our models the presence of other objects (other balls, cushions) which may influence the optimization and the possibility to break a cluster of balls.

We anticipate to build a function estimating the difficulty of a shot. This difficulty

coefficient should t a k e into account t h e difficulty t o sink t h e object ball, b u t also t o reach a

point close to the stop target. Moreover, even if the stop target is not well approached, may be the next ball remains "easy", and thus the shot should not be discarded. A planner should consider all the remaining balls on the table. Probably a dynamic programming model can prove useful for this planning.

Of course, we will like to add spectacular shots, such as masses, and our optimization model can already deal with various angular velocities, it remains to have a model to convert the actual shot (cue stick angle and force, point to hit on the cue ball) into the appropriate VQ, OJQ initial conditions of the cue ball.

Optimization d'un joueur de billard

virtuel - aspects tactiques

R e s u m e

Dans cet article, nous explorons l'aspect tactique necessaire pour la creation d'un joueur de billard virtuel intelligent. Quelques details et ameliorations sont apportes au dernier modele comme l'addition de coups indirects, le bris de tas sur la table et la definition d'une heuristique pour le choix de l'ordre des boules a faire. Nous pro-posons une approche pour le choix de reposition de la blanche sur la table afin d'etre bien prepare pour le prochain coup etant donne qu'aucune autre approche dans la litterature affrontais ce probleme directement. Plusieurs tests sont effectues pour tenter d'isoler la meilleure tactique de repositionnement a utiliser et une discussion finale analyse ces resultats.

Commentaires

Ma contribution a cet article concerne surtout le developpement et 1'application des heuristiques et algorithmes pour resoudre le probleme. Je me suis occupe de tester les principes proposes et de generer les graphes et figures tout en redigeant les parties correspondantes dans le texte. Jean-Pierre Dussault s'est charge de la majorite de la redaction du texte de Particle, me laissant plus de temps pour appliquer et tester les

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R D E BILLARD VIRTUEL - T A C T I Q U E S

differentes approches elaborees. Le travail pour cet article fut effectue conjointement avec la preparation pour le 2e Computational Pool Olympiads. Une adaptation du code a la librairie PoolFiz a permis la generations de mini tournois afin d'obtenir quelques donnees sur l'efficacite des differentes approches utilisees. Cet article fut accepte et presente a la conference Computer and Games Conference 2006 a Turin, Italie [8].

Jean-Pierre Dussault and Jean-Frangois Landry

Departement d'informatique, Universite de Sherbrooke, Sherbrooke, Quebec, Canada J1K 2R1

[ J e a n - P i e r r e . D u s s a u l t I Jean-Francois.Landry2]@USherbrooke.CA

Keywords: Computer pool, tactical optimization. Abstract

We explore in this paper the tactical aspects needed for the creation of an intelli-gent computer pool player, and propose a few modifications to our previous model to improve its technical abilities. Taking for granted an optimization procedure that computes shot parameters to reposition the cue ball on a given target, we take a look at possible heuristics to generate a sound selection of targets to aim for when repositioning. We thus obtain a greedy but rather good billiard player.

Our work concerns the development and improvement of an automated billiard player mod-estly named PoolMaster. The billiard game presents mechanical continuous aspects (how the balls roll, bounce, etc), and planning aspects (strategic play). Most classical games (chess, checker, go, etc) present only a planning difficulty, addressed using some form of search tree. Billiard requires planning, but also accuracy. Following our previous effort, we concentrate on the physical accuracy of our player. We now approach the point where planning will be unavoidable in order to improve the player, but as we show, the accuracy improvement we present yields a much stronger PoolMaster than the previous one, itself runner up in the first Computer Pool tournament.

Recall that in [7], we presented a key computation to develop an optimal billiard player, namely the skill to sink a ball while achieving good position for the next one. This is a very basic skill. We then developed an optimization model to compute the actual shot parameters required to sink the ball, and reach a precise target without addressing the non trivial aspect related to how we choose the target point.

In this paper, we take for granted the basic optimization methodology developed in [7*] and build some tactical behavior within a player exploiting the aforementioned optimization model. We will take the occasion to somewhat refine the optimization model to represent new aspects of the position play unveiled by the tactical aspect of the game.

To begin with, we make precise the context in which the player evolves. We state the basic assumptions on which we rely to construct the player.

We present next a model (similar to [19]) to compute the actual difficulty of sinking a given ball, based on the maximum allowable error in the shots parameters still ensuring the ball is sunk; we consider direct shots as well as kick and bank shots. This basic model allows to rank the remaining balls from the easiest to the most difficult (other approaches have also been developed in [1.6] [5] [3] [2]).

We follow by identifying t h e best spot t o p u t t h e cue ball on. T h i s will b e used as a

target, or as the spot on which we put the cue ball when having ball in hand. Actually, we develop an optimization heuristic to build an ordered list of a few good spots. We discuss variants of the evaluation function.

ac-C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TAac-CTIQUES

count three related deficiencies of the original proposition. Sometimes, the cue ball scratches (gets sunk), other times, the object ball does not reach the pocket, and finally, in some situations, it is not required to pin the cue ball at a given target, but to hit some cluster (to break it) with a minimum speed. Those three situations call for incorporating terms in the optimization model that reflect non vanishing speeds of either the cue or the object ball.

Before concluding, we present some limited statistical observations confirming that Pool-Master is indeed getting stronger.

3.1 Context of computer pool

In order to produce an automated billiard player, the physics of the game must be under-stood, modeled, and simulated. This paper does absolutely not discuss physical modelling. We take as a black box a simulator [15] which "executes" the proposed shots. If the game is played using the same simulator, the computed shots and their outcome will match. If the game is played on a real table, the behavior of the shots will certainly differ as no simulator can ever be perfect. The simulator's input consists of five parameters:

• a and b represent the horizontal and vertical offset of the cue tip from the center of the cue ball;

• 9 the elevation of the cue stick; • <f> the aiming direction ; • V, the speed of the cue stick.

Our player specifies legal values for those five parameters. Given the values, the simula-tor provides information called the table state, indicating every ball position, or flagging pocketed ones.

The first computer pool tournament was held during the 10*'1 computer Olympiad, a parallel event with ACG 11 in Taipei [10]. The simulator was used by the "computer referee", and acted as the table as well. In the tournament, some Gaussian noise is added (with [12]) to the prescribed shot parameters to represent the unavoidable inaccuracies of any player, human or robot; this also allows, albeit indirectly, to acknowledge the limitations of the

simulator which neglects uneven cloth thickness, imperfect cue stick tip, and so many real life details.

Our actual implementation uses the simulator as a black box. Therefore, any improved version of the simulator is likely to improve the realism of our player. We plan to develop our own simulation engine in order to allow some cooperation between the simulator and the optimizer. In particular, the simulator could be enriched to yield some additional information useful for sensitivity analysis. The optimizer ([26]), which handles non linear constrained least squares models, relies on some derivative information. Since we now use the simulator as a black box, derivative are obtained via finite differences.

3.2 Shot difficulty

This section provides functions to compute the difficulty of a shot. We are given the cue ball's position, the object ball position, and the aimed pocket. It is based on the margins of error concept described in [19], [1], section 3.06 and 3.07, and the related technical proofs available from the book's W W W page. We also provide extensions to estimate the difficulty of bank and kick shots. We use the estimate of shot difficulty to obtain a priority order, since we consider not only the easiest shot to pocket, but also consider the repositioning. Therefore, works like [16] [5] [3] [2] are not directly applicable for us. We present below a simple model to perform the ranking we require.

3.2.1 Direct shots

The difficulty of pocketing a given object ball is proportional to the error margin the player has, compared to the perfect shot. The object ball need not reach the center of the pocket. The possible angle 9 depends on the distance of the object ball to the pocket and the pocket width. Once this angle is known, we may compute a target window the cue ball must reach in order that the object ball's trajectory remain within the angle 9P. The maximum angle

8C allowing the cue ball to reach the target window is a measure of the shot difficulty.

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

with

ci = Pd - Pc (3.2)

c2 = pC2 - Pc (3.3)

where pci,pC2 are the two positions the cue will have to hit the object ball to pocket it at

the two extremities of the pocket. As in [7], vectors like Ci and c2 are denoted in bold, their norm is in math italics C\ and their normalization inherits a hat: Ci = ^ . Ci and c2 are the two vectors going from the cue ball position (pc) to the cue ball hit points, as seen on

Figure 3.2.

However, as can be seen in Figure 3.3, this approach may give undesired results in the situations where we have a very big cut angle. Further calculations would need to be added to take this into account and that is why we choose instead to use another method to estimate this difficulty which will require a few less operations and still give a good approximation.

We can easily compute the cosine of the cut angle cos a — vo p • vc o where vo p represents the object ball-pocket vector, and vc o the cue ball-object ball vector.

For a corner shot, get our shot difficulty coefficient KC:

cos a .„ ..

KC = (3.4)

^opVco

For a side shot, we still use a similar method but this time, we add a new parameter 7 which will represent the angle between the object ball and the x axis of the pocket. This will make sure to account for the fact that a shot in direct line with the pocket is a lot easier then one having an angle. 7 is defined as: cos 7 = vo p.x with x = x = [1,0,0] .

We then get the side shot difficulty with:

cos a cos 7 , r,

KS = (3.5)

VopVeo

which will address the fact that a side shot often needs more precision then a corner shot. It is also important to note that a cut-off is used for 7 to make sure that in the case of a very small angle, which would result in an impossible shot, we ignore that shot (although it should never be the case with a good detection of possible shots). In our case, through empirical results, we defined the cut-off value at 0.25.

Figure 3.1: Difficulty Figure 3.2: Difficulty co-efficient estimated with co-efficient represented by cut angle and distance. a,

Figure 3.3: Problems with the exact difficulty coefficient,

observations in section 3.6.1.

3.2.2 K i c k / B a n k Shots

In some cases, often when there is a very big cut angle, it is possible that the easiest of the direct shots is still harder to do than a bank or kick shot. To assess the difficulty of these indirect shots, we will create a mirror image of the current table on the corresponding side. In the case of a kick shot, we mirror the position of every ball on the table except the cue ball, while for a bank shot we do the same except for the object ball too. This enables us to calculate the difficulty of the shot as if it was direct; by taking the coordinates and aiming for the pockets of the mirror table. Of course the coefficient calculated are only valid if we do a standard shot with no spin added, but it still gives us a very good idea of how hard the shot is. This mirror approach is of course incorporated in our optimization model and can

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

be very helpful when there is no possible direct shots. Figure 3.4 shows an example of kick shot.

We use the mirror table to estimate the shot's difficulty. When actually computing the shot, the optimization process benefits from an initial guess obtained by the mirror paradigm, but the actual optimized shot will of course take into account the rebound properties of the rail. While the rebound properties of the rail certainly affect the shot difficulty, we do not use it in our estimate.

3.2.3 Combinations

At times, it may be possible that even a hard shot like a combination is the best shot to do. To correctly determine how hard this shot will be, we can use the same approach as for direct shots, but we need to add two more parameters since we now have two cut angles and 3 distances. The above formula now becomes (for a corner shot):

cos ax cos a2 . .

K — (3.6)

Vo2pVcOl Voi 02

where the notation vP l P 2 stands for the vector joining the positions pi and p%, and vPlP2 is

its norm; o\ is the position of the first object ball, o2 the second, c the cue ball, and p the

pocket; a\,a2 are the two cut angles. Of course, for a side shot, we must scale K by cos7.

3.3 Global table difficulty

In this section, we present variants of a global table difficulty coefficient. We are given the ball's positions, and we provide a way to combine the individual object balls difficulties into a global measure. This ability is a very important one, especially at the beginning of the game if the table is still open as it will let the player choose the best balls to aim for (low or

high) based on how each ball is positioned. It will also come in very h a n d y when t h e player

has a ball-in-hand as we will see in the next section.

Since we already have a coefficient of difficulty for any particular shot from Section 3.2, we can now use this as part of our calculations. We consider three strategies to combine individual shot difficulty coefficients into a global table difficulty.

Figure 3.4: In this case, since there is no direct shot possible (stripes), we will do a kick shot using the north rail by computing a mirror image of the table.

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

A naive approach would be to simply do a summation of all the shot difficulty coefficients, but as we were able to find out, it would seem that such a basic approach is too general and will not privilege the positioning for one particular ball.

Instead, we suggest to use the average and the maximum of the shot difficulty coefficients, which will insure a more appropriate value if the positioning is closer to one ball. We can see the best cue ball positions as outlined by the contours in Figure 3.6. The main difference between the sum formula and the average lies in the fact that the number of considered balls increase the sum, but not the average. Therefore, the average focuses slightly more on the best candidates. This can be seen on figures 3.6 and 3.5. The same comparison can be made with the maximum formula, but to a higher degree since we now only reposition for one ball and forget the other possible ones, thus we have a more aggressive repositioning. In 3.5, the sum clearly shows a preferred position while in figure 3.6, the average measure let appear two other rather good spots. If we look at the max in 3.7, its also very straightforward to see how we reposition for one ball only, which would definitely of use if we want to look a few shots in advance.

In the event of the cue ball not having a clear path to the object ball, we still have to take it into account by adding a small value to our coefficient since the path might clear out after the first shot is made. When that happens we will evaluate the shot difficulty as if the path was clear, but add only half of that value to our computation.

3.4 Best and worst spots on t h e table

In this section, we provide an heuristic optimization procedure to identify the best spots to aim for on the table. We discuss the simplifying assumptions on which our optimization relies.

Spacially, on the table, the table difficulty is a non linear complicated function. We are interested in approximating good spots, corresponding to good local maxima. There might be several of those, and clearly, we are not only interested in the global maximum of this function. Therefore, we propose to search (using an ascent method) local maxima starting from points on a grid. Several starting points usually lead to the same local maximum, and we are left with a short list of good spots. It is as simple as:

Figure 3.5: Outline of the best regions for the cue ball positioning (aiming for the stripes) using the sum measure. In this case, it is obvious that the best area is close to the 9 ball as it will be an easy direct shot and also has all of the other stripes in direct view. Other good spot where fewer balls are in direct view are simply not competitive.

C H A P I T R E 3. O P T I M I Z A T I O N D'UN J O U E U R DE BILLARD VIRTUEL - TACTIQUES

Figure 3.6: Outline of the best regions for the cue ball positioning (aiming for the stripes) using the average measure. In this case, the number of relatively easy balls increases the measure. Apart from the spot close to the 9, two other rather good spot appear: close to the 11, where the average considers only the 11 since all other balls are hidden, and close to the 13. Observe that the spot close to the 11 ball is small, since positions where another ball becomes visible but more difficult because of the distance affect the average.

Figure 3.7: Outline of the best regions for the cue ball positioning (aiming for the stripes) using the maximum measure. In this case, we immediately see each zone is focused on one ball for one pocket and nothing else. It is a much more aggressive play but one that a more talented player will probably use since he will rarely miss his shots.

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

max/(a:,y). (3.7) with f(x, y) representing our evaluation function.

Not only can we use this technique to find the best spot on the table, but in the event that it is not possible to reach one of these spots, we can use the inverse of the function to find the worst spot and put the opponent in a tough position with a safe shot.

There are many other ways to explore a 2D function over a space simply described by a rectangle. We could evaluate / on a coarse grid, and perform a similar search on refined promising sub-grids (displayed by the square zones on Figures 3.6, 3.5, 3.7). However, even a 100 x 50 grid yields an accuracy of only 2cm. We prefer the technique described above since it takes advantage of the continuous nature of the function. Further enhancements will include heuristics to restrict the search to easy to attain regions.

3.5 Optimization with ball speed considerations

Since we now have solved the problem of finding a good target to reach with the cue ball after sinking an object ball, we can concentrate on assembling a much more complete objective function. We will use this section to refine our previous optimization model and solve some important issues to create a really solid player.

3.5.1 Sinking object ball with minimum speed

We start by addressing one of the most important matters, which consists in making sure that the object ball is actually sunk in a pocket during the optimization of the cue ball spin and velocity. Since our optimization method will try to do everything in its power to get the cue ball at the designed target, it is possible that in doing so it will reduce its velocity which in turn might cause the object ball not to be sunk. In our previous model, we had the objective function:

min_. (||p(t) - c||2 - AR2f + \\p(ff) - s||2 + ||p(i) - b | |2 (3.8) vo,u>o,c,t,t

minl(\\MTf)-c\\2 + \\Po(Tf)-p\\2). (3.9)

v0>Wo Z

pc, p0 are the positions of the cue and object balls at their final resting time, c the cue ball target, and p the pocket aimed.

By adding a new component to this objective function to penalize the object ball speed, we are able to ensure it will reach the aimed pocket with the desired minimum speed. Let v0(r/) be the speed of the object ball at its final time r / and vm i n the minimum speed. We can rewrite equation (3.9):

min -(||pc(T/) - c||2 + ||p0(T/) - p f + max(vmi„ - v0{rf), 0)2). (3.10) vo,<*>o Z

3.5.2 Cue ball scratching

A second problem encountered is the cue ball scratching. More often than not, if the cue ball is pocketed after sinking the object ball, it will stay that way because the objective function will then hit a stationary area while minimising its position and will consider it as a local minimum. To help the function get out of this bad situation, we introduce a new term vc(jf) — Vo which represents the velocity of the cue ball at its final resting time (v0

representing a null speed). If the cue ball is immobilised on the table at its final time, then this component will not affect the rest of the function. However, if the cue ball is pocketed, its final velocity will not be null and the optimization to continue its work t o find a better minimum and climb out of the pocket. We slightly complicate our previous equation to :

min -(||pc(T/) - c||2 + ||p0(r/) - p | |2 + max(vmin - v0(rf), 0)2 + (vc(rf) - v0)2). (3.11)

vo,«o Z

3.5.3 Breaking clusters

This small trick will also help us a lot when comes the time t o try and break clusters of balls. Sometimes in a game it may happen that there is simply no way to reposition the cue ball correctly. When that problem arises, it may be a lot more beneficial to try and break a cluster containing one or more of our balls to plan ahead for future shots. We already have

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

most of the necessary components of denned in our last formulation, all we need to add is a time of impact r* to our minimisation:

m i n odlPcfa) ~ CH2 + IIP°(T/) _ P f + m a x( * W - v„(Tf), 0)2 + (vc(Ti) - vcmin)2). (3.12)

We can now easily define a minimum speed at which the cue ball should hit the aimed cluster with vcmin.

Actually, PoolMaster does not wait until being without a good position shot to attempt breaking the clusters; we use a simple heuristic to trigger the choice of breaking the annoying cluster instead of aiming for a reposition. This strategic aspect is one that will benefit from a planification approach, perhaps using a game tree.

3.6 Improved PoolMaster

Now it is the time to look at the actual benefits obtained by incorporating the features described above into our player. We will present observations to assess the gain in strength, and also will document some of our choices by comparisons. We present in section 3.6.1 em-pirical qualitative comparisons between actual success rate and our K difficulty coefficient. Thereafter, all our results will be in the form of mini-tournaments between competing vari-ants. The results were obtained on a local server with a referee created to the exact image of the one used for the official tournaments (exact same rules and noise). As mentioned before, the server adds some additive noise to the shot parameters. The higher the noise level, the closer the different competitors. This makes sense, since a very high noise level makes the probability to miss a shot important, and so buries the expected reward of the refined player. We thus present two classes of results. Small noise to simulate a professional player, and higher noise to simulate a good amateur player.

Adding noise evaluate the robustness of the strategy used. We believe that no line of play will have a significant success under high enough noise levels, but some form of stochastic optimization will certainly improve under small noise levels.

3.6.1 Shot difficulty

As discussed in section 3.2, the difficulty of a given shot depends on the error margin the player has in executing the shot. This is a complicated function of all the shot parameters. Moreover, given a noise specification on the shot's parameters, what is of interest is the probability of success.

When comes the time to choose shots, we wish to rank the different options. In order to do so, let us illustrate that the measure we propose, c o s a is qualitatively reasonable. In figures 3.8 and 3.9, we compare the actual success rate of several cases labeled {d\—d%) with

d\ the distance between the object ball and the pocked, and d2 from the object ball to the

cue ball. We conclude that our formula is a very good predictor for ranking purposes. As seen on the graphs, y/cosa would seems to fit slightly better to the observations, but we still prefer the plain cos, having an intuitive physical interpretation (reduction of a window with increasing angle). Remark that the fit cannot be perfect: for small enough values of the noise, the success may well be 100% for all but the most severe cut angles. This does not mean that the difficulty is constant, but that the difficulty lies within the player's accuracy. To fine tune defensive play, i.e. to decide when attempt a shot, and when use rather a defensive line, we need a refined estimate. A greedy approach is to simulate the shot N times, record the successes, and use this as an estimate.

3.6.2 Choice of t h e global table coefficient

We compare the three table coefficients introduced in section 3.3, using both the optimisation method and a 2d grid zone scan method. Our simulated tournaments indicate that all target

solution variants are more or less equivalent for t h e m o m e n t being. We also t e s t e d one m a t c h

of Optim vs Zones to see which would be the real winner but the results weren't enough descriptives to do every matchup which would've taken 36 tournaments of 100 games. It is quite possible that the differences are so slight that they are covered by other flaws of the player or simply by the noise added to the shots.

CHAPITRE 3. OPTIMIZATION D'UN JOUEUR DE BILLARD VIRTUEL - TACTIQUES * 40cm-40cm 60cm-40cm - 40cm-60cm sqrt(cos()) cos() 1.0 Cut Angle (radians)

Figure 3.8: Shot percentage for (40-40), (40-60) and (60-40). We compare the actual

percentage of success for the tournament noise level. The qualitative fit with ^/cos{a) is

good. In abscissa, we sample a quarter of a circle.

1.0 Cut Angle (radians)

Figure 3.9: Shot percentage for (20-50) and (30-70). Here, we consider easier shots involving situations where the distance of the object ball to the pocket differs significantly from the distance of the object ball to the cue ball. Again, the fit with ^/cos(a) is still plausible, and the equivalence between (20—50) and (50—20), as well as (30—70) and (70—30) is observed.

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

Table 3.1: Optim vs 2d zone scan

Player

Average vs Sum

Average vs Max

Max vs Sum

Small noise

Zones Optim

50vs50 50vs50

50vs50 45vs55

60vs40 46vs54

Higher noise

Zones Optim

46vs54 57vs43

50vs50 44vs56

50vs50 51vs49

Table 3.2: Zones Max vs Optim Max (small noise)

Player

Zones Max

Optim Max

Wins

56

44

3.6.3 PoolMaster Old vs PoolMaster — Improved vs Simple Player

P o o l M a s t e r Old vs PoolMaster—Improved

The original PoolMaster used a simplified global table coefficient, and only back-top spin. No ball speed was incorporated in the optimization model, so that sometimes, the object ball was going in the right direction, but simply did not reach the pocket.

PoolMaster—improved uses the refined global table coefficient, any possible spin, bank and kick shots, and is able to break clusters. Under the higher noise model, though, the differences are quite a deception. The noise destroys the positioning computed by both players; since the differences lie in the refined positioning capabilities, the results are very close. This is not a surprise: a refined player relies on precision, destroyed by too much noise. And this is also why a realistic virtual pool tournament would probably benefit from a much better/advanced noise model instead of just approximate values.

In the table 3.3 and 3.4, by small noise level, we mean Gaussian noise with mean zero and standard deviation a : 0.08, b : 0.08,9 : 0.003, <j> : 0.0185, v : 0.0085 while higher level is

a : 0.8, b : 0.8,9 : 0.03, <f): 0.185, v : 0.085.

Simple Player v s PoolMaster—improved

Here we compare PoolMaster improved to a simple greedy player, one which always chooses the easiest ball on the table, and doesn't deal with indirect or safety shots. The results are quite obvious, the simple player rarely manages to win a game since he never tries to

Table 3.3: PoolMaster Old vs PoolMaster Improved Player PoolMaster Old PoolMaster Improved Small noise Wins Points 37 699 63 800 Higher noise Wins Points 52 740 48 767

correctly reposition for the next shot. We can also see he won a few more games playing on a higher noise table, which probably indicates the other player also missed a few more shots due to noise.

Table 3.4: Simple Player vs PoolMaster Improved

Player Simple Player PoolMaster Improved Small noise Wins Points 3 321 97 970 Higher noise Wins Points 12 443 88 880

Conclusion

We pursued in this paper the development of an optimized billiard player. Building on the strength provided by accurate position play, as described in [7], we studied choices of target to aim for when repositioning. Again, we use optimization ideas to set up a short list of good targets based on estimating shot difficulties, and on a global evaluation function.

We further refined the positioning model in several aspects. First, we take into account non vanishing final velocities, allowing to include mini breaks of clusters of balls as a position target; in this case, the cue ball must hit the cluster with some velocity in order to actually break it. Also, we considered kick, bank and combination shots.

The resulting player is rather strong, but we do not take into account any look ahead strategy yet. Without noise in the execution of the optimized shot, the player never misses but sometimes gets stuck with nothing to play. With kick, bank, combinations shot added to the possibilities, it is very rare that the player finds itself without play, but it happens, most often when he fails to break a cluster.

C H A P I T R E 3. O P T I M I Z A T I O N D ' U N J O U E U R DE BILLARD VIRTUEL - TACTIQUES

which may dictate, in some circumstances, to opt for defensive play instead of taking the risk of the easiest shot, still too difficult. The use of a look ahead strategy will allow to avoid to recourse to a defensive shot most of the time. This is the next step toward an optimal player, namely the planning part of the game, which will probably be addressed by stochastic dynamic programming.

Acknowledgements

We wish to warmly thank the referees, whose comments were so helpful in improving the original version of this paper.

Creation d

5

un joueur de billard

virtuel complet a l'aide de techniques

d'optimisation

R e s u m e

Cet article represente une cumulation des deux articles precedents avec plusieurs ameliorations. On fait tout d'abord un rappel sur le modele propose dans [7] et sur les differentes heuristiques pour le choix de reposition dans [8]. On elabore ensuite la problematique reliee au temps de calcul inutilement eleve resultant du mauvais choix de points de departs pour l'optimisation. Une approche utilisant des tables pre-calculees est proposee et les details de sa generation y sont fournis. On discute ensuite d'un probleme rencontre dans le 2nd Computational Pool Olympiads

concernant le choix du coup a faire en fonction du bruit ajoute aux parametres. Une petite modification a l'heuristique du choix des coups permet d'obtenir un joueur plus robuste choisissant des coups un peu moins risques lorsque possible. Plusieurs

tests p e r m e t t e n t de m e t t r e en evidence les ameliorations effectuees.

C o m m e n t a i r e s

Apres avoir discute et elabore les solutions aux problemes avec le coauteur Jean-Pierre Dussault, j'ai procede a Pimplementation et a la validation de ces nouveaux