The Oedipe System at CLEF-QA 2005

*,+-.0/214365798:.<;>=?+@;BADCE79F>GD1BHI-KJL.M/N7?OP.<;>GQSR41UTV147?/WD7?/N/N7YX

Z HI[]\_^a`Nbdc

^B`

Z,e

CgfhCji>RkX21lR41l;VmiD.<RaCi>RUX:1l-n79GD1l.po6;V+rqR479GVm@7sHt;Vm14;>7?7?/:1l;Vmu^v.0Jw+</x.rX:+@/zyL{

5S|}v|~K\tW€

e@e

~‚ƒW>+;XN7r;D.?y„\….MiV†„\_*]+8z798

Z

79GV7?†wADWD/2.<;>3Y7

‡0ˆŠ‰V‹@Œ0wŽ@M>‘:‰r’dˆŠŒ@V‰M“@’‘z”>‰@„V‰<•>d–—˜>„‰™2Ž@‰„Œ™…”„

šK›wœ:YžYŸ„ r

c,F>1l8P.M/zX2143?R47j¡>/N798N79;@X28¢Q79G>14¡Š7@A,X2FV7'£„i>798…X214+;.<;D8…q7?/214;Vm¤8zyV8…X:7r-¥X2FD.0Xq,.<8¦i>8z7rGJ„y§X:F>7

^a` Z,e C©¨ª+@/n1kX28K¡L.M/zX21l3?14¡L.0X21U+;)X:+EX2FV7 Z ^«HIW«\…¬­[ eM®@® ‚P7YT@.<RliD.0X21U+;¯|'c,FV7¦^a`Z,e Cg¡L.M/zX21l3?1U\

¡L.0X:798-n+@/N76¡D/N793?1l8z79RUy¦X:+nX2FV7s-p+@;>+R41l;VmiD.<RwX:/2.<32O¦GV79GD1439.0X:79GPX:+nX2FV7sWD/N79;D32FERl.<;VmiD.Mm@7@|°c,F>7

-.M1l;¢3xFD.M/2.<3xX:7?/:1l8…X21l36+¨IQ7rG>1U¡Š7ƒ1481UX:88:14-p¡DR41l3?1UX±yw²1UXS-u.<1l;>RUyP/N79R414798]+;PX2FV7ƒ.<8N8z+V3?1ª.0X21U+;P+¨.

Rl14;VmiD148…X21l3K¡D/N7Y\³¡D/N+d3?798N8N+</SX2FD.0X´;V+@/:-u.<Rl1Uµ9798q+@/:GD8­.<;DG/N793?+<m;D1Uµ?7r8­;D.<-p79G79;X:1UX21U7r86.<;>GX:F>7

¡>/214;>391U¡DR4798,+¨aX:F>7­¶t793xX:+@/Sb„¡L.<3Y7s-n+VGV79R·|

¸'¹»ºd¼B½I¾t¿DÀ±¼aÁ¹tÃ]ÄÆÅ,ÇÈÊɄ¼aˊºÍ̼aÁd˯¿DÀ±Î]ºd¾t¿>Á

ÏK|ÐÑÒ9ÓaÔ_ÕLÖ<×ÙØVÚ0ÛÜՊÓ݊Ú0ÕLÖ0ØDÞLߢØLÓaàáßÚrÖMÛÜ߄â>ØLãªä³²ÏK|ÐV|lå Z +;@X:79;@X][S;D.<Rky>8N1l8,.<;>G`…;>G>7Y†V1l;VmKæ^a1l;Vmi>1l8…X21l3

}I/N+V3Y798N8:14;>m>çÏK|ÐV|Ð]`…;D¨ª+@/:-u.0X21U+;pb„70.M/:32Fp.<;>Gƒ*]7YX:/:1U7?T<.<Rægb„7r.M/:32Fƒ¡>/:+d3Y7r8N8?çÏK|ÐV|è´by>8…X:79-u8.<;>Gpbd+@¨ªX±q,.M/N7

æé}v7?/:¨ª+@/:-.M;D3Y767YT@.<R4iL.0X:14+;)fh7?êu3?1479;>3xyE.M;DG¦7?ëŠ7r3xX21kT79;V7r8N82{xçDÏK|ÐV|ìƒí­14m1kXx.<RB^a14J>/2.M/:14798

Ã]¼a¿V¹Iïƒðn¼a¿DñòÁ

CE7r.<8:iV/N79-p79;@XrAD}v7?/:¨ª+@/:-.M;D3Y7@ADH†d¡Š7?/:1l-n79;Xx.0X:14+;

ó ¼¯ôÊõ)¾t¿>ÄSÁ

¬Si>798…X214+;¢.<;>8zq7?/:1l;VmDADCE+;V+R41l;VmiD.<RB£diV798zX:14+;¢.<;D8…q7?/214;VmDADC1l;>1l-u.<RB£„iV7r8…X21U+;E.<;>8…qÊ7?/:14;>mu8…y>8…X:79-

ö ÷wøKù«úvûuüPýPþIùvÿYûsø

¬Si>798…X214+;¤[S;>8zq7?/:1l;VmÙ1l8u.0XnX:F>7¦79G>m@7¦+¨]`…;>¨ª+@/:-.0X:14+;¤*]7YX:/:1U7?T<.<R,.<;DGÂ`…;>¨+@/:-u.0X21U+;ÍH»†dX:/2.<3xX21U+;¯|¤c,F>148

¡Š+8N1UX21U+;FL.<8RU7rG§X:+)X2FV7jG>7YT79RU+@¡D-p79;@XP+@¨6JŠ+<X:F8:14-p¡DRU7Ù.M¡>¡>/N+„.<32F>798?A]-u.<1l;>RUy§JL.<8N79G+;`…;>¨+@/:-u.0X21U+;

*,7?XN/21U7YT@.<R»X:+d+R48?A».<;>G'T@7?/NyÙ8z+@¡DFD148…X21l39.0X:79GÙ+;V798?Av8NiD32F .<8uÑ ät+@/Ñìrät¨+@/s1l;>8…Xx.<;>[email protected]>7r.?TV1lRkyÙ/N79RUy'+;

­.0X2iV/2.<Rt^v.M;>miD.Mm@7}I/N+V3Y798N8:14;>m¢X:+d+R489|E}I/N7?TV1U+iD8s7YT@.<R4iL.0X:14+;>8ƒ1l;ÙX2FV7¢¬SiV7r8…X21U+;Â[S;>8…qÊ7?/:14;>m>79RlGÂFD.?T7

3?R47r.M/:Rkyƒ8NFV+rq;sX2FD.0XFD1UmFK/N798:i>RkX28»39.<;>;>+<X»JŠ7°+@J>X2.<1l;V79Gsq1UX2F´X:+d+­8:14-n¡LRU7Ê8…yV8zX:79-p89|vÏ]+rqÊ7YT@79/9A<1UX8zX214RlRd8z7?7r-p8

;V+<X­3?R47r.M/9AL+@/­.0XSR47r.<8…X­1UX­148;>+<X­.u8:FD.M/N79G¢OV;V+rqRU7rGVm@[email protected].<3xX2iL.MRlRkyE;V7r3Y798N82.M/zy¦X:+JDi>1lR4Gj.u£„i>798…X214+;

.<;>8zq7?/:1l;VmÙ8zyV8zXN7r- X2FL.rX148p3?+-n¡L.M/2.MJDR47@A°14;¤X:7?/:-p8n+¨]/N798NiDRkX28?AIX:+'X2FV7PJŠ798…XuOd;>+rq;Í8…y>8…X:79-u8?|¤c,F>1l8p148

qF@yƒq7FD.?T7]GV7r3?14G>79GnXN+K.<GV+@¡>X°.<;p1l;>3Y/:79-n79;Xx.<R>-n7?X:F>+dGu¨ª+@/IJDi>1lR4G>1l;VmnQ79GD1U¡Š7@AX2FV7]£diV798…X214+;d\±.<;>8zq7?/:1l;Vm

8…y>8…X:79- +¨»X2FV7ƒ^a` Z,e C AŠ8zXx.M/zX214;Vmq1kX2F'.8N1l-n¡LRU7ƒ8zyV8…X:7r- X:FL.0Xq1lR4RvJŠ7K¡D/N+@m@/N798N8:1kT79RUy¢79;>/:143xFV79G¯|sQ79GD1U¡Š7

q,.<8>/:8zXKGV7YT79R4+@¡Š79G)1l; eM®@® èE¨ª+@/´X2FV7H¬SHt*ò7?T<.<RliD.0X21U+;Ñ4åxäÊ.0Jw+@iVXK£„i>798…X214+; .M;D8…q7?/214;Vm8…y>8…X:79-u8´1l;

WD/N79;>3xF¯|»`_Xtq,.<8tGV798:1Um;>79Gu-.<14;>RUyu¨ª+@/D;DG>14;>ms¡L.<8:8:.Mm@7S.<;>8zq7?/:8°.<;DGu1UX:8t+rT7?/2.<R4RLG>798N14m;nq,.<8t;>+<X32FD.<;Vm@7rG

¨ª+@/Ê1kX28¡L.M/zX2143?14¡L.0X21U+;uX:+KX2FV7­W>/:79;>32F¦-n+;V+Rl14;VmiD.<RLX:/2.<3:Op+¨ Z ^aHtW«\…¬­[ eM®@® ‚d|c,FV7­-u.<1l;¦.MGL.M¡VXx.0X:14+;pqÊ7

G>1lGÙ¨ª+@/ Z ^aHtW«\…¬­[òq,.<8­X2FV7.<G>GD1kX21U+;Ù+¨.¦-p+dGDi>RU7nX2FD.0X´7?†„X:/2.<3xX28´8NF>+@/zXK.<;>8…qÊ7?/:8´14;'¡Š.<8N8:.Mm@7.<;>8zq7?/:8

c,FV7s.0/232F>1UX:793xX2iV/N76+¨¯X2FV7sQS79G>14¡Š7´8…yV8zX:79-PAD.M8,14RlR4iD8…X:/2.0X:79G¦Jy¦W»1UmiV/N7på@A>148].K39Rl.<8N8:143r.MRw+;V7´¨ª+@/].ƒ£„i>798…X214+;

.<;>8zq7?/:1l;Vm§8zyV8zXN7r-P| H°.<32F £diV798zX21U+; 1l8">/:8zXE8:iVJD-u1kXNX:79GXN+.Í8z7r.M/:3xF 79;Vm14;>7'X2FD.0X¢/N7?X:i>/:;>8E.Í8z7YXE+¨

GV+V3?i>-p79;@X289|tc,FV798N7´G>+d39i>-n79;X28#>/:8…Xm@+nX2FV/N+iVmFE.pR41l;Vmi>148zX2143´¡>/N7Y\_¡>/N+V3Y798N8N+@/ÊX:+u;V+@/:-.<R414µ?76X:F>791U/,q+@/:GD8

.<;>GÂ1lGV79;X214¨ly)X2FV7914/n;D.<-p79G 79;@X21UX21U7989|jc,F>7¦8:.<-p7¡>/:+d3Y7r8N8:14;Vm'148n.M¡D¡DR41479G XN+X2FV7¦£„i>798…X214+;¯AI¨ª+RlRU+rq79GÂJy

.Ù8N¡Š793?1$D3.<;D.<Rky>8N1l8nX:+ G>7YX:7?/:-p1l;V7¢X:F>7PX±yV¡Š7¢+¨6.<;>8zq7?/u7?†d¡Š793YXN7rG¨ª+@/pX2F>1l8u£diV798zX21U+;¯| c,F>1l88z7r.M/:3xF§148

¡Š7?/:¨+@/:-n79G'X2FV/N+iVmFjX2F>/N7?7uR47YT79Rl8´+¨°m148zX:1l;VmD²D/:8…XrABX2F>7p¡Š.M8:8:[email protected]ƒ.M/N7nX2FV7-n+8zXs8zX:/N+;VmRky'/N79Rª.0XN7rG

X:+X2F>7n3?+;@X:79;X6+¨X2FV7p£diV798zX:14+;Ù.M/N7n7Y†„X:/2.<3xX:7rGj¨/N+- X:F>7pGV+V3?i>-n7r;@X28­/N7YX2iV/:;>79GjJyX2FV7p8z7r.M/:3xFj79;Vm1l;V7@|

c,FV79;BAX2FV7P8z7r;@X:79;>3?798ƒ+¨ÊX2FV7r8z7¦¡L.<8:8:[email protected]/N7PRl1UO@79RUy X:+'3?+@;Xx.<14;Í.<;Í.<;D8…q7?/ƒX:+jX2FV7P£„i>798…X214+;Í.M/N7

8z7rRU793YXN7rG¯|c,FV798N7E8z7r;@X:79;>3?79839.<;.<Rl8z+)JŠ73Y+;>8N1lGV7?/N7rG.<8p¡Š.<8N8:.Mm@7.<;>8zq7?/:89|W»1l;D.<R4RUyA,-p14;D14-.MRU\_R479;Vm<X2F

.<;>8zq7?/:8S.M/N7s7Y†„X:/2.<3xX:79GE¨/N+- X2FV798N7s8N79;XN7r;>3Y798Jy¢R4+V39.0X:1l;VmuX2FV7914/¡DFV/2.<8z7r8ÊX2FL.rXSJŠ798…X­3Y+@/N/N798N¡Š+;>G¦X:+uX:F>7

£„i>798…X214+;P¨7r.0X2iV/N7989|

W»1UmiV/N7uå@²I[/:32F>1UX:793xX2iV/N7´+¨BX2FV7ƒQ79GD1U¡Š7s8…y>8…X:79-

% &úvû üûuþý" ']øƒùù«û()*),+,#

-.0/ 12435

^B`…Cj[ÑÐ0ä_AvqF>143xFÂ8…Xx.<;>G>8ƒ¨ª+@/n^B`…3 e - Cji>RkX21lR41l;VmiD.<R°[S;D.<RUydµ97?/9A1l8ƒ.-n+VG>i>Rª.M/ƒR41l;Vmi>148zX2143¦.<;L.MRUydµ97?/sX2FD.0X

¡Š7?/:¨+@/:-p8°X:7Y†„X,¡>/N+V3Y798:8N1l;Vmp¨ª/N+-!X:+<O@79;D1Uµr.0X21U+;XN+p8zyV;@Xx.<3xX21l36.<;D.<Rky>8N1l8ʨª+@/]~ƒRl.<;VmiL.0m@7r86r|CE+@/N7­¡D/N793?1l8z79RUyA

¨ª+@/ Z ^aHtW«\…¬S[KA»X2FV7ERl14;VmiD148…X21l3E.<;D.<RUyV8N1l8p+¨JŠ+<X:F§GV+V3?i>-p79;@X28.<;>G§£„iV7r8…X21U+;D8p/N79R41479GÍ+;¤X2FV7E¨ª+R4R4+rq14;>m

-n+VG>iDRU7989²

•

X:+@O<7r;>1Uµ97?/

798*:<;>=?;@ACB<D EFAGDC;9=AGHI;CJLKFHI;9BFMN:4OQPRB<D@S=T:4OVUXWYACBFS=?:4OVZ;9H?[\ACB4OQ]H?AC^YSMAB<_a`:FSB<;9=I;bKYEF@@c=edFBf?ACMNfTSMACBFAC@dX=TS=S=LgBF@d

•

-n+@/N¡LFV+RU+@m1l39.<RB.<;D.<RUyV8:148

•

GV7?XN7r3xX21U+;P+@¨v14GD1U+-u.0X21l3­7?†d¡>/N7r8N8N14+;>8

•

¡L.M/zXN\³+¨ª\_8z¡Š7?7r32F¦X2.Mm@m1l;Vm

•

3Y+;XN7r;@X,q+@/:G¢14G>79;@X21$D39.0X21U+;

•

;D.<-p79G¦79;X:1UX±y¦/N793Y+@m;>1UX21U+;

r)7°GV793914GV7rGs;V+<XaX:+i>8N7X2FV7°-n+VG>i>R47I+¨V^a`…C[¤¨ª+@/a7?†„X:/2.<3xX21l;Vm3Y+-p¡Š+i>;>G´;>+i>;>8v.<8a¡>/N7YT>1U+i>8a7?†d¡Š7?/:1l-n79;X28

+;'c,*H Z GD.0Xx.FL.<G'8:FV+rq;X2FL.rXs¨+</s£„i>798…X214+@;).<;>8zq7?/:1l;VmDAa8NiD32FÙ3Y+-n¡Š+i>;DG>8´.M/N7ui>8z7r¨i>R¨+</s8z79R4793xX21l;Vm

GV+V3?i>-p79;@X28nÑ~rä»JLidX´.M/N7n;V+<X6;V7r3Y798:8:.M/:14RUy1l;@X:7?/N7r8…X214;>m¦¨ª+@/68z7rRU793xX21l;VmP8z79;XN7r;>3Y798qF>7?/N7p.<;j.<;D8…q7?/639.<;JŠ7

¨ª+i>;DG¯²tX:F>7?/N7K148;V+/N7r.<8z+;E¨ª+@/S.<;>8zq7?/:8¨ª+@/JŠ7914;>m8zyV8…X:7r-u.0X21439.<RlRkyE.0X­.p8:FV+@/zXSG>1l8…Xx.<;>3Y7s+¨.p3Y+-p¡Š+i>;>G

+¨vX2FV7914/­£„iV7r8…X21U+;¯|SCE+@/N7?+rT7?/9ALX2FV7K¡Š+8:1kX21kT7K7?ëw793xXS+¨3Y+-p¡Š+i>;>G>8S¨ª+@/X2FV7ƒ8z7rRU793YX:14+;E+¨GV+V3?i>-p79;@X28]q,.<8

.<RU/:7r.<Gdy¦+@JVXx.<14;V7rGX2FV/N+iVmF¦X:F>7´8N7r.M/:32F¦7r;Vm14;>76q7´i>8N79G¯|

-.Ts ÅʼB¹I¿>Ëut ¼«Ã]½°ÀzÃ,¼

WD+@/SX2FV7a>/28…X´8z7rRU793xX214+;j+¨tGV+V3?i>-n7r;@X28­¨/N+- X2FV7p3?+@RlRU7r3xX21U+;¯A¯q7pi>8N79GX2FV7p^B`

Z,e

C8z7r.M/:3xF7r;Vm14;V7AwX2FD.0X

FD.<GE.<R4/N7r.<GdyP¡L.M/zX2143?14¡L.0X:79G¦X:+pX:F>7ƒbd-u.<RlRBCiDRkX214Rl14;>miD.<Rac«/2.<3:O+¨

Z

^«HIWÍ14;

eM®@®

ТÑèMä».<;>G eM®@®

èÑ‚9ä_|°c,F>148

8z70.M/:32FÂ79;Vm1l;V7P1l8n3Y+;>3Y79¡VXN\³JL.<8z7rG¯AqF>143xF¤-p7r.<;>8KX:FL.0Xp1kXu¨ª+V3?i>8N798ƒ+;¤14GV7r;@X214¨ªyV1l;VmÙ1l;Í.£diV7?/zy¤1kX28n-p+@8zX

8N14m;>1$D39.<;@Xu3Y+;>3?7?¡VX28?AIm@79;V7?/2.<RlRky /:7?¡>/N798N79;XN7rG¤.<8n-KiDRkX21k\³XN79/:-p8p.M;DGÂ;D.<-p79G 7r;@X21kX214798?A°.<;>G¤¨Ü.?T+@/:8n14;¤1UX28

/N798:i>RUX:86X2FV7uG>+d39i>-n79;X28­X2FD.0XK3Y+;@Xx.<1l;'+;V7u+V3?3?iV/N/N7r;>3Y7u+¨t7r.<3xFÙ£diV7?/zyÙ3Y+;D3Y7?¡VXrA«+@/K.PRU70.M8zX´X2FV7uRª.M/Nm@798…X

;„iD-sJŠ79/6+¨IX2FV798z7u3Y+;>3?7?¡VX28?ABqFD.0X:7YT7?/­X2F>791U/s¨ª+@/:-wvp.<;>GjX2FV7914/s;„i>-KJŠ7?/´+@¨t+V3?3?i>/N/N79;>3?798´14;ÙGV+V3?i>-p79;@X289|

c,FV7Ê8N7r.M/:32FK7r;Vm14;>7/N79Rl1U7r8v+;ƒ^B`…Cj[§¨ª+@/vX2FV7ÊRl14;>mi>148zX2143,.M;L.<Rky>8N148»+¨>JŠ+<X2FsX2FV7ÊG>+d39i>-n79;X28».<;>GKX2FV7Ê£„i>7?/:1U7r8?|

c,FV7S3Y+;xDmiV/2.0X21U+;u+¨w^a`…C[q,.<8IX2FV7S8:.<-p7S.<8X2FV7+;V7SGV7r8N3Y/:14JŠ79G14;pX2F>7]¡D/N7YTV14+i>8°8N793xX21U+;¯A„7?†V3Y7?¡>XtX2FD.0X

X2FV7´3?+-n¡Š+i>;>GP7Y†„X:/2.<3xX:+@/Êq,.<8].<GDGV79G¯|

WD+@/ Z ^aHtW«\…¬S[KA,;V+Í8N¡Š793?1$D3Ù.<GD.M¡VXx.0X214+@;+¨6X2FV7Ù8N7r.M/:32F79;>m14;V7'q,.<8PGV+;V7@| Ht.<3xF £diV798…X214+;q,.<8

8Ni>JD-p1UXNXN7rGjX:+¢X2FV78z7r.M/:3xFÙ79;Vm1l;V7pq1UX:F>+idXƒ.<;y'¡>/:7Y\³¡>/N+V3Y798:8N1l;Vm.<;>GÙX2FV7w>/:8zXn‚

®

G>+d3?iD-n79;X286m1UT@7r; .<8

/N798:i>RUX,q7?/N7´8N79R4793xX:79G¢¨ª+@/ÊX2FV7s;V7Y†dX8…X:7?¡D89|

-.?- y

Ç]¼aÁº>À_¾Ã ¹IÃ]¹tï³ô]ÁVÀ±Á

c,FV7´.<;D.<RUyV8:148Ê+@¨«£„iV798zX21U+;>8].M1l-p8,.0X]GV7YX:7?/:-u14;>1l;VmƒX2FV7SX±yd¡Š7­+¨BX:F>7­7Y†d¡Š7r3xX:79GP.<;>8zq7?/9|C+@/N768z¡Š79391kD3r.<R4RUyA

1kX6GV7?XN79/:-p1l;V798S14¨»X2FV7n.<;>8…q79/S1l8­.u;L.<-n79GE79;X:1UX±y¢+@/­;V+<XrA¯.M;DG14;¢X2F>7>/:8…X639.<8z7@ALX2FV7sX±yV¡Š7K+¨vX:F>7sX2.M/Nm@7?X

;D.<-p79G7r;@X21kX±y„|zr)76GD148zX:1l;Vmi>1l8NFP+;>RkyEì´X±yd¡Š7r8Ê+¨«;D.<-n7rG79;X21kX21U7r8?²¡Š7?/28z+;¯AV+@/Nm„.<;>14µr.0X21U+;¯ADRU+V39.0X21U+;BA>GD.0X:7

.<;>GnX214-n7A;di>-n79/:1439.<RD-p7r.<8Ni>/N7@A@7?T@79;X.<;>Gp¡>/N+VG>iD3xXr|»H†V3?7?¡VX°¨+@/X2FV7,X±q+sRl.<8zXt+;V7989A@X2FV7Yyp3Y+@/N/N798N¡Š+;>GƒX:+

X2FV7,X±yV¡Š798I+¨w;D.<-p79Gp79;@X21UX21U798°GV7 D;>79GpJ„yƒX2F>7]C{ Z 7YT@.<RliD.0X21U+;>89|»c,F>7S.<;D.<Rky>8N1l8I+¨w£diV798zX:14+;>8t1l8°.<32F>147YT79G

Jyj.u8N7YXS+¨

e è /:iDRU798S1l-n¡DR479-p79;@X:79Gj.<8|D;D1kX:7Y\_8zX2.0X:7n.<idX:+-u.0Xx.V|]c,F>798z7n.<idX:+-u.0Xx..M/N7ƒ8N1l-p14Rª.M/]X:+X:F>+8z7

GV7 L;V79GE¨+@/]/N7r3Y+@m;>1Uµr14;Vm;D.<-p79GP7r;@X21kX214798S.<;>GE1lG>[email protected]´7Y†d¡D/N798:8N1U+;D814;E^B`…Cj[s|LH°.<32FE/:i>R47s1l8S.nOV14;>GE+¨

RU7?†V1l3Y+<\_8…y>;@Xx.<3xX2143Ê¡Š.0XzX:79/:;KX2FD.0Xt39.<;p.<Rl8z+´14;X:7?m@/2.0X:7,8z7r-u.<;@X21l3Ê3?Rl.<8:8z7989|r FV79;p1kXt1l8vX:/:14m@m@7?/N79GBA@1UX°.<8N8N+d391l.0X:798

X:+X2FV7ƒ£diV798zX21U+;E+;>7sX±yV¡Š7n.<-n+;>muX2FV7¦å?脀u£diV798zX:14+;EX±yd¡Š798q7nG>148zX214;VmiD148:F¯|­[S8]X2F>1l8]X±yV¡Š+RU+@m<yFV7r.?T>14RUy

/N79Rl1U7r8]+;¢X2FV7K8:iV/:¨h.M3?7K¨ª+@/:- +@¨vX2FV7ƒ£„iV7r8…X21U+;D8?Aw.-u.M¡>¡D1l;Vm1l8G>7 D;V7rG¢Jw7YX±q7?79;¢X2FV7ƒ£„i>798…X21U+;¢X±yV¡Š798S.<;DG

X2FV7n.<;>8…q79/]X±yd¡Š7989|][ £diV798zX21U+;39.<;EFD.9T@7ƒ8z7?T@7?/x.<R«.<;D8…q7?/X±yd¡Š798]qFV79;¢X2FV7K/:i>R4798­.M/N7K;>+MX­8:iVêu3?1479;X­¨ª+@/

32F>+„+8:14;Vm.M-p+;VmƒX2FV7r-P|Ic,F>1l8,1l8X2FV7s39.<8z7s¨ª+@/]8z+-p7s.M-KJD14mi>1kX214798,JŠ7YX±q7?7r;¦¡Š7?/28z+;>8].<;DG¦+@/Nm„.<;>14µr.0X21U+;>89|

c,F>7n/2i>RU7r86¨+@/´£diV798…X214+;Ù.<;L.<RkyV8:148­q79/N7p79Rl.MJŠ+@/2.0X:79GÙ¨ª+RlRU+rq1l;Vm.¦8z79-u1k\±.<iVXN+-.rX21l3n-n7YX2FV+VG¯A*D/:8…XsGV7?\

T79RU+@¡Š7rGu¨ª+@/IX2FV7H¬SHt*7YT@.<R4iD.0X214+@;Ñe ä¯.<;>GnX2FD.0X°148°14;D8z¡D14/N79GpJ„yp[SR414m;>-n7r;@XN\_5,.<8z79Gu^B70.M/:;>14;>mр0ä³|c,F>148

-n7?X:F>+dGÙ8…Xx.M/zX286¨ª/N+-.¦3?+</:¡Di>86+¨t£diV798zX:14+;>89Aa-u.<G>7p14;'+iV/s39.<8z7p+@¨IX:/2.<;>8:Rl.0X:79G'c,*H Z \z¬S[!£diV798…X214+;>80}

.<;>G¢£„iV7r8…X21U+;>8]¨ª/N+- X2FV7´¡>/N7YT>1U+i>8 Z ^aHtW«\…¬­[ 7?T<.<RliD.0X21U+;>89|IW»1U/28…XrAdX2F>7679G>1UXG>1l8…Xx.<;>3Y7´+¨«^a7YT@7r;>8NFX:7914;¢148

3Y+-n¡LidX:79GE¨ª+@/7r.<3xF¢¡L.<1U/+¨»£diV798zX21U+;>89|,c,FV79;¯ADX2FV7K^a+;Vm@798…X Z +-p-n+;bViVJD8zXN/214;Vm¦.<RUm@+@/:1UX2F>- 148­.M¡>¡DRl1U79G

~G8*:<;l=?;0AGHIMN:;0B<DSiB<;M0ACBHI;9M>gDBFS9;ALM9gB<M9;>W<f€gCj4fT:<;XE<;9Hed#SjQSfAGWFW‚;9ACHI=SB#AL_FgXM9EF[\;0Bf€AC=An=edFB<g BCdF[ƒgCH€AL=?E<^FfI;9H?[b

„C…mgCHI;WFHI;9M0S=I;0@d OGfT:<;9=I;*XE<;9=If?SgB<=cM9g[†;jkHIg[{f?:F;*8*‡lP€`uˆŠ‰FO8*‡lP€`uˆŠ‹zACB<_\8l‡lP€`uˆTŒ9;>hCAC@EFAŽfTSgB<=RACB<_;9HI;€f?H?ACB<=?@iAŽf?;9_

-n+;8:iVJD8…X:/:1l;Vm8Ê.M/N7­8z+@/zX:79GuX:+’D;>GuX2FV76£„iV7r8…X21U+;X±yd¡Š798°qFV7?/N70.M8tX2F>7­G>148zX214;>3YX¡Š.M/zX:83r.<;JŠ7Sm@/N+iV¡Š7rGpX:+

¨ª+@/:- 3?Rl.<8:8z798Ê+¨«79;X:1UX21U798Êq1UX2FP8:14-u14Rª.0/]3xFD.M/2.<3xX:7?/:1l8…X21l3?8?²

r FD.0X148ÊX2FV7´3r.M¡D1kXx.<R¯+¨*“°iVm@+8NRª.?TV1ª.4”

r FL.rX1l8ÊX:F>7´39.M¡D1UXx.<R¯+@¨vCj.<GD.Mm„.<8N3r.M/C”

• –c—C˜™ešI›pœ{™eGžc—

² qFD.0X<Ÿ,1l8 ŸX2F>7‚Ÿ,39.M¡L1kXx.<ReŸ,+¨

 X¡k¢Y˜G˜,›9£, G›p–xœu™e¤Cš?—G˜

² “ti>m@+8NRl.?T>1l.VADC'.MGL.Mm.<8:39.M/

c,F>1l8,-p7YX2FV+VGP148]14-p¡DR479-n79;X:79G¦J„yX:F>7 Z +}tcX:+„+R·ADGV7YT79RU+@¡Š7rGPJyP[S;@X:+;>14+u5Ê.<RUT@7?X0¥|

-.e¦ §

¹IÁdÁ„¹»½I¼ ¼*¨ÊºV¿V¹ËŠº>À_¾°Ã,©´¿V¹tÃ!ª,À…Ã]½¹tÃÄ Á„¼«ï_¼aˊº>À_¾°Ã

[S¨lX:7?/pX2FV78z79R4793xX21U+;§+¨­.'/N798zXN/2143xX:79G8z7YX+¨SG>+d3?iD-n79;X28pJy¤X2FV7E8N7r.M/:32FÍ7r;Vm14;>7@A]Q79GD1U¡Š7GV79Rl14-u1kX28pX:F>7

¡L.<8:8:.Mm@798+¨vX2FV7nGV+V3?i>-p79;@X28]X2FD.0X´.M/N7ƒRl1UO@79RkyEX:+¦3Y+;@Xx.<1l;j.<;j.<;>8zq7?/X:+X2FV7ƒ3?+@;D8N1lGV7?/N79Gj£„i>798…X214+;¯|­c,F>148

GV79Rl14-u1kXx.0X21U+;/N79Rl1U7r8°+;uX:F>7SG>7YX:793xX214+@;+¨wX2FV76.M/N7r.<8t+¨BGV+V3?iD-n79;X28Iq1kX2FuX2FV7­F>14mFV798zXÊGV79;D8N1UX±yu+¨wq+@/:G>8°+¨

X2FV7s£diV798zX21U+;¯|t`_XS1l8]GV+;V7´J„y¦m1UTV1l;VmpX:+p70.<32F¢¡Š+8N1kX214+;P+¨».pGV+V3?i>-n7r;@X­.M;E.<3YX:[email protected]:14+;¦T@.<RliV7@²IqFV79;E8Ni>3xF

.6¡Š+8:1kX214+@;3Y+;Xx.<14;>8.6q+@/:Gp+¨ŠX2FV7S£„iV7r8…X21U+;BAV.«V†V79GpT@.<R4iV7S1l8.<G>G>79GnX:+ƒ1kX28.<[email protected]+;pT@.<R4iV76.<;>GpXN+sX:F>7

.<[email protected]+;T@.<R4iV76+¨¯X2FV7­¡w+@8:1kX214+;>8,.M/N+i>;DG¦1UX´f

activSpread

¡Š+8:1kX21U+;D8+;X2FV7­/:14mF@X.<;>GX:F>76R479¨lX]8:14GV7r82{x|

W»14;L.<R4RUyAVX2FV7sGV79Rl14-u1kX:79GP¡L.<8N82.Mm@798,3Y+@/N/N798N¡Š+;>GX:+nX2FV7´3?+@;X21UmiV+iD8Ê¡Š+8N1UX21U+;>8,+¨BX2FV7sGV+V3?i>-p79;@X¨ª+@/ÊqF>1l32F

X2FV7s.<3YX:[email protected]:14+;¦T<.<RliV7s148]F>14mFV7?/,X2FD.<;¢.’V†V79G¦X2FV/N798:FV+R4GB|

[ 8N3Y+@/N7148pX2FV79;3Y+-p¡DidX:79G¨ª+@/7r.<32F§7Y†dX:/2.<3xX:79GÍ¡Š.<8N8:.Mm@7@| c,F>1488N3Y+@/N7PXx.MO@79814;X:+Â.<3?3Y+iD;@XuX2F>/N7?7

¨Ü.<3xX:+@/:89²

•

X2FV7;di>-KJŠ7?/.<;>GnX2FV78:1Um;>1$D39.<;>3?7]+¨ŠX:F>7,q+@/:G>8I+¨LX2FV7S£„iV7r8…X21U+;pX:FL.0X.M/N7]¡>/N7r8z79;X°1l;nX2FV7¡L.<8N82.0m@7|

c,FV78N14m@;D1kD3r.<;>3Y7+¨Ê.¢£diV798…X214+;Ùq+@/:G)148s7YT@.<R4iL.0XN7rGÙJ„y)1kX28K;V+@/:-u.<Rl1Uµ?7rG)14;D¨ª+@/:-u.0X21U+;BAv3Y+-n¡LidX:79G

¨ª/:+- e y7r.M/:8,+¨BX2FV7’¬

—«­®›pœR¯°—

;>7Yq8z¡L.M¡Š79/9ç

•

X2FV7S¡D/N798z7r;>3Y7­1l;uX2F>7S¡L.<8N82.Mm@7+¨«.K;D.<-n79G7r;@X21kX±yuX2FD.0X,3Y+@/N/N798N¡Š+;>G>8tX:+KX2FV7S7?†d¡Š793YXN7rG¦.<;>8zq7?/°X±yd¡w7 qFV7r;X2FV7K.M;D8…q79/ÊX±yd¡Š7s148.n;D.<-n7rG¦79;X21kX±ywç

•

X2FV7sGV79;>8:1kX±y¦+¨aX:F>7­q+@/:G>8,+¨BX2F>7´£„i>798…X214+@;¢1l;X2F>76¡L.<8N82.Mm@7@|

CE+@/N7´¡>/N7939148z7rRkyA>X:F>7´8N3?+@/N76+¨v.ƒ¡Š.M8:8:.Mm@7

p i

148?²

score(p i ) = α · wordScore(p i ) + β · neScore(p i ) + γ · densityScore(p i )

^fzå0{

qF>7?/N7

α

^A

β

.<;>G

γ

.M/N7´-p+dGDi>Rl.0X:+@/:8G±6.<;>G¢.<RlRwX:F>7´8:3Y+@/N798.0/:76JŠ7YX±q7?7r; ® |® .<;>G'å@|® | c,F>7­q+@/:G¢8N3Y+@/N7´1l8,m1kT79;PJy¯²

wordScore(p i ) =

X

k

signif icance(w k ) number of question plain words

fe {

qF>7?/N7

w k

1l8].Kq+@/:GP+¨

p i

X:FL.0X]1l8].KqÊ+@/:G¦+¨BX2F>7´£„i>798…X214+;¯|

c,F>7P;D.<-n7rGÂ79;@X21UX±y¤8N3Y+@/N7P1l8ƒ79£diD.<R°X:+¤å@|® 14¨S.j;D.<-n79GÂ7r;@X21kX±y X2FD.0Xu3Y+@/N/N798N¡Š+;>G>8KX:+jX2FV7¦7?†d¡Š793YXN7rG

.<;>8zq7?/X±yV¡Š7ƒ148S¡>/N7r8z79;X614;

p i

.<;>G¢X:+ ® |® +<X2FV79/zq148N7@|Sc,FV7nGV79;D8N1kX±y8:3Y+@/N7ƒ148­GV7 L;V79GEq1kX2F/N798z¡w793xXSXN+P.

/N79¨7?/N79;D3Y7´8:1Uµ97´¨ª+@/.ƒ¡Š.<8N8:.Mm@7@A>m1kT79;PJy¯²

reference size = 2 ∗ activSpread + number of question words (p i )

^f³Ð{

² `€gCHIWYE<=l³uHIg M9;9=I=TSB<D#8QgXg@=9OpA9hCACS@AG^p@;AŽf0Jx´µµG¶u·Š¸C¸ ¹Gº9¶µu»e¼Gº9½X¾X¹Ž¿CÀ ºŽ¾CÁ¿x»ÃÂX¿Žµ

ÄGKFgCHgE<H`QPuKQˆeÅ]ÆH?EFB4O

α

ACB<_

β

;9HI;z;9XEFA@Yf?gŒCb<bu8*:<;hCAC@E<;ngCj

γ

_F;9Wp;0B<_F=gB!fǏg\j$AGM>fIgHI=9bu8*:<;ÈYHI=If*gB<;nS=fT:<;

M9gCHI;L[†gX_YEF@AGfIgHhCA@E<;n=I;>fAC=lA\WYAGH?AC[\;Nf?;9HLÉ$;9<EFAC@‚f?gŒCbSB|g E<H*M0AG=I;ŽÊNbu8*:FS=lj$AGM>f?gCHS=[†gX_YEF@AŽf?;9_m^d,J

number of question words (p i ) ⁴ (number of question words (p i ) + 1) ⁴

:FSM>:[\ACoC;9=fT:<;n_F;0B<=?SfÇd|=?M>gHI;n@;>=?=S[†W‚gCHefTACBfl:<;0B!fT:<;nBXEF[L^‚;9Hgjx<E<;>=IfTSgB|gCHI_F=*f?:FAGfAGHI;nWFHI;9=I;0BfSB

p i

S=:FSD:4b

`…¨VX:F>78N14µ?7°+¨

p i

1l8vR4798N8aX2FL.M;

reference size

A<1UX28vGV79;D8N1kX±yK8N3?+@/N7148«7r£„iD.<RX:+S1UX:8»-u.0†V1l-u.<R@T@.<R4iV7A

š?ˊ—<Ë å@|® |

Q]X2FV79/zq148N7@Ad1UX148Ê.rXNX:79;diD.0X:79Gpq1kX2Fu/N798N¡Š793xXtX:+ƒFV+rq-ƒi>3xFnX2FV7S8:1Uµ97]+¨

p i

1l8tm@/N7r.0X:7?/IX2FD.<;

ref erence size

^A

Jy¦JŠ7r14;Vmu79£„iL.<RwXN+D²

densityScore(p i ) = 1 r passage size

ref erence size

fhèd{

Q­;>3Y7ÊX2FV7914/I8:3Y+@/N7,148I3Y+-n¡LidX:79G¯A<X2FV7Ê¡Š.<8N8:[email protected]/N7]8z+@/zX:79Gp.<3?3?+@/:G>14;>m­X:+SX2F>7,GV793?/N7r.<8N1l;Vm6+@/:GV7?/+¨DX2FV7914/

8N3?+@/N767YX,X2FV7«D/:8…X

N

¡L.<8:8:.Mm@798].M/N76O@7?¡>X¨ª+@/ÊX2FV7s¨iV/NX:F>7?/]8zXN79¡D80Ì@|

Í &úvû )ll),+!ù«û()KøÎ,ú

Q7rG>1U¡Š7tq,.<8lD/:8…X»GV7YT79RU+@¡Š7rGK.<8v.]£diV798zX:14+;K.<;>8zq7?/:1l;VmS8…y>8…X:79- GV79GD1439.0X:79G´X:+!L;>G´¡L.<8:8:.Mm@7.<;>8…q79/:8a/2.0X2FV7?/

X2FD.<;X:+aL;>GP8:FV+@/zX.<;>8zq7?/:89|r 7s.<GD.M¡VX:79GP1kX¨+@/X2F>7

Z

^aHtW«\…¬S[ 7YT@.<R4iL.rX214+;¦JDiVXÊq1UX2FV+idX32FL.M;>m14;Vmu1UX28

+rT7?/2.<R4RBG>798N14m;¯|IÏ79;>3Y7AD1kX!>/:8zX8z7r.M/:3xFV798]¨ª+@/,¡L.<8N82.Mm@7s.<;>8…q79/:8].<;>G¦X2FV79;X:/:14798X:+ÏL;>GP8:FV+@/zXS.<;>8…q79/:8,1l;

X2FV79-¢|

¦!.0/ Ð

¿V¾°ñ ιIÁdÁ„¹»½I¼«Á ºd¾!ιÁdÁ„¹½I¼ ¹IÃÁõ)¼a¿>Á

Q7rG>1U¡Š7'R4+d39.0X:798¦.)¡Š.M8:8:.Mm@7'.<;>8…q79/1l;7r.<3xF8z7rRU793xX:7rG¡L.<8N82.Mm@7@| c,FD148¡>/N+V3Y798:83?+;>8N1l8…X281l;-n+rT>14;>m¤.

q14;DGV+rq +rT@79/X2FV7­Xx.M/Nm@7YX,¡L.<8N82.Mm@7´.<;>GX:+p3Y+-n¡LidX:7´.ƒ8:3Y+@/N7´.0X,70.M3xF¦¡Š+8N1UX:14+;+¨BX2FV7­q14;DGV+rq.M393Y+@/:G>1l;Vm

X:+´1kX28I3Y+;@X:79;Xr|»c,FV7,8:1Uµ97Ê+¨DX2F>148»q1l;>GV+rq14879£diD.<RdX:+­X2FV7,8:1Uµ?7,+¨DX:F>7].<;>8zq7?/»X:+67?†„X:/2.<3xXCÑM|»c,FV7,7Y†dX:/2.<3xX:79G

.<;>8zq7?/]148ÊX2FV7s3Y+;@X:79;X]+¨BX2FV76q1l;>GV+rq ¨+</1kX28,¡Š+8N1UX:14+;q1UX2FX2FV7sF>14m@F>7?/]8:3Y+@/N7@|

c,F>7sq,.?yPX2FV7sq1l;>GV+rq 148S-p+rT@7rGEG>7?¡Š79;>GD8+;¢X2FV7K7Y†d¡w793xX:79Gj.<;>8zq7?/X±yd¡Š7@|`…¨vX2FV7K7Y†d¡w793xX:79Gj.<;>8zq7?/

148s;V+<Xƒ.P;D.<-n7rG'79;@X21UX±yAaX:F>7nq14;DGV+rq!1l8´-p+rT@79GÙ+rT@79/´7r.<3xF'¡DRª.<14;'q+@/:G'+¨tX2FV7p¡Š.M8:8:.Mm@7@|QX:F>7?/zq1l8z7@Av1kX

148­-p+rT@7rG+@;DRkyE+0T@7?/SX2FV7ƒ¡Š+8N1UX21U+;>8qF>7?/N7p.;D.<-n7rGE79;X21kX±y¢X2FL.rX´3Y+@/N/N798N¡Š+;>G>8X:+X2FV7ƒ7Y†d¡Š7r3xX:79G'.<;>8zq7?/

X±yd¡w7´1l8Ê¡D/N798z7r;@Xr|`…;¦JŠ+<X2FP39.<8N798?AVX2FV7s8N3Y+@/N7s3Y+-n¡DiVXN7rG¢.0X]7r.<32FP¡Š+8N1kX214+;P1l8X2F>7´8NiD- +@¨aX±q+u8:iVJV\_8:3Y+@/N7989²

•

.8:3Y+@/N7¦[email protected];>mX2F>7¦;di>-sJw7?/p.<;DG)X2F>7¦8:1Um;>1$D39.<;>3?7¦+¨ÊX2FV7P£„iV7r8…X21U+; q+@/:GD8sX2FD.0Xu.M/N7P1l;)X2F>7 q1l;>GV+rqs|Ic,FD148]8N3Y+@/N7s148ÊX2FV7´82.M-p7s.<8X2FV76q+@/:GP8:3Y+@/N7´¨ª+@/,¡Š.<8N8:.Mm@798´f·8N7?7 e {xç

•

.´8:3Y+@/N7,X2FD.0X1l8°G>1U/:793xX2Rkyp7r£„iD.<RDXN+sX:F>7]¡>/N+@¡Š+@/zX214+;p+¨ŠX2FV7;L.<-n79Gu79;@X21UX:14798t+¨ŠX2FV7£diV798zX21U+;pX2FD.0XÊ.M/N7 1l;X2FV76q14;DGV+rqs|

WD+@/£„i>798…X214+;>8tqFV+8z7S7Y†V¡Š793xX:79G¦.<;>8zq7?/Ê148;V+<X,.K;D.<-n79G79;X:1UX±yAV1UXÊ1l8¨ª/N79£diV79;X°X:+ƒFD.?T7­8N7YT@79/2.<Rw.<GYÒz.<3Y7r;@X

¡Š+8N1UX21U+;>8]q1UX2F¢X:F>7K82.<-n7K8:3Y+@/N7@|#r F>79;8NiD32Fj.u3r.<8z7ƒFD.M¡>¡Š79;D8,q1UX2F¢X2FV7ƒF>14mFV798…X68N3Y+@/N7K+¨vX2FV7K¡L.<8N82.Mm<7A

X2FV7ƒ8z79R4793xX:79GE¡Š.<8N8:.Mm@7ƒ.<;>8zq7?/­148]Xx.MO<7r;E¨/N+- X2FV7ƒ-p1lG>G>R47K+¨«X2F>1l8µ?+;V7n.<;>G;V+<X­¨ª/:+@- 1kX28JŠ7?m1l;>;>1l;VmDA¯.<8

X2FV7s.<;D8…q7?/]+¨lX:79;P3?+-n798].<¨lX:7?/,X2FV7­q+@/:GD8Ê+¨BX2FV7s£diV798…X214+;¯|

W»1l;D.<R4RUyA«.<86¨ª+@/6¡L.<8N82.0m@7r8?Aa.<R4R«X2FV7n¡L.<8:8:.Mm@7p.<;>8zq7?/:8´.M/N7n8N+@/zX:79G'.<3?3?+</2G>14;>mX:+X2FV7pGV7r3Y/N7r.<8N1l;VmP+</2GV7?/

+¨»X:F>791U/68N3Y+@/N7|`…¨»X2FV7ƒ8:3Y+@/N7K+¨»X:F>7ƒFD1UmFV798zX´.<;>8zq7?/­1l8]X:+d+R4+rqsA

šIˊ—<Ë

JŠ79RU+rq.wV†d7rG¢X2F>/N798NF>+@RlG¯AaQ79GD1U¡Š7

.<8N8:i>-n7r8X:FL.0XÊX2FV79/N7´1l8,;V+.<;D8…q7?/,X:+ƒX2FV7s3Y+;>8:14GV79/N79G¢£„iV7r8…X21U+;¯|

¦!.Ts Ð

¿V¾°ñ ιIÁdÁ„¹»½I¼ ¹tÃÁõÙ¼«¿>Á)ºd¾ Áxt,¾°¿dºÂ¹tÃÁõÙ¼«¿>Á

r F>79;¢X2FV7K7Y†V¡Š793xX:79Gj.<;D8…q7?/­1486.u;D.<-p79GE79;@X21UX±yALX2FV7K7Y†dX:/2.<3xX21U+;E+¨8NFV+@/zX´.<;>8…q79/:8S148S8zXN/x.M14mF@X2¨ª+@/zq].M/:G¯²

X2FV7´¡Š.<8N8:.Mm@7K.<;>8…q79/Êq1UX2F¦X2FV7sFD1UmFV798zX8:3Y+@/N7s1488z79R4793xX:79G.<;>G¦X2FV7s;L.M-p79GP79;X21kX±y¦+;PqF>143xF¦X2FV7´¡Š.<8N8:.Mm@7

.<;>8zq7?/,7Y†„X:/2.<3xX214+;q14;DGV+rq q,.<8,3Y7r;@X:7?/N79G¢148,/N7YX2iV/:;V7rG¢.<8].ƒ8:FV+@/zXS.M;D8…q79/9|»`…;¦X:F>76+<X:F>7?/]39.<8z7AD.n8z7r.M/:3xF

¨ª+@/n.E8NFV+@/zXn.<;D8…q7?/KJL.<8N79G)+@;¤.¢8N-.<R4Rt8N7YXK+¨ÊFV7riV/:1l8…X2143?8ƒ148s¡w7?/:¨ª+@/:-p79G¯|¢c,F>1l8K8z7r.M/:3xFÂ.<8N8:i>-n798´X2FD.0Xn.

8NF>+@/zX,.<;>8zq7?/Ê1l8Ê.K;V+i>;¡DFV/x.M8N7@|Ï79;>3Y7ADQ7rG>1U¡Š76R4+d3r.rX:798,.<R4RLX2FV7­;>+i>;>8°¡LFV/2.<8z7r8°+¨wX2FV7S¡Š.<8N8:.Mm@76.<;>8zq7?/

Jy¢.M¡>¡LRky>14;VmpX2FV7s¨ª+R4R4+rq14;>mu-p+</:¡DFV+<\_8zyV;@Xx.<3xX21l3­¡L.0XNXN79/:;¯²

Ó

N

S=;><EFAC@‚f?g#ÔC†jkgCH*fT:FS=l;NhCA@EFAŽfTSgB°b

c,F>79;¯AD1kX]3Y+-p¡DidX:798,.n8N3Y+@/N76¨+@/Ê7r.<3xF+¨BX2FV79-P|Ic,F>1l8Ê8:3Y+@/N7SXx.MO@798Ê1l;@X:+u.<393Y+i>;@X]JŠ+<X:FX2FV768:1Uµ?76+¨BX:F>7

.<;>8zq7?/.<;>G¢1kX28]3Y+;@X:7Y†dXr²

•

1UX:8,JL.<8N7´148,¡>/N+@¡Š+@/zX214+;D.<RŠX:+nX2FV7s8N1Uµ976+¨aX2FV7s.<;>8zq7?/9AVq1kX2FE.’V†d7rGPRl14-u1kXrç

•

1UX61l8­1l;>3Y/N7r.<8N79GJy'.ÛV†d7rGET@.<R4i>7ƒ7r.<32FEX21l-n7p.8z¡Š79391kD3ƒ79RU7r-n79;X61l8­¨ª+iD;>Gj1l;j1kX2863?RU+8N7n3Y+;@X:7Y†dXpfe q+@/:GD82{x|c,F>14879R479-n7r;@X,39.<;JŠ7­+;V7S+¨¯X2FV7­;L.M-p79G79;X21kX214798+@¨¯X2FV7­£diV798zX:14+;+</Ê-p+@/N7Sm@79;V7?/2.<RlRkyAD.<;

79R479-n7r;@X,X2FD.0X148,3xFD.M/2.<3xX:7?/:1l8…X21436+¨BX2FV76¡D/N798z7r;>3Y76+¨».ƒG>7 D;D1kX21U+;BAD8Ni>3xF¢.<8].ƒ¡Š7?/:14+VG¯AL.ƒ¡L.M/N79;X:F>798N1l8

+@/ÊX2FV76T@7?/NJÜ_X:+pJŠ7Ý2|

c,FV7†D;D.<R„8:3Y+@/N7t+¨>.8NFV+@/zX.<;>8zq7?/v1l8BX2FV78NiD- +¨V1UX28a¡L.<8:8:.Mm@7.<;>8zq7?/v8:3Y+@/N7.<;>Gs+¨V1UX28«8:FV+@/zX.<;>8…q79/v8N3Y+@/N7|

c,FV7s8NFV+@/zXS.<;>8zq7?/Êq1UX2FX2FV7sF>1UmFV7r8…X8:3Y+@/N7´1l8Ê/N7?X:i>/:;V79GE.<8X2FV7K.<;>8…q79/ÊXN+nX2FV7s3Y+;>8:14G>7?/N79G¢£„iV798zX21U+;¯|

Þ ßà!aý"á:ùz

r)7´8:iVJD-u1kXNX:79GP+;>Rky¦+;V7´/:i>;P+¨BX2F>7KQ7rG>1U¡Š7s8zyV8zX:79- ¨ª+@/ÊX2F>7

Z

^aHtW«\…¬­[

eM®@®

‚K7YT@.<RliD.0X21U+;¯|QS;¦X2FV7 eM®@®

X:798…X6£„i>798…X214+;>8?AaQ79GD1U¡Š7ƒ/N7YX2i>/:;V79Gj.u/:14mF@X´.<;>8zq7?/­¨+</­+@;DRky e

£„iV7r8…X21U+;D8ƒf³~u+¨vX2FV7r- q7?/N7sX:79-p¡Š+@/2.<R4RUy

/N798zX:/:143xX:79G¨Ü.<3xX:+14GŠ{2AVд14;>7Y†>.<3xXÊ.<;D8…q7?/:8°.<;DG.<;ui>;D8NiV¡D¡Š+@/zX:79G.<;>8zq7?/°¨ª+@/tès+<X2FV7?/°£„i>798…X214+;>8?|CE+@/N7?+rT7?/9A

X2FV7uG>7YX:793xX214+@;Ù+¨.PRª.<3:Oj+¨Ê.<;>8…q79/´Jy Q79GD1U¡Š7pq,.<8´/:1UmFXK¨ª+@/´+;>RUy'+;V7u£diV798zX:14+;).<-p+;Vm¦X2FV7pX2FV/N7?71kX

-u.M/NO@79G¯AIqFV7?/N70.<8KX2F>1l8pRª.M32OÂFD.<GÂX:+'JŠ7¢¨ª+iD;>Gͨª+@/ eM® £„iV7r8…X21U+;>89|¤c».MJDRU7ÙåPm@1UT798ƒX2FV7PJŠ798…Xu/N798NiDRkX28n+¨

X2FV78z7YT79;)¡L.M/zX214391U¡L.<;X28´XN+¢X2FV7-n+;V+Rl14;VmiL.<RXN/x.M32O'¨+@/KW>/:79;>32FB|P[S8´q73r.<;)8z797¨ª/:+- X2F>148´Xx.MJDR47@A«X:F>7

/N798:i>RUX:8]+¨QS79G>14¡Š7fh8zyV8zXN7r- ì@{,.M/N7s;>+<X]m@+d+VGPJDiVX]X:F>7YyE.0/:7s;V+<X]X:+„+¨h.M/¨ª/N+- X2FV7s/N798NiDRkX28,+¨»FD.<R4¨v+¨«X:F>7

¡L.M/zX21l3?1U¡Š.M;X28?|

cv.MJLRU7på@² Z ^«HIW«\…¬­[ e0®@® ‚K/N798:i>RUX:8]¨ª+@/ÊX2F>7´WD/N79;>3xFP-p+;V+R41l;VmiD.<RwX:/2.<3:O

8…y>8…X:79-p8 â!/:14mF@XS.<;>8zq7?/:8 â 1l;V7Y†D.<3xXS.<;D8…q7?/28 â i>;>8:iV¡>¡Š+@/zX:79G

.<;D8…q7?/28

8N3Y+@/N76q1kX2FP£diV798zX:14+;

G>1Uêu39i>RkX±y

å å e e

~ìd|‚

e ì ® å ® ® Ð ® |kì<‚

Ð è„~ ì è å0ìd|kì<‚

è Ђ å å0‚d|e ‚

‚ Ð@Ð

eM®

Ð å0ìd|kì<‚

~ e € e å e å0‚

ã äuå æ ç èQéêŠãcë

c«+¢Xx.0O@714;X:+j.M393Y+i>;X´X2FV7¨Ü.<3YX´X2FD.0Xn.MRlR»X2F>7u£diV798zX21U+;>8KGV+E;>+<XKFD.?T7pX2FV782.<-n7uR47YT79RI+¨G>14êu3?i>RUX±yA

q76FL.?T@7´3Y+-n¡LidX:79GP.ƒ8z¡Š7r3?1kL3­8:3Y+@/N7pf·8z7?7SX2FV7´Rl.<8…X]3Y+Rli>-p;+¨«cv.MJLRU7nå0{x|Ic,FV76G>14êu3?i>RUX±y+¨«.ƒ£diV798…X214+;¦148

7YT@.<R4iL.0XN7rGjJyEX2FV7p;„i>-KJŠ7?/6+¨I8zyV8zXN7r-p8X2FD.0X´GV+P;>+<X6/N7YX2iV/:;Ù./:14m@FXs.<;>8…qÊ7?/6¨ª+@/´1kXr|r)7n3?+@-p¡DidX:79GX:F>7

-n70.M;jf·GV79;>+<XN7rG

M diff

{°.M;DGuX:F>7­8…Xx.<;>GD.M/:GGV7YT>1l.0X21U+;'fhG>79;V+<X:79G

SD diff

{+¨wX:F>7SGD1Uêu39i>RkX±yuT@.<R4iV7r8¨ª+@/°X:F>7

eM®@®

£„iV798zX21U+;>8Žìs.<;>GP8N7YX,X2FV7´8:3Y+@/N76+¨».ƒ/:1UmFXS.M;D8…q79/ÊX:+u.n£diV798zX21U+;E.<8,¨ª+RlRU+rq89²

score = 0.25 if difficulty ≤ M diff − SD diff

score = 0.5 if difficulty ≤ M diff

score = 0.75 if difficulty ≤ M diff + SD diff

score = 1 if difficulty > M diff + SD diff

c,FD148n8:3Y+@/N7¦3?+;x>/:-u8sX2FV7P¨Ü.<3xXnX2FD.0XƒX2FV7¦/N798:i>RUX:8ƒ+¨SQ7rG>1U¡w7¢.0/:7¦;>+MXnT@79/zy¨Ü.M/p¨ª/N+- X2FV7¦/N798NiDRkX28ƒ+¨

-n+8zXS+¨vX2F>7K¡L.M/zX214391U¡L.<;X28­.<;>GEX2FD.0XX2F>7Yy3Y+i>RlGEJŠ7n14-p¡>/N+rT79G£di>1kX:7n£di>1432OdRUyj.<8S1UX6-p1l8N8z7r8S8z+-n7«Ü…7r.<8…y4Ý

£„i>798…X214+;>8?|

íCîPR8zJFACHef?SM9@;COpï³lJ<WYHegWp;9HlB<gEYB4OFïz`J<M9g[[\gBB<gEYB4O<]zîzðFJYAG_Gñe;>M>fTSh ;COY³R‡lPR³*JFWFHI;>W‚gC=TSfTSgB

ò

r)7SFD.?T7¡>/N798N79;@X:79G14;pX2F>1l8.0/NX:1l3?R47]X:F>7,T7?/:8:1U+;u+¨ŠX2FV76Q7rG>1U¡Š7S8zyV8zXN7r- X2FD.0X°¡Š.0/NX:1l3?14¡L.0X:79GpX:+´X2FV7SWD/N79;>3xF

-n+;>+@Rl14;>miD.<R»XN/x.M32O+¨tX2FV7

Z

^«HtWa\z¬S[

eM®@®

‚7YT@.<R4iD.0X214+@;B|u`_X286/:798NiDRkX28s.M/N7;V+<X´T@79/zyÙF>1UmFÙJDidXsX2FV7Yy).M/N7

3Y+FV79/N79;@Xsq1kX2F'X2FV7GV7?m@/N797u+@¨8:14-p¡DR41l3?1UX±y'+¨tX2FV78…y>8…X:79-P|[ £di>1l3:O).M;L.<Rky>8N148s+@¨1UX28´/N798NiDRkX28s8NF>+rq8­X2FD.0X

8NiD32FP.K8:14-p¡DRU768zyV8…X:79- 39.<;JŠ768NiVêu391U79;XX:+p.<;D8…q7?/X:+p.M/N+i>;>G eM®°÷

+¨a¨Ü.<3YXN+1lG¦£„iV7r8…X21U+;D8°JLidX]148°X:+<Xx.MRlRky

14;>7?êu3?1479;@XK¨ª+@/K.<;>8…q79/:14;>m¦X:+P-p+@/N7p3Y+-n¡LRU7Y†'£diV798…X214+;>8´8Ni>3xFÙ.<8´GV7 L;>1kX214+;Ù£„iV7r8…X21U+;>89|nÏ79;D3Y7@A¯qÊ7ƒq1lR4R

¨ª+V3?iD8+iV/­¨ÜidX2iV/N7sq+@/NO¢+;¢X2FD.0X´.<8z¡Š793xXr|­[ 8:FV+@/zXN\·X:7?/:- 14-p¡>/N+rT79-n79;X´.MJŠ+idX­1UX63Y+i>RlGEJŠ7KX:+-.0O@7ƒiD8z7

+¨BX2FV7s39.M¡Š.0JL14Rl1kX214798,+¨«^a`…C[ .MJŠ+idX8…y>;@Xx.<3xX2143s.<;D.<RUyV8N1l8]¨ª+@/]G>79R41l-p1UX:1l;Vm8NFV+@/zXS.<;>8…qÊ7?/:8,1l;>8…X:70.MGP+¨vi>8N1l;Vm

.<;E.M¡D¡>/N+r†>14-u.0X:7´¡L.0XNX:7?/:;¯|°`…;.p-n+@/N7sR4+@;>m<\·X:7?/:- ¡LRl.<;¯ADq76q+i>RlGER414O<7´X:+u79Rª.MJŠ+@/2.0X:7K.<;¢1l;>8…Xx.<;>3Y7?\³JL.<8z7rG

.M¡>¡>/:+.<3xFE¨ª+@/7Y†„X:/2.<3xX21l;Vm8:FV+@/zX­.<;>8zq7?/:89A>qF>1l32F3Y+i>R4Gj.?T@+1lG¢JDi>1lR4GD14;Vm+¨».u8z7?X+¨»-u.<;„iL.<RB¡L.0XNX:7?/:;>8S.<8

1kX1l8,+¨lX:79;¢GV+;V7@|

ßà!F,ú]ø¦þ†!

Ñ4åYä Z FV/:1l8…X:79RlRU7¦[»y„.<32F>7@| Z .M-p¡L.Mm;V7H¶[^«í][«øMH°¬mSHI* ²vHT@.<R4iD.0X214+;)7r; £diV798…X214+;d\³/0ù?¡w+@;D8z7@A»/2.M¡>¡w+</NX

D;L.MRBGV7sRª.n39.<-n¡L.Mm;V7sH»¶[S^aí][«øMH¬SHt*´|Ica7r32F>;D1439.<R¯/:7?¡Š+@/zXrA>Ht^aí][KA eM®@® ‚d|

Ñe äs[S;@X:+;>14+'5Ê.<RUT@7YXrAIC79F>G>1Ht-KJL.M/N7?OwAI.<;>G§QSRl1kTV147?/pWD7?/N/N7YXr|'Cj14;>1l-u.<Rl148:-n77YXp£diV798…X214+;d\³/0ù?¡w+@;D8z7'²uR47

8…y>8…XŽú9-p7ÙQ79GD1U¡Š7@| `…;üû

 X™T—C˜{¯°—{¡9ý

û

™T— ¡iš?—¤þ«ÿ\–c—¯°—{¡k¢  G›pœX£¤Ž—œR C—

û|¬ @AÊ¡Š.Mm@798Pì@ì ® A

íS+iV/:GL.M;BA>WD/2.<;>3?7@AuÙ@i>;>7 eM®@® ‚d|

ÑÐ0äs*,+-.M/:143ƒ5798:.<;>=?+@;Ù.<;>G6.9R

Z

FD.<R479;>GD.M/ufhG>70{x|s^.<;D.<Rky>8z79i>/68…y>;@Xx.0†V1l£„i>7pGV7n^a`…C[GD.<;>86Rl.¦39.<-ƒ\

¡L.Mm;V7sGùYT@.<R4iL.0X:14+;¢Ht[6b4“n|I`…;û

 X™T—C˜,¯°—¡k¢

—’ G›pœX£¤0—œR C—¢pœcœu–c— ¡Š¡$—«˜C–x¤¡Ã—x¤0¢pše™T—w—œu™

û

–x™?›w¢

™ešI• –c—¯°—C˜

¬

¢pœp–c—C˜

¢‚™e–x¤Ž— ¡Ç¡Ã—C˜! "

û|¬#<A>íS+iV/2GD.<;¯ADWD/2.<;>3Y7@AuÙi>;V7 e0®@®

‚d|

Ñè<äs*,+-.M/:143¢5°7982.<;>=Y+;¯A$6.9R

Z

FL.<RU79;DGD.M/fhG>70{xA,QSRl1kT>1U7?/WD7?/N/N7?X9A

Z

FV/:148zX21l.<;§W»Rli>FV/9A]QSRl1kT>1U7?/CE798:;D.M/:G¯A

.<;>G ÏSiVJŠ79/zXÆ­.M7YX28?|&%

th

^' ›p¤(<˜)x›Cž ›9£®™*)V—+¤0›Y˜G˜,

¬

¢pœp–V¢-°—Æþ/.F¢‚¡×–c¢‚™eše›pœ10›p¤C–2 3+

¬

þ40 5@A

32FL.M¡VX:7?/

Z

+;>3?7?¡VXN\³JL.<8N79G¤b„7r.M/:32FD14;Vm'.<;>G CE7?/Nm1l;Vm¨+</ƒCi>RUX214Rl14;VmiL.<RI`…;>¨+@/:-u.0X21U+;)*]7YX:/:1U7YT@.<R·²ƒW»14/:8…X

H†d¡Š79/:14-p79;@X28].0X Z ^«HIW eM®@® ÐVAV¡Š.0m@7r86å0ì0èŠå

èD| e0®@® èD|

Ñ‚räs*,+-.M/:143ƒ5798:.<;D=Y+;¯A«QSRl1kT>1U79/´W>79/N/N7YXrAa.<;>G Z FV/:1l8…X21l.<;'W»R4iDFV/9|K^B` Z,e C¥7Y†d¡Š79/:14-n7r;@X28´.0X Z ^aHtW eM®@® èD|

`…;

th

^' ›p¤(<˜)x›Cžƒ›9£™*)V—6+¤0›Y˜G˜,

¬

¢pœp–V¢-°—þ7.Y¢‚¡×–V¢‚™ešI›pœ0l›p¤C–8 9+

¬

þ0

%DA eM®@® è>|

Ñ~0äƒQSRl1kT>1U79/W>79/N/N7YXrA5/:1Um1UXNXN7:S/2.<i¯A Cj.M/zX214;>7 ÏSiV/2.<iDRkXN\_}tRl.<;XN7?XrA;6.MJ>/21U79R¤`…RlRU+iVµA .<;DG Z F>/:148zX:1ª.<;

Ù.<3?£diV79-u14;B|ÍíS+d3?iD-n79;X8z79R4793xX21U+;Í/N7 L;V79-p79;@XuJL.<8N79G¤+;ÍRl14;>m@iD148zX:1l3¢¨ª7r.0X2iV/N7r8p¨ª+@/¬­[^ Z At.j£diV798z\

X21U+;Ù.<;>8…q79/:14;>mP8…yV8zX:79-P|6`…;

#—Ž C—œu™

û

¯.Y¢pœR C—C˜šŠœ

¢‚™e–Q¤0¢‚¡

¬

¢pœp–c¢-°—=<¤0›< G—C˜G˜CšŠœ?>*

û@!¬

<

A

A

eM®@® å@|

ÑìräsíS+-p1l;>1l£„iV7­^«.<i>/N79;@X].M;DG}.rX:/:1l3:ObQù?mixù9Rª.V|v¬S*]`Nbdcv[S^IAd8…y>8…XŽú9-p7SGV7­£„i>798…X214+@;D8…\³/0ù9¡Š+;>8z7r8?|B`…;

û|¬

<A>¡L.Mm@798‚<Ð-V~

e A eM®@®

‚„|

Ñ äsíƒ|LCE+R4GV+0T<.<;¯Awbw|DÏ­.M/2.MJL.Mm14iBAD*´|B­1U/?҅i¯AL}v|LCE+@/2.M/N798:3?i¯AD[K|c+rT>148:32F>1·AL[K|D5,.MGL.<RU798:3?i¯AŠ.<;>GQƒ|D5+RU+<\

FD.<;¯|I^«3?3SX:+d+R48,¨+@/]£„i>798…X214+@;E.<;>8zq7?/:1l;VmD|`…;

#þC+

„A>;„iD-sJŠ7?/

eM®<®

ÐV|

р0äsCE7r-p;V+6T@.<;ED¯.@.<;V79;¯|F

›<›Y˜™e¤0¢Cž4žRšÇœHGR™e¤C–c X™e–Q¤Ž—|šÇœR™?›

¬

¢pœp–c¢-°—-I

û

¡išJpœ€—w—œu™*

F

¢Y˜X—Ž¯

¬

—Ž¢p¤CœcšŠœ

|w}tF>í

X2FV798:1489AVS;>1UT7?/:8N1UX±y¦+¨«^a7?79G>89A eM®@® å@|