Appendix A: Results of the Monolingual. Bilingual, Multilingual and Domain-Specific Tracks: Run Statistics

Université de Neuchâtel Switzerland UniNEru2 Mono RU TD Auto

Université de Neuchâtel Switzerland UniNEru3 Mono RU TDN Auto Y

Thomson Legal USA tlrde2fr1 Bi-FR DE TD Auto Y

Thomson Legal USA tlrde2fr2 Bi-FR DE TD Auto

Thomson Legal USA tlrde2fr3 Bi-FR DE TD Auto

Thomson Legal USA tlrde2fr4 Bi-FR DE TD Auto

Thomson Legal USA tlrfi1 Mono FI TD Auto Y

Thomson Legal USA tlrfi2 Mono FI TD Auto

Thomson Legal USA tlrpt1 Mono PT TD Auto Y

Thomson Legal USA tlrpt2 Mono PT TD Auto

Thomson Legal USA tlrru1 Mono RU TD Auto Y

Thomson Legal USA tlrru2 Mono RU TD Auto

SICS Sweden stockholmAmEnA Bi-EN AM TD Manual Y

SICS Sweden stockholmAmEnI Bi-EN AM TD Manual Y

SICS Sweden SICSsvfrQF Bi-FR SV TD Auto Y

CEA/LIC2M France lic2men1 Multi EN TD Auto Y

CEA/LIC2M France lic2men2 Multi EN TDN Auto

CEA/LIC2M France lic2mfr1 Multi FR TDN Auto Y

Universidad de Evora Portugal alentejo1 Mono PT T Auto Y

Universidad de Evora Portugal alentejo2 Mono PT T Auto Y

ILTG Canada ltrcbifren1 Bi-EN FR TDN Auto Y

ILTG Canada ltrcbifren2 Bi-EN FR TDN Auto Y

ILTG Canada ltrcbifren3 Bi-EN FR TDN Auto Y

ILTG Canada ltrcbifren4 Bi-EN FR TDN Auto Y

ILTG Canada ltrcbifren5 Bi-EN FR TDN Auto Y

ILTG Canada ltrcmonofr1 Mono FR TDN Auto

ILTG Canada ltrcmonofr2 Mono FR TDN Auto

ILTG Canada ltrcmonofr3 Mono FR TDN Auto Y

ILTG Canada ltrcmonofr4 Mono FR TDN Auto

U Jaen/SINAI Spain UJAMLRL Multi EN TD Auto

U Jaen/SINAI Spain UJAMLRL2P Multi EN TD Auto Y

U Jaen/SINAI Spain UJAMLRL3P Multi EN TD Auto Y

U Jaen/SINAI Spain UJAMLRLRP Multi EN TD Auto

U Jaen/SINAI Spain UJAMLRSV2 Multi EN TD Auto Y

CLIPS France CLIPS04ENFI1 Bi-FI EN TDN Auto Y

CLIPS France CLIPS04ENRU1 Bi-RU EN TDN Auto

CLIPS France CLIPS04ENRU2 Bi-RU EN TDN Auto

CLIPS France CLIPS04FRFI1 Bi-FI FR TDN Auto Y

CLIPS France CLIPS04FRRU1 Bi-RU FR TDN Auto

CLIPS France CLIPS04FRRU2 Bi-RU FR TDN Auto

CLIPS France CLIPS04MNFI1 Mono FI TDN Auto Y

CLIPS France CLIPS04MNFR1 Mono FR TDN Auto Y

CLIPS France CLIPS04MNRU1 Mono RU TDN Auto Y

Hummingbird Canada humFI04t Mono FI T Auto Y

Hummingbird Canada humFI04td Mono FI TD Auto Y

Hummingbird Canada humFI04tde Mono FI TD Auto

Hummingbird Canada humFR04t Mono FR T Auto Y

Hummingbird Canada humFR04td Mono FR TD Auto

Hummingbird Canada humFR04tde Mono FR TD Auto

Hummingbird Canada humPT04t Mono PT T Auto Y

Hummingbird Canada humPT04td Mono PT TD Auto Y

Hummingbird Canada humPT04tde Mono PT TD Auto

U Amsterdam Netherlands UAmsC04FiFi4GiWd Mono FI TD Auto

U Amsterdam Netherlands UAmsC04FiFiAll Mono FI TD Auto

U Amsterdam Netherlands UAmsC04FrFr4GiSb Mono FR TD Auto Y

U Amsterdam Netherlands UAmsC04FrFr4GiWd Mono FR TD Auto

U Amsterdam Netherlands UAmsC04FrFrAll Mono FR TD Auto

U Amsterdam Netherlands UAmsC04PoPo4GiSb Mono PT TD Auto Y

U Amsterdam Netherlands UAmsC04PoPo4GiWd Mono PT TD Auto Y

U Amsterdam Netherlands UAmsC04PoPoAll Mono PT TD Auto

U Amsterdam Netherlands UAmsC04RuRu4GiSb Mono RU TD Auto Y

U Amsterdam Netherlands UAmsC04RuRu4GiWd Mono RU TD Auto

U Amsterdam Netherlands UAmsC04RuRuAll Mono RU TD Auto

FU Hagen Germany FUHdm GIRT-DE DE TD Auto Y

FU Hagen Germany FUHdrw GIRT-DE DE TD Auto Y

FU Hagen Germany FUHds1 GIRT-DE DE TD Auto Y

FU Hagen Germany FUHdw1 GIRT-DE DE TD Auto Y

FU Hagen Germany FUHdw2 GIRT-DE DE TD Auto Y

FU Hagen Germany FUHe1 GIRT-EN DE TD Auto Y

FU Hagen Germany FUHe2 GIRT-EN DE TD Auto Y

FU Hagen Germany FUHe3 GIRT-EN DE TD Auto Y

FU Hagen Germany FUHe4 GIRT-EN DE TD Auto Y

FU Hagen Germany FUHe5 GIRT-EN DE TD Auto Y

Daedalus Spain de2frR2FWnormal Bi-FR DE TD Auto

Daedalus Spain de2frR3FWnormal Bi-FR DE TD Auto Y

Daedalus Spain de2frcomb1s101 Bi-FR DE TD Auto

Daedalus Spain de2frcomb2s201 Bi-FR DE TD Auto

Daedalus Spain frav7 Mono FR TD Auto Y

Daedalus Spain frav8 Mono FR TD Auto

Daedalus Spain frcomb1s201 Mono FR TD Auto

Daedalus Spain frcomb2s201 Mono FR TD Auto

Daedalus Spain nl2frR4FWnormal Bi-FR NL TD Auto Y

Daedalus Spain nl2frR5FWnormal Bi-FR NL TD Auto

Daedalus Spain nl2frav Bi-FR NL TD Auto

Daedalus Spain nl2frcomb1s101 Bi-FR NL TD Auto

Daedalus Spain ruR1FWnormal Mono RU TD Auto Y

Daedalus Spain ruav7 Mono RU TD Auto

Daedalus Spain ruav8 Mono RU TD Auto Y

Daedalus Spain rucomb1s101 Mono RU TD Auto

U Padua Italy IMSBDEFRN100 Bi-FR DE TD Auto

U Padua Italy IMSBDEFRN50 Bi-FR DE TD Auto

U Padua Italy IMSBDEFRP100 Bi-FR DE TD Auto

U Padua Italy IMSBDEFRP50 Bi-FR DE TD Auto

U Padua Italy IMSBDEFRUd651 Bi-FR DE TD Auto

U Padua Italy IMSBDEFRUd655 Bi-FR DE TD Auto Y

U Padua Italy IMSMFIN Mono FI TD Auto

U Padua Italy IMSMFIP Mono FI TD Auto

U Padua Italy IMSMFIPqe Mono FI TD Auto

U Padua Italy IMSMFIUa65 Mono FI TD Auto Y

U Padua Italy IMSMFIUd65 Mono FI TD Auto Y

U Padua Italy IMSMFRN Mono FR TD Auto

U Padua Italy IMSMFRP Mono FR TD Auto

U Padua Italy IMSMFRPqe Mono FR TD Auto

U Padua Italy IMSMFRUd65 Mono FR TD Auto Y

U Padua Italy IMSMRUN Mono RU TD Auto Y

U Alicante Spain IRn-MF-Pexp Mono FI TD Auto

U Alicante Spain IRn-MF-nexp Mono FI TD Auto

U Alicante Spain IRn-MFr-Dexp Mono FR TD Auto Y

U Alicante Spain IRn-MFr-Pexp Mono FR TD Auto

U Alicante Spain IRn-MFr-nexp Mono FR TD Auto

U Alicante Spain IRn-Mu-Pexp Multi EN TD Auto Y

U Alicante Spain IRn-Mu-Dexp Multi EN TD Auto Y

U Alicante Spain IRn-Mu-nexp Multi EN TD Auto

U Alicante Spain IRn-MP-nexp Mono PT TD Auto Y

NII Group Japan NiiDic03 Bi-FR DE TD Auto

NII Group Japan NiiDic04 Bi-FR DE TD Auto

NII Group Japan NiiDic05 Bi-FR DE TD Auto

NII Group Japan NiiFF01 Mono FR TD Auto Y

NII Group Japan NiiMt02 Bi-FR DE TD Auto Y

U Hildesheim Germany UHIenru1 Bi-RU EN TD Auto

U Hildesheim Germany UHIenru2 Bi-RU EN TD Auto

U Hildesheim Germany UHIenru3 Bi-RU EN TD Auto

U Hildesheim Germany UHIfi1 Mono FI TD Auto Y

U Hildesheim Germany UHIfi2 Mono FI TD Auto Y

U Hildesheim Germany UHImlt1 Multi EN TD Auto Y

U Hildesheim Germany UHImlt2 Multi EN TD Auto Y

U Hildesheim Germany UHIru1 Mono RU TD Auto Y

U Hildesheim Germany UHIru2 Mono RU TD Auto

U Hildesheim Germany UHIru3 Mono RU TD Auto

FCUL Lisbon Portugal XLDBTumba01 Mono PT missing Manual Y

FCUL Lisbon Portugal XLDBTumba02 Mono PT missing Manual Y

FCUL Lisbon Portugal XLDBTumba04 Mono PT missing Manual

FCUL Lisbon Portugal XLDBTumba05 Mono PT missing Manual

IRIT France run1T GIRT-EN EN T Auto Y

IRIT France run2TD GIRT-EN EN TD Auto Y

IRIT France run3TDfc GIRT-EN EN TD Auto Y

IRIT France run4TDWN GIRT-EN EN TD Auto Y

IRIT France run4TWN GIRT-EN EN T Auto Y

UC Berkeley USA BKGRBLEG1 GIRT-DE EN TD Auto Y

UC Berkeley USA BKGRBLEG2 GIRT-DE EN TD Auto Y

UC Berkeley USA BKGRBLEG3 GIRT-DE EN TD Auto Y

UC Berkeley USA BKGRBLEG4 GIRT-DE EN TD Auto Y

UC Berkeley USA BKGRBLEG5 GIRT-DE EN TD Auto Y

UC Berkeley USA BKGRBLGE1 GIRT-EN DE TD Auto Y

UC Berkeley USA BKGRBLGE2 GIRT-EN DE TD Auto Y

UC Berkeley USA BKGRBLGE3 GIRT-EN DE TD Auto Y

UC Berkeley USA BKGRBLGE4 GIRT-EN DE TD Auto Y

UC Berkeley USA BKGRBLGE5 GIRT-EN DE TD Auto Y

UC Berkeley USA BKGRMLEE1 GIRT-EN EN TD Auto Y

UC Berkeley USA BKGRMLEE2 GIRT-EN EN TD Auto Y

UC Berkeley USA BKGRMLGG0 GIRT-DE DE TD Auto Y

UC Berkeley USA BKGRMLGG1 GIRT-DE DE TD Auto Y

UC Berkeley USA BKGRMLGG2 GIRT-DE DE TD Auto Y

UC Berkeley USA BKRUBLER1 Bi-RU EN TDN Auto

UC Berkeley USA BKRUBLER2 Bi-RU EN TDN Auto

UC Berkeley USA BKRUBLER3 Bi-RU EN TD Auto

UC Berkeley USA BKRUBLER4 Bi-RU EN TDN Auto Y

UC Berkeley USA BKRUBLJR1 Bi-RU JP TD Auto

Dublin City University Ireland exemult1s Multi EN TD Auto Y

Dublin City University Ireland exemult2s Multi EN TD Auto Y

Dublin City University Ireland exemult3s Multi EN TD Auto Y

Dublin City University Ireland exemult4s Multi EN TD Auto

Dublin City University Ireland exemult5s Multi EN TD Auto

Dublin City University Ireland exerussbi04a Bi-RU EN TD Auto Y

Dublin City University Ireland exerussbi04l Bi-RU EN TD Auto

Dublin City University Ireland exerussbi04q Bi-RU EN TD Auto

Dublin City University Ireland exerussbi04t Bi-RU EN TD Auto

Dublin City University Ireland exerussmono04a Mono RU TD Auto Y

Dublin City University Ireland exerussmono04l Mono RU TD Auto

Dublin City University Ireland exerussmono04q Mono RU TD Auto

Dublin City University Ireland exerussmono04t1 Mono RU TD Auto

Dublin City University Ireland exerussbi04amg Bi-RU EN TD Auto

Ricoh Japan rdedetde04 GIRT-DE DE TD Auto Y

Ricoh Japan rendetde04 GIRT-DE EN TD Auto Y

SUNY Buffalo USA UBmonoEN01 Mono EN TD Auto Y

SUNY Buffalo USA UBmonoFI01 Mono FI TD Auto Y

SUNY Buffalo USA UBmonoFR01 Mono FR TD Auto Y

SUNY Buffalo USA UBmonoRU01 Mono RU TD Auto Y

SUNY Buffalo USA UBmult01 Multi EN TD Auto Y

SUNY Buffalo USA UBmult02 Multi EN TD Auto Y

SUNY Buffalo USA UBmulti03 Multi EN TD Auto

JHU/APL USA aplbibgrub Bi-RU BG TD Auto Y

JHU/APL USA aplbidefra Bi-FR DE TD Auto Y

JHU/APL USA aplbienpta Bi-PT EN TD Auto Y

JHU/APL USA aplbiesfia Bi-FI ES TD Auto Y

JHU/APL USA aplbiespta Bi-PT ES TD Auto Y

JHU/APL USA aplbiesptb Bi-PT ES TD Auto Y

JHU/APL USA aplbifrfia Bi-FI FR TD Auto

JHU/APL USA aplbinlfra Bi-FR NL TD Auto Y

JHU/APL USA aplmofia Mono FI TD Auto Y

JHU/APL USA aplmofib Mono FI TD Auto

JHU/APL USA aplmofic Mono FI TD Auto

JHU/APL USA aplmofra Mono FR TD Auto Y

JHU/APL USA aplmofrb Mono FR TD Auto

JHU/APL USA aplmofrc Mono FR TD Auto

JHU/APL USA aplmopta Mono PT TD Auto Y

JHU/APL USA aplmoptb Mono PT TD Auto Y

JHU/APL USA aplmoptc Mono PT TD Auto

JHU/APL USA aplmorua Mono RU TD Auto Y

JHU/APL USA aplmorub Mono RU TD Auto

JHU/APL USA aplmoruc Mono RU TD Auto

U Chicago USA UCEMULT2 Multi EN T Auto Y

U Chicago USA UCEMULTD1 Multi EN TD Auto Y

U Chicago USA UCEMULdecT3 Multi EN T Auto

U Chicago USA UCEMULlppT4 Multi EN T Auto

U Chicago USA UCERBiT1 Bi-RU EN T Auto Y

U Glasgow UK UOGLQ Mono FR TDN Auto

U Glasgow UK UOGLQQE Mono FR TDN Auto Y

U Glasgow UK UOGSQ Mono FR T Auto

U Glasgow UK UOGSQQE Mono FR T Auto

0.0 0.2 0.4 0.6 Recall

0.0

0.2

0.4

0.6

0.8 1.0

Precision

0.3427 U Amsterdam 0.3341 U Jaen/SINAI 0.3090 U Neuchâtel 0.2748 SUNY Buffalo 0.2745 Dublin City U

Multilingual; Recall−Precision Graph Top performing experiments by the best five groups; TD topic fields; Automatic

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.2982 JHU/APL

Bilingual to FI; Recall−Precision Graph

Top performing experiments by the best five groups; TD topic fields; Automatic

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.4704 U Neuchâtel

0.4537 JHU/APL

0.4366 U Amsterdam

0.3243 U Alicante

Bilingual to PT; Recall−Precision Graph

Top performing experiments by the best five groups; TD topic fields; Automatic

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.6367 Hummingbird 0.5562 Thomson Legal 0.5453 U Neuchâtel 0.5443 JHU/APL 0.5203 U Amsterdam

Monolingual FI; Recall−Precision Graph

Top performing experiments by the best five groups; TD topic fields; Automatic

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.5188 U Neuchâtel 0.4750 Hummingbird 0.4691 JHU/APL 0.4469 Thomson Legal 0.4326 U Amsterdam

Monolingual PT; Recall−Precision Graph

Top performing experiments by the best five groups; TD topic fields; Automatic

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.4053 UC Berkeley

0.3855 IRIT

0.0369 FU Hagen

GIRT to EN; Recall−Precision Graph

Top performing experiments by the best five groups; TD topic fields; Automatic

Dublin City University, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0168 Query 202: 0.6188 Query 203: 0.2400 Query 204: 0.3655 Query 205: 0.2451 Query 206: 0.4209 Query 207: 0.1509 Query 208: 0.3036 Query 209: 0.0644 Query 210: 0.1058 Query 211: 0.8375 Query 212: 0.1212 Query 213: 0.5757 Query 214: 0.0001 Query 215: 0.0314 Query 216: 0.0030 Query 217: 0.2025 Query 218: 0.8338 Query 219: 0.0063 Query 220: 0.0344 Query 221: 0.4257 Query 222: 0.0263 Query 223: 0.0907 Query 224: 0.8160 Query 225: 0.5328 Query 226: 0.0054 Query 227: 0.0094 Query 228: 0.2341 Query 229: 0.3706 Query 230: 0.6629 Query 231: 0.1193 Query 232: 0.0179 Query 233: 0.4403 Query 234: 0.0183 Query 235: 0.0045 Query 236: 0.2464 Query 237: 0.5349 Query 238: 0.3109 Query 239: 0.0474 Query 240: 0.8048 Query 241: 0.0124 Query 242: 0.1766 Query 243: 0.2872 Query 244: 0.3472 Query 245: 0.6897 Query 246: 0.0858 Query 247: 0.2498 Query 248: 0.3241 Query 249: 0.0162 Query 250: 0.0425 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1244 Interpolated Recall - Precision Averages: at 0.00 0.5587 at 0.10 0.4829 at 0.20 0.4151 at 0.30 0.3570 at 0.40 0.2961 at 0.50 0.2474 at 0.60 0.2115 at 0.70 0.1790 at 0.80 0.1360 at 0.90 0.0971 at 1.00 0.0290 Avg. prec. (non-interpolated) for all rel. documents 0.2626 Precision: At 5 docs: 0.3880 At 10 docs: 0.3540 At 15 docs: 0.3160 At 20 docs: 0.3020 At 30 docs: 0.2580 At 100 docs: 0.1572 At 200 docs: 0.0987 At 500 docs: 0.0464 At 1000 docs: 0.0249 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2747

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run exemult1s

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run exemult1s

Comparison to median by topic

Dublin City University, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0389 Query 202: 0.6315 Query 203: 0.0258 Query 204: 0.2331 Query 205: 0.2408 Query 206: 0.5065 Query 207: 0.0915 Query 208: 0.2007 Query 209: 0.0644 Query 210: 0.1035 Query 211: 0.7854 Query 212: 0.1190 Query 213: 0.5328 Query 214: 0.0002 Query 215: 0.0227 Query 216: 0.0246 Query 217: 0.1477 Query 218: 0.6926 Query 219: 0.0037 Query 220: 0.0005 Query 221: 0.6059 Query 222: 0.0263 Query 223: 0.0867 Query 224: 0.8039 Query 225: 0.5447 Query 226: 0.0069 Query 227: 0.0094 Query 228: 0.0008 Query 229: 0.0132 Query 230: 0.5924 Query 231: 0.1155 Query 232: 0.0159 Query 233: 0.3669 Query 234: 0.1750 Query 235: 0.0045 Query 236: 0.7253 Query 237: 0.5624 Query 238: 0.3764 Query 239: 0.0221 Query 240: 0.8510 Query 241: 0.0136 Query 242: 0.1002 Query 243: 0.2701 Query 244: 0.3430 Query 245: 0.5299 Query 246: 0.1103 Query 247: 0.2793 Query 248: 0.3674 Query 249: 0.0088 Query 250: 0.0000 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1119 Interpolated Recall - Precision Averages: at 0.00 0.5510 at 0.10 0.4363 at 0.20 0.3703 at 0.30 0.3252 at 0.40 0.2659 at 0.50 0.2439 at 0.60 0.2217 at 0.70 0.1866 at 0.80 0.1453 at 0.90 0.1023 at 1.00 0.0438 Avg. prec. (non-interpolated) for all rel. documents 0.2479 Precision: At 5 docs: 0.3600 At 10 docs: 0.3300 At 15 docs: 0.3107 At 20 docs: 0.2920 At 30 docs: 0.2540 At 100 docs: 0.1478 At 200 docs: 0.0894 At 500 docs: 0.0413 At 1000 docs: 0.0224 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2786

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run exemult2s

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run exemult2s

Comparison to median by topic

Dublin City University, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0397 Query 202: 0.6374 Query 203: 0.0656 Query 204: 0.6372 Query 205: 0.2313 Query 206: 0.4287 Query 207: 0.1551 Query 208: 0.2473 Query 209: 0.1066 Query 210: 0.1024 Query 211: 0.7800 Query 212: 0.0759 Query 213: 0.5223 Query 214: 0.0009 Query 215: 0.0227 Query 216: 0.0326 Query 217: 0.1477 Query 218: 0.6035 Query 219: 0.0038 Query 220: 0.0006 Query 221: 0.6569 Query 222: 0.0178 Query 223: 0.0885 Query 224: 0.7901 Query 225: 0.5449 Query 226: 0.0068 Query 227: 0.0094 Query 228: 0.0475 Query 229: 0.0388 Query 230: 0.5496 Query 231: 0.2023 Query 232: 0.0445 Query 233: 0.4629 Query 234: 0.2281 Query 235: 0.0016 Query 236: 0.7983 Query 237: 0.5536 Query 238: 0.4027 Query 239: 0.0398 Query 240: 0.8643 Query 241: 0.0152 Query 242: 0.1056 Query 243: 0.2418 Query 244: 0.3476 Query 245: 0.7625 Query 246: 0.0998 Query 247: 0.4108 Query 248: 0.3828 Query 249: 0.0188 Query 250: 0.0000 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1232 Interpolated Recall - Precision Averages: at 0.00 0.6192 at 0.10 0.4659 at 0.20 0.4110 at 0.30 0.3579 at 0.40 0.2868 at 0.50 0.2677 at 0.60 0.2363 at 0.70 0.2075 at 0.80 0.1602 at 0.90 0.1190 at 1.00 0.0459 Avg. prec. (non-interpolated) for all rel. documents 0.2715 Precision: At 5 docs: 0.3720 At 10 docs: 0.3500 At 15 docs: 0.3280 At 20 docs: 0.3160 At 30 docs: 0.2667 At 100 docs: 0.1542 At 200 docs: 0.0967 At 500 docs: 0.0460 At 1000 docs: 0.0246 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2846

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run exemult3s

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run exemult3s

Comparison to median by topic

Dublin City University, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0397 Query 202: 0.6388 Query 203: 0.0671 Query 204: 0.5573 Query 205: 0.2668 Query 206: 0.3837 Query 207: 0.0033 Query 208: 0.2699 Query 209: 0.1136 Query 210: 0.1037 Query 211: 0.7851 Query 212: 0.1154 Query 213: 0.5111 Query 214: 0.0016 Query 215: 0.0202 Query 216: 0.0479 Query 217: 0.1104 Query 218: 0.6587 Query 219: 0.0063 Query 220: 0.0003 Query 221: 0.6777 Query 222: 0.0372 Query 223: 0.0845 Query 224: 0.7686 Query 225: 0.5465 Query 226: 0.0081 Query 227: 0.0094 Query 228: 0.0916 Query 229: 0.0342 Query 230: 0.5432 Query 231: 0.1234 Query 232: 0.0311 Query 233: 0.4552 Query 234: 0.2431 Query 235: 0.0013 Query 236: 0.7808 Query 237: 0.6123 Query 238: 0.3304 Query 239: 0.0485 Query 240: 0.8520 Query 241: 0.0160 Query 242: 0.1787 Query 243: 0.2287 Query 244: 0.3536 Query 245: 0.7939 Query 246: 0.1436 Query 247: 0.4594 Query 248: 0.3915 Query 249: 0.0783 Query 250: 0.0000 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1244 Interpolated Recall - Precision Averages: at 0.00 0.6117 at 0.10 0.4712 at 0.20 0.3996 at 0.30 0.3500 at 0.40 0.2946 at 0.50 0.2730 at 0.60 0.2378 at 0.70 0.2049 at 0.80 0.1586 at 0.90 0.1157 at 1.00 0.0442 Avg. prec. (non-interpolated) for all rel. documents 0.2725 Precision: At 5 docs: 0.3800 At 10 docs: 0.3520 At 15 docs: 0.3267 At 20 docs: 0.3060 At 30 docs: 0.2633 At 100 docs: 0.1546 At 200 docs: 0.0973 At 500 docs: 0.0457 At 1000 docs: 0.0249 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2937

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run exemult4s

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run exemult4s

Comparison to median by topic

Dublin City University, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0495 Query 202: 0.6620 Query 203: 0.0657 Query 204: 0.6505 Query 205: 0.2406 Query 206: 0.4997 Query 207: 0.1571 Query 208: 0.2749 Query 209: 0.1032 Query 210: 0.1085 Query 211: 0.7785 Query 212: 0.0717 Query 213: 0.5326 Query 214: 0.0006 Query 215: 0.0245 Query 216: 0.0484 Query 217: 0.1496 Query 218: 0.6893 Query 219: 0.0040 Query 220: 0.0007 Query 221: 0.6689 Query 222: 0.0139 Query 223: 0.0907 Query 224: 0.7678 Query 225: 0.5390 Query 226: 0.0067 Query 227: 0.0094 Query 228: 0.0114 Query 229: 0.0190 Query 230: 0.5848 Query 231: 0.1948 Query 232: 0.0635 Query 233: 0.4645 Query 234: 0.1644 Query 235: 0.0014 Query 236: 0.7121 Query 237: 0.5591 Query 238: 0.4019 Query 239: 0.0285 Query 240: 0.9038 Query 241: 0.0133 Query 242: 0.0986 Query 243: 0.2635 Query 244: 0.3437 Query 245: 0.7542 Query 246: 0.0913 Query 247: 0.4304 Query 248: 0.3951 Query 249: 0.0191 Query 250: 0.0000 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1216 Interpolated Recall - Precision Averages: at 0.00 0.5964 at 0.10 0.4770 at 0.20 0.3993 at 0.30 0.3567 at 0.40 0.2915 at 0.50 0.2732 at 0.60 0.2468 at 0.70 0.2153 at 0.80 0.1639 at 0.90 0.1271 at 1.00 0.0520 Avg. prec. (non-interpolated) for all rel. documents 0.2745 Precision: At 5 docs: 0.3640 At 10 docs: 0.3560 At 15 docs: 0.3307 At 20 docs: 0.3150 At 30 docs: 0.2753 At 100 docs: 0.1566 At 200 docs: 0.0976 At 500 docs: 0.0458 At 1000 docs: 0.0243 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2890

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run exemult5s

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run exemult5s

Comparison to median by topic

U Alicante, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0591 Query 202: 0.4645 Query 203: 0.5426 Query 204: 0.3881 Query 205: 0.4279 Query 206: 0.1440 Query 207: 0.1457 Query 208: 0.1731 Query 209: 0.0557 Query 210: 0.1748 Query 211: 0.7880 Query 212: 0.0570 Query 213: 0.4545 Query 214: 0.0463 Query 215: 0.0555 Query 216: 0.0433 Query 217: 0.1386 Query 218: 0.8496 Query 219: 0.0451 Query 220: 0.0018 Query 221: 0.4329 Query 222: 0.0196 Query 223: 0.0069 Query 224: 0.6071 Query 225: 0.1090 Query 226: 0.0232 Query 227: 0.0010 Query 228: 0.0953 Query 229: 0.1578 Query 230: 0.5664 Query 231: 0.2199 Query 232: 0.0625 Query 233: 0.2858 Query 234: 0.0452 Query 235: 0.0183 Query 236: 0.4006 Query 237: 0.7501 Query 238: 0.1847 Query 239: 0.0791 Query 240: 0.3450 Query 241: 0.0550 Query 242: 0.2250 Query 243: 0.1056 Query 244: 0.1968 Query 245: 0.5942 Query 246: 0.0917 Query 247: 0.5116 Query 248: 0.3486 Query 249: 0.0249 Query 250: 0.0330 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1337 Interpolated Recall - Precision Averages: at 0.00 0.5581 at 0.10 0.4167 at 0.20 0.3395 at 0.30 0.2885 at 0.40 0.2525 at 0.50 0.2238 at 0.60 0.2070 at 0.70 0.1799 at 0.80 0.1487 at 0.90 0.1065 at 1.00 0.0378 Avg. prec. (non-interpolated) for all rel. documents 0.2330 Precision: At 5 docs: 0.3280 At 10 docs: 0.3100 At 15 docs: 0.3093 At 20 docs: 0.2900 At 30 docs: 0.2620 At 100 docs: 0.1600 At 200 docs: 0.1025 At 500 docs: 0.0494 At 1000 docs: 0.0267 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2713

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run IRn−Mu−Dexp

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run IRn−Mu−Dexp

Comparison to median by topic

U Alicante, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0235 Query 202: 0.4834 Query 203: 0.4645 Query 204: 0.3163 Query 205: 0.5034 Query 206: 0.2012 Query 207: 0.1368 Query 208: 0.1729 Query 209: 0.0831 Query 210: 0.2733 Query 211: 0.7863 Query 212: 0.0273 Query 213: 0.3419 Query 214: 0.0499 Query 215: 0.0242 Query 216: 0.0437 Query 217: 0.1253 Query 218: 0.8297 Query 219: 0.0485 Query 220: 0.0014 Query 221: 0.3583 Query 222: 0.0059 Query 223: 0.0085 Query 224: 0.4897 Query 225: 0.1125 Query 226: 0.0128 Query 227: 0.0013 Query 228: 0.0861 Query 229: 0.1824 Query 230: 0.5356 Query 231: 0.0882 Query 232: 0.0227 Query 233: 0.2106 Query 234: 0.0685 Query 235: 0.0130 Query 236: 0.1698 Query 237: 0.7498 Query 238: 0.1611 Query 239: 0.0692 Query 240: 0.3485 Query 241: 0.0398 Query 242: 0.1402 Query 243: 0.1108 Query 244: 0.2154 Query 245: 0.8301 Query 246: 0.1270 Query 247: 0.4779 Query 248: 0.3842 Query 249: 0.0180 Query 250: 0.0439 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1252 Interpolated Recall - Precision Averages: at 0.00 0.5432 at 0.10 0.4074 at 0.20 0.3349 at 0.30 0.2816 at 0.40 0.2428 at 0.50 0.2148 at 0.60 0.1836 at 0.70 0.1607 at 0.80 0.1276 at 0.90 0.0774 at 1.00 0.0321 Avg. prec. (non-interpolated) for all rel. documents 0.2204 Precision: At 5 docs: 0.3400 At 10 docs: 0.3120 At 15 docs: 0.2867 At 20 docs: 0.2710 At 30 docs: 0.2447 At 100 docs: 0.1442 At 200 docs: 0.0937 At 500 docs: 0.0457 At 1000 docs: 0.0250 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2585

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run IRn−Mu−nexp

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run IRn−Mu−nexp

Comparison to median by topic

U Alicante, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0514 Query 202: 0.4645 Query 203: 0.5461 Query 204: 0.4044 Query 205: 0.4242 Query 206: 0.1451 Query 207: 0.1581 Query 208: 0.1731 Query 209: 0.0529 Query 210: 0.1748 Query 211: 0.7713 Query 212: 0.0555 Query 213: 0.4339 Query 214: 0.0412 Query 215: 0.0514 Query 216: 0.0518 Query 217: 0.1388 Query 218: 0.8363 Query 219: 0.0456 Query 220: 0.0017 Query 221: 0.4224 Query 222: 0.0186 Query 223: 0.0069 Query 224: 0.5957 Query 225: 0.1371 Query 226: 0.0202 Query 227: 0.0009 Query 228: 0.0856 Query 229: 0.1457 Query 230: 0.5662 Query 231: 0.2215 Query 232: 0.0532 Query 233: 0.2908 Query 234: 0.0448 Query 235: 0.0193 Query 236: 0.4139 Query 237: 0.7542 Query 238: 0.2099 Query 239: 0.0832 Query 240: 0.3594 Query 241: 0.0465 Query 242: 0.2097 Query 243: 0.1057 Query 244: 0.1974 Query 245: 0.6017 Query 246: 0.1004 Query 247: 0.5989 Query 248: 0.3734 Query 249: 0.0263 Query 250: 0.0324 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1338 Interpolated Recall - Precision Averages: at 0.00 0.5408 at 0.10 0.4138 at 0.20 0.3399 at 0.30 0.2971 at 0.40 0.2611 at 0.50 0.2294 at 0.60 0.2104 at 0.70 0.1836 at 0.80 0.1505 at 0.90 0.1074 at 1.00 0.0418 Avg. prec. (non-interpolated) for all rel. documents 0.2353 Precision: At 5 docs: 0.3440 At 10 docs: 0.3180 At 15 docs: 0.3040 At 20 docs: 0.2880 At 30 docs: 0.2633 At 100 docs: 0.1574 At 200 docs: 0.1021 At 500 docs: 0.0492 At 1000 docs: 0.0268 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2714

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run IRn−Mu−Pexp

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run IRn−Mu−Pexp

Comparison to median by topic

CEA/LIC2M, Multilingual English, Auto, TD Average precision (individual queries): Query 201: 0.0000 Query 202: 0.3181 Query 203: 0.0410 Query 204: 0.0309 Query 205: 0.1282 Query 206: 0.2408 Query 207: 0.0077 Query 208: 0.4000 Query 209: 0.0004 Query 210: 0.0567 Query 211: 0.0356 Query 212: 0.0214 Query 213: 0.0507 Query 214: 0.0312 Query 215: 0.0655 Query 216: 0.1022 Query 217: 0.0073 Query 218: 0.2545 Query 219: 0.0009 Query 220: 0.0002 Query 221: 0.1192 Query 222: 0.0032 Query 223: 0.1131 Query 224: 0.4957 Query 225: 0.0000 Query 226: 0.0649 Query 227: 0.0007 Query 228: 0.1943 Query 229: 0.1342 Query 230: 0.1631 Query 231: 0.0229 Query 232: 0.1379 Query 233: 0.3412 Query 234: 0.0018 Query 235: 0.1095 Query 236: 0.0296 Query 237: 0.0799 Query 238: 0.1263 Query 239: 0.0363 Query 240: 0.4750 Query 241: 0.0011 Query 242: 0.1083 Query 243: 0.2866 Query 244: 0.0000 Query 245: 0.5783 Query 246: 0.2185 Query 247: 0.5473 Query 248: 0.0514 Query 249: 0.1707 Query 250: 0.0024 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 736 Interpolated Recall - Precision Averages: at 0.00 0.5402 at 0.10 0.3357 at 0.20 0.2453 at 0.30 0.1750 at 0.40 0.1503 at 0.50 0.0923 at 0.60 0.0597 at 0.70 0.0418 at 0.80 0.0216 at 0.90 0.0048 at 1.00 0.0000 Avg. prec. (non-interpolated) for all rel. documents 0.1281 Precision: At 5 docs: 0.2880 At 10 docs: 0.2320 At 15 docs: 0.2080 At 20 docs: 0.1790 At 30 docs: 0.1473 At 100 docs: 0.0792 At 200 docs: 0.0522 At 500 docs: 0.0256 At 1000 docs: 0.0147 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1669

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run lic2men1

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run lic2men1

Comparison to median by topic

CEA/LIC2M, Multilingual English, Auto, TDN Average precision (individual queries): Query 201: 0.0000 Query 202: 0.3310 Query 203: 0.1258 Query 204: 0.0826 Query 205: 0.1310 Query 206: 0.1662 Query 207: 0.0379 Query 208: 0.4000 Query 209: 0.0004 Query 210: 0.1046 Query 211: 0.0366 Query 212: 0.0364 Query 213: 0.0415 Query 214: 0.0333 Query 215: 0.0764 Query 216: 0.0122 Query 217: 0.0596 Query 218: 0.5056 Query 219: 0.0050 Query 220: 0.0001 Query 221: 0.2497 Query 222: 0.0116 Query 223: 0.1406 Query 224: 0.6217 Query 225: 0.0000 Query 226: 0.0132 Query 227: 0.0008 Query 228: 0.1918 Query 229: 0.1247 Query 230: 0.1476 Query 231: 0.0229 Query 232: 0.0741 Query 233: 0.3599 Query 234: 0.0023 Query 235: 0.1102 Query 236: 0.0363 Query 237: 0.2500 Query 238: 0.1269 Query 239: 0.0396 Query 240: 0.6076 Query 241: 0.0009 Query 242: 0.0531 Query 243: 0.1723 Query 244: 0.0018 Query 245: 0.5624 Query 246: 0.2135 Query 247: 0.2951 Query 248: 0.0455 Query 249: 0.1511 Query 250: 0.0073 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 777 Interpolated Recall - Precision Averages: at 0.00 0.5338 at 0.10 0.3595 at 0.20 0.2483 at 0.30 0.1807 at 0.40 0.1518 at 0.50 0.1182 at 0.60 0.0925 at 0.70 0.0548 at 0.80 0.0129 at 0.90 0.0061 at 1.00 0.0061 Avg. prec. (non-interpolated) for all rel. documents 0.1364 Precision: At 5 docs: 0.2920 At 10 docs: 0.2220 At 15 docs: 0.2040 At 20 docs: 0.1810 At 30 docs: 0.1640 At 100 docs: 0.0850 At 200 docs: 0.0577 At 500 docs: 0.0277 At 1000 docs: 0.0155 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1683

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run lic2men2

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run lic2men2

Comparison to median by topic

CEA/LIC2M, Multilingual French, Auto, TDN Average precision (individual queries): Query 201: 0.0020 Query 202: 0.0450 Query 203: 0.0104 Query 204: 0.0121 Query 205: 0.2232 Query 206: 0.3350 Query 207: 0.0080 Query 208: 0.3660 Query 209: 0.0697 Query 210: 0.0838 Query 211: 0.2546 Query 212: 0.0322 Query 213: 0.2814 Query 214: 0.0001 Query 215: 0.0457 Query 216: 0.0001 Query 217: 0.0541 Query 218: 0.5548 Query 219: 0.0002 Query 220: 0.0000 Query 221: 0.1491 Query 222: 0.0271 Query 223: 0.0045 Query 224: 0.1445 Query 225: 0.0049 Query 226: 0.0689 Query 227: 0.0000 Query 228: 0.0320 Query 229: 0.0587 Query 230: 0.4579 Query 231: 0.0281 Query 232: 0.0181 Query 233: 0.3226 Query 234: 0.0000 Query 235: 0.0125 Query 236: 0.0475 Query 237: 0.1608 Query 238: 0.1231 Query 239: 0.0245 Query 240: 0.2966 Query 241: 0.0056 Query 242: 0.0028 Query 243: 0.3033 Query 244: 0.0000 Query 245: 0.5394 Query 246: 0.0032 Query 247: 0.6624 Query 248: 0.4099 Query 249: 0.0199 Query 250: 0.0008 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 753 Interpolated Recall - Precision Averages: at 0.00 0.4438 at 0.10 0.3053 at 0.20 0.2530 at 0.30 0.1850 at 0.40 0.1440 at 0.50 0.0970 at 0.60 0.0734 at 0.70 0.0500 at 0.80 0.0237 at 0.90 0.0030 at 1.00 0.0005 Avg. prec. (non-interpolated) for all rel. documents 0.1261 Precision: At 5 docs: 0.2600 At 10 docs: 0.2360 At 15 docs: 0.2120 At 20 docs: 0.1970 At 30 docs: 0.1673 At 100 docs: 0.0872 At 200 docs: 0.0516 At 500 docs: 0.0261 At 1000 docs: 0.0151 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1657

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run lic2mfr1

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run lic2mfr1

Comparison to median by topic

U Amsterdam, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0651 Query 202: 0.6218 Query 203: 0.6517 Query 204: 0.5185 Query 205: 0.2820 Query 206: 0.5162 Query 207: 0.2021 Query 208: 0.4544 Query 209: 0.0467 Query 210: 0.0410 Query 211: 0.8738 Query 212: 0.1075 Query 213: 0.6020 Query 214: 0.1661 Query 215: 0.0942 Query 216: 0.1245 Query 217: 0.4342 Query 218: 0.8479 Query 219: 0.0074 Query 220: 0.0296 Query 221: 0.6058 Query 222: 0.0110 Query 223: 0.0809 Query 224: 0.6264 Query 225: 0.0886 Query 226: 0.0482 Query 227: 0.5000 Query 228: 0.3613 Query 229: 0.2795 Query 230: 0.6353 Query 231: 0.4086 Query 232: 0.1375 Query 233: 0.5002 Query 234: 0.0812 Query 235: 0.1428 Query 236: 0.3780 Query 237: 0.7234 Query 238: 0.3056 Query 239: 0.0848 Query 240: 0.8413 Query 241: 0.0519 Query 242: 0.1627 Query 243: 0.2618 Query 244: 0.0680 Query 245: 0.7549 Query 246: 0.1113 Query 247: 0.8763 Query 248: 0.4066 Query 249: 0.0180 Query 250: 0.0516 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1416 Interpolated Recall - Precision Averages: at 0.00 0.6937 at 0.10 0.5393 at 0.20 0.4859 at 0.30 0.4342 at 0.40 0.3696 at 0.50 0.3301 at 0.60 0.2731 at 0.70 0.2391 at 0.80 0.2004 at 0.90 0.1362 at 1.00 0.0553 Avg. prec. (non-interpolated) for all rel. documents 0.3258 Precision: At 5 docs: 0.4680 At 10 docs: 0.4200 At 15 docs: 0.3933 At 20 docs: 0.3660 At 30 docs: 0.3147 At 100 docs: 0.1878 At 200 docs: 0.1163 At 500 docs: 0.0534 At 1000 docs: 0.0283 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3393

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UAmsC04EnMu3Way

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UAmsC04EnMu3Way

Comparison to median by topic

U Amsterdam, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0360 Query 202: 0.4819 Query 203: 0.1550 Query 204: 0.5441 Query 205: 0.3402 Query 206: 0.3705 Query 207: 0.1770 Query 208: 0.2503 Query 209: 0.0113 Query 210: 0.0653 Query 211: 0.8440 Query 212: 0.0476 Query 213: 0.0381 Query 214: 0.2349 Query 215: 0.3023 Query 216: 0.0274 Query 217: 0.0188 Query 218: 0.8501 Query 219: 0.0035 Query 220: 0.0180 Query 221: 0.3463 Query 222: 0.0108 Query 223: 0.0539 Query 224: 0.3288 Query 225: 0.0415 Query 226: 0.0029 Query 227: 0.0417 Query 228: 0.2660 Query 229: 0.1035 Query 230: 0.6053 Query 231: 0.4305 Query 232: 0.0709 Query 233: 0.3751 Query 234: 0.0065 Query 235: 0.0028 Query 236: 0.3449 Query 237: 0.6050 Query 238: 0.2615 Query 239: 0.0768 Query 240: 0.3645 Query 241: 0.0366 Query 242: 0.1622 Query 243: 0.2529 Query 244: 0.0201 Query 245: 0.7038 Query 246: 0.0809 Query 247: 0.7825 Query 248: 0.4534 Query 249: 0.0153 Query 250: 0.0043 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1185 Interpolated Recall - Precision Averages: at 0.00 0.5689 at 0.10 0.4264 at 0.20 0.3527 at 0.30 0.3085 at 0.40 0.2646 at 0.50 0.2334 at 0.60 0.1953 at 0.70 0.1715 at 0.80 0.1401 at 0.90 0.0769 at 1.00 0.0260 Avg. prec. (non-interpolated) for all rel. documents 0.2333 Precision: At 5 docs: 0.3400 At 10 docs: 0.3340 At 15 docs: 0.3040 At 20 docs: 0.2820 At 30 docs: 0.2533 At 100 docs: 0.1468 At 200 docs: 0.0916 At 500 docs: 0.0435 At 1000 docs: 0.0237 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2462

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UAmsC04EnMu4Gr

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UAmsC04EnMu4Gr

Comparison to median by topic

U Amsterdam, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0549 Query 202: 0.6291 Query 203: 0.6831 Query 204: 0.5587 Query 205: 0.4317 Query 206: 0.5103 Query 207: 0.2129 Query 208: 0.6073 Query 209: 0.0351 Query 210: 0.0585 Query 211: 0.8954 Query 212: 0.0965 Query 213: 0.6079 Query 214: 0.1869 Query 215: 0.1657 Query 216: 0.0907 Query 217: 0.4131 Query 218: 0.8872 Query 219: 0.0083 Query 220: 0.0341 Query 221: 0.5475 Query 222: 0.0073 Query 223: 0.0805 Query 224: 0.6831 Query 225: 0.1759 Query 226: 0.0253 Query 227: 0.5000 Query 228: 0.3522 Query 229: 0.2800 Query 230: 0.6589 Query 231: 0.4851 Query 232: 0.1205 Query 233: 0.4832 Query 234: 0.1492 Query 235: 0.0720 Query 236: 0.4365 Query 237: 0.7519 Query 238: 0.3500 Query 239: 0.0840 Query 240: 0.7763 Query 241: 0.0487 Query 242: 0.1786 Query 243: 0.2740 Query 244: 0.0839 Query 245: 0.7400 Query 246: 0.1529 Query 247: 0.9472 Query 248: 0.4573 Query 249: 0.0185 Query 250: 0.0452 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1395 Interpolated Recall - Precision Averages: at 0.00 0.7025 at 0.10 0.5676 at 0.20 0.5130 at 0.30 0.4569 at 0.40 0.3923 at 0.50 0.3596 at 0.60 0.2879 at 0.70 0.2503 at 0.80 0.2097 at 0.90 0.1428 at 1.00 0.0539 Avg. prec. (non-interpolated) for all rel. documents 0.3427 Precision: At 5 docs: 0.4800 At 10 docs: 0.4420 At 15 docs: 0.4053 At 20 docs: 0.3780 At 30 docs: 0.3273 At 100 docs: 0.1864 At 200 docs: 0.1179 At 500 docs: 0.0529 At 1000 docs: 0.0279 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3538

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UAmsC04EnMuAll

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UAmsC04EnMuAll

Comparison to median by topic

U Amsterdam, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0250 Query 202: 0.6224 Query 203: 0.5160 Query 204: 0.4673 Query 205: 0.4110 Query 206: 0.2436 Query 207: 0.1978 Query 208: 0.5003 Query 209: 0.0250 Query 210: 0.0753 Query 211: 0.8307 Query 212: 0.0639 Query 213: 0.3720 Query 214: 0.1990 Query 215: 0.1354 Query 216: 0.0298 Query 217: 0.2310 Query 218: 0.8369 Query 219: 0.0025 Query 220: 0.0395 Query 221: 0.2836 Query 222: 0.0019 Query 223: 0.0353 Query 224: 0.1843 Query 225: 0.0636 Query 226: 0.0043 Query 227: 0.5000 Query 228: 0.2191 Query 229: 0.1304 Query 230: 0.5234 Query 231: 0.3274 Query 232: 0.0832 Query 233: 0.2942 Query 234: 0.0263 Query 235: 0.0424 Query 236: 0.3621 Query 237: 0.7398 Query 238: 0.2077 Query 239: 0.0444 Query 240: 0.6562 Query 241: 0.0512 Query 242: 0.0892 Query 243: 0.2271 Query 244: 0.1190 Query 245: 0.6621 Query 246: 0.0478 Query 247: 0.5125 Query 248: 0.2971 Query 249: 0.0091 Query 250: 0.0302 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1350 Interpolated Recall - Precision Averages: at 0.00 0.5548 at 0.10 0.4453 at 0.20 0.3739 at 0.30 0.3359 at 0.40 0.2869 at 0.50 0.2635 at 0.60 0.2122 at 0.70 0.1922 at 0.80 0.1599 at 0.90 0.1007 at 1.00 0.0334 Avg. prec. (non-interpolated) for all rel. documents 0.2520 Precision: At 5 docs: 0.3160 At 10 docs: 0.3040 At 15 docs: 0.2840 At 20 docs: 0.2840 At 30 docs: 0.2567 At 100 docs: 0.1634 At 200 docs: 0.1048 At 500 docs: 0.0498 At 1000 docs: 0.0270 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2762

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UAmsC04EnMuAll2

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UAmsC04EnMuAll2

Comparison to median by topic

U Amsterdam, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0386 Query 202: 0.5950 Query 203: 0.3541 Query 204: 0.3741 Query 205: 0.4677 Query 206: 0.3442 Query 207: 0.1623 Query 208: 0.7566 Query 209: 0.0165 Query 210: 0.2285 Query 211: 0.8164 Query 212: 0.1267 Query 213: 0.5384 Query 214: 0.1018 Query 215: 0.0935 Query 216: 0.2333 Query 217: 0.5421 Query 218: 0.7430 Query 219: 0.1865 Query 220: 0.0403 Query 221: 0.2515 Query 222: 0.0013 Query 223: 0.0539 Query 224: 0.7167 Query 225: 0.2965 Query 226: 0.0498 Query 227: 0.0179 Query 228: 0.1586 Query 229: 0.1156 Query 230: 0.5307 Query 231: 0.4036 Query 232: 0.0794 Query 233: 0.3292 Query 234: 0.1813 Query 235: 0.0814 Query 236: 0.3331 Query 237: 0.8127 Query 238: 0.2410 Query 239: 0.0738 Query 240: 0.8905 Query 241: 0.1011 Query 242: 0.2339 Query 243: 0.3097 Query 244: 0.2020 Query 245: 0.5567 Query 246: 0.1490 Query 247: 0.9620 Query 248: 0.1655 Query 249: 0.0244 Query 250: 0.1192 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1375 Interpolated Recall - Precision Averages: at 0.00 0.7673 at 0.10 0.5527 at 0.20 0.4214 at 0.30 0.3623 at 0.40 0.3303 at 0.50 0.3012 at 0.60 0.2589 at 0.70 0.2281 at 0.80 0.1894 at 0.90 0.1231 at 1.00 0.0480 Avg. prec. (non-interpolated) for all rel. documents 0.3040 Precision: At 5 docs: 0.4440 At 10 docs: 0.4060 At 15 docs: 0.3627 At 20 docs: 0.3270 At 30 docs: 0.2887 At 100 docs: 0.1728 At 200 docs: 0.1081 At 500 docs: 0.0502 At 1000 docs: 0.0275 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3325

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UAmsC04EnMuWSLM

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UAmsC04EnMuWSLM

Comparison to median by topic

SUNY Buffalo, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0009 Query 202: 0.4502 Query 203: 0.0001 Query 204: 0.6187 Query 205: 0.3978 Query 206: 0.2048 Query 207: 0.0553 Query 208: 0.0409 Query 209: 0.0007 Query 210: 0.0142 Query 211: 0.7026 Query 212: 0.0259 Query 213: 0.4634 Query 214: 0.1472 Query 215: 0.1321 Query 216: 0.2628 Query 217: 0.1239 Query 218: 0.8202 Query 219: 0.0107 Query 220: 0.0105 Query 221: 0.0613 Query 222: 0.0037 Query 223: 0.0862 Query 224: 0.0534 Query 225: 0.0535 Query 226: 0.0302 Query 227: 0.0000 Query 228: 0.0269 Query 229: 0.0074 Query 230: 0.5837 Query 231: 0.0472 Query 232: 0.0446 Query 233: 0.2444 Query 234: 0.0164 Query 235: 0.0319 Query 236: 0.7050 Query 237: 0.0863 Query 238: 0.2001 Query 239: 0.0052 Query 240: 0.4264 Query 241: 0.0013 Query 242: 0.0989 Query 243: 0.1319 Query 244: 0.0000 Query 245: 0.6956 Query 246: 0.2087 Query 247: 0.1189 Query 248: 0.0654 Query 249: 0.0066 Query 250: 0.0213 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1092 Interpolated Recall - Precision Averages: at 0.00 0.4635 at 0.10 0.3245 at 0.20 0.2627 at 0.30 0.2094 at 0.40 0.1908 at 0.50 0.1674 at 0.60 0.1392 at 0.70 0.1186 at 0.80 0.0930 at 0.90 0.0548 at 1.00 0.0189 Avg. prec. (non-interpolated) for all rel. documents 0.1709 Precision: At 5 docs: 0.2320 At 10 docs: 0.2360 At 15 docs: 0.2280 At 20 docs: 0.2210 At 30 docs: 0.1987 At 100 docs: 0.1218 At 200 docs: 0.0783 At 500 docs: 0.0390 At 1000 docs: 0.0218 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2003

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UBmult01

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UBmult01

Comparison to median by topic

SUNY Buffalo, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0235 Query 202: 0.5203 Query 203: 0.0000 Query 204: 0.4570 Query 205: 0.2561 Query 206: 0.0837 Query 207: 0.1372 Query 208: 0.0439 Query 209: 0.0032 Query 210: 0.0150 Query 211: 0.3776 Query 212: 0.0570 Query 213: 0.4267 Query 214: 0.0825 Query 215: 0.1152 Query 216: 0.2036 Query 217: 0.0970 Query 218: 0.4522 Query 219: 0.0109 Query 220: 0.0104 Query 221: 0.0539 Query 222: 0.0062 Query 223: 0.0284 Query 224: 0.2111 Query 225: 0.1408 Query 226: 0.0699 Query 227: 0.0000 Query 228: 0.0737 Query 229: 0.0890 Query 230: 0.2846 Query 231: 0.0505 Query 232: 0.0647 Query 233: 0.1975 Query 234: 0.1374 Query 235: 0.0209 Query 236: 0.5295 Query 237: 0.0822 Query 238: 0.2501 Query 239: 0.0251 Query 240: 0.4224 Query 241: 0.0006 Query 242: 0.1242 Query 243: 0.2100 Query 244: 0.0000 Query 245: 0.5611 Query 246: 0.0748 Query 247: 0.0888 Query 248: 0.0707 Query 249: 0.0086 Query 250: 0.0174 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1135 Interpolated Recall - Precision Averages: at 0.00 0.4771 at 0.10 0.3102 at 0.20 0.2306 at 0.30 0.1861 at 0.40 0.1583 at 0.50 0.1312 at 0.60 0.0981 at 0.70 0.0816 at 0.80 0.0626 at 0.90 0.0365 at 1.00 0.0122 Avg. prec. (non-interpolated) for all rel. documents 0.1453 Precision: At 5 docs: 0.2800 At 10 docs: 0.2400 At 15 docs: 0.2227 At 20 docs: 0.2080 At 30 docs: 0.1920 At 100 docs: 0.1126 At 200 docs: 0.0773 At 500 docs: 0.0403 At 1000 docs: 0.0227 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1884

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UBmult02

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UBmult02

Comparison to median by topic

SUNY Buffalo, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.1083 Query 202: 0.5239 Query 203: 0.2554 Query 204: 0.4787 Query 205: 0.5209 Query 206: 0.2514 Query 207: 0.1734 Query 208: 0.6023 Query 209: 0.1676 Query 210: 0.0678 Query 211: 0.5157 Query 212: 0.1865 Query 213: 0.3894 Query 214: 0.0708 Query 215: 0.0265 Query 216: 0.2620 Query 217: 0.3280 Query 218: 0.5660 Query 219: 0.1872 Query 220: 0.0194 Query 221: 0.2078 Query 222: 0.0256 Query 223: 0.1392 Query 224: 0.4235 Query 225: 0.2704 Query 226: 0.2861 Query 227: 0.0833 Query 228: 0.0789 Query 229: 0.1203 Query 230: 0.4966 Query 231: 0.3270 Query 232: 0.0440 Query 233: 0.4259 Query 234: 0.1854 Query 235: 0.2306 Query 236: 0.5908 Query 237: 0.5448 Query 238: 0.2171 Query 239: 0.0880 Query 240: 0.4863 Query 241: 0.0062 Query 242: 0.3729 Query 243: 0.2786 Query 244: 0.3900 Query 245: 0.6019 Query 246: 0.2794 Query 247: 0.4323 Query 248: 0.3798 Query 249: 0.0074 Query 250: 0.0171 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1463 Interpolated Recall - Precision Averages: at 0.00 0.7095 at 0.10 0.5358 at 0.20 0.4385 at 0.30 0.3554 at 0.40 0.2990 at 0.50 0.2553 at 0.60 0.2093 at 0.70 0.1617 at 0.80 0.1246 at 0.90 0.0983 at 1.00 0.0324 Avg. prec. (non-interpolated) for all rel. documents 0.2748 Precision: At 5 docs: 0.4360 At 10 docs: 0.3720 At 15 docs: 0.3493 At 20 docs: 0.3160 At 30 docs: 0.2733 At 100 docs: 0.1604 At 200 docs: 0.1065 At 500 docs: 0.0538 At 1000 docs: 0.0293 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2817

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UBmulti03

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UBmulti03

Comparison to median by topic

U Chicago, Multilingual English, Auto, T

Average precision (individual queries): Query 201: 0.1028 Query 202: 0.4060 Query 203: 0.0156 Query 204: 0.2997 Query 205: 0.2304 Query 206: 0.1653 Query 207: 0.0798 Query 208: 0.0877 Query 209: 0.0710 Query 210: 0.0220 Query 211: 0.4354 Query 212: 0.2934 Query 213: 0.1407 Query 214: 0.0384 Query 215: 0.0070 Query 216: 0.0584 Query 217: 0.0813 Query 218: 0.4153 Query 219: 0.0087 Query 220: 0.0000 Query 221: 0.2553 Query 222: 0.0070 Query 223: 0.0192 Query 224: 0.4157 Query 225: 0.1329 Query 226: 0.0042 Query 227: 0.0086 Query 228: 0.0000 Query 229: 0.0587 Query 230: 0.3056 Query 231: 0.2010 Query 232: 0.1299 Query 233: 0.2513 Query 234: 0.1108 Query 235: 0.1769 Query 236: 0.3246 Query 237: 0.4429 Query 238: 0.2885 Query 239: 0.0080 Query 240: 0.3150 Query 241: 0.0178 Query 242: 0.2583 Query 243: 0.0141 Query 244: 0.0259 Query 245: 0.3916 Query 246: 0.2410 Query 247: 0.2089 Query 248: 0.1515 Query 249: 0.0011 Query 250: 0.0001 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1142 Interpolated Recall - Precision Averages: at 0.00 0.5437 at 0.10 0.3378 at 0.20 0.2748 at 0.30 0.2026 at 0.40 0.1680 at 0.50 0.1387 at 0.60 0.1034 at 0.70 0.0762 at 0.80 0.0441 at 0.90 0.0117 at 1.00 0.0006 Avg. prec. (non-interpolated) for all rel. documents 0.1545 Precision: At 5 docs: 0.2920 At 10 docs: 0.2480 At 15 docs: 0.2307 At 20 docs: 0.2260 At 30 docs: 0.2013 At 100 docs: 0.1222 At 200 docs: 0.0806 At 500 docs: 0.0408 At 1000 docs: 0.0228 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1916

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UCEMULdecT3

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UCEMULdecT3

Comparison to median by topic

U Chicago, Multilingual English, Auto, T

Average precision (individual queries): Query 201: 0.0913 Query 202: 0.3811 Query 203: 0.0018 Query 204: 0.1831 Query 205: 0.1750 Query 206: 0.1669 Query 207: 0.0751 Query 208: 0.1153 Query 209: 0.0638 Query 210: 0.0093 Query 211: 0.3385 Query 212: 0.1778 Query 213: 0.0838 Query 214: 0.0395 Query 215: 0.0069 Query 216: 0.0375 Query 217: 0.0703 Query 218: 0.3873 Query 219: 0.0031 Query 220: 0.0000 Query 221: 0.1491 Query 222: 0.0091 Query 223: 0.0260 Query 224: 0.4754 Query 225: 0.0715 Query 226: 0.0053 Query 227: 0.0007 Query 228: 0.0000 Query 229: 0.0611 Query 230: 0.2031 Query 231: 0.1450 Query 232: 0.1423 Query 233: 0.1885 Query 234: 0.0609 Query 235: 0.1140 Query 236: 0.3113 Query 237: 0.2913 Query 238: 0.2511 Query 239: 0.0076 Query 240: 0.3318 Query 241: 0.0002 Query 242: 0.2404 Query 243: 0.0137 Query 244: 0.0007 Query 245: 0.4359 Query 246: 0.2240 Query 247: 0.2108 Query 248: 0.1569 Query 249: 0.0013 Query 250: 0.0001 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1022 Interpolated Recall - Precision Averages: at 0.00 0.5264 at 0.10 0.2954 at 0.20 0.2231 at 0.30 0.1655 at 0.40 0.1418 at 0.50 0.1151 at 0.60 0.0866 at 0.70 0.0632 at 0.80 0.0388 at 0.90 0.0115 at 1.00 0.0006 Avg. prec. (non-interpolated) for all rel. documents 0.1307 Precision: At 5 docs: 0.2640 At 10 docs: 0.2180 At 15 docs: 0.2040 At 20 docs: 0.1890 At 30 docs: 0.1727 At 100 docs: 0.1088 At 200 docs: 0.0717 At 500 docs: 0.0364 At 1000 docs: 0.0204 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1753

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UCEMULlppT4

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UCEMULlppT4

Comparison to median by topic

U Chicago, Multilingual English, Auto, T

Average precision (individual queries): Query 201: 0.0908 Query 202: 0.3821 Query 203: 0.0098 Query 204: 0.1831 Query 205: 0.1750 Query 206: 0.1669 Query 207: 0.0838 Query 208: 0.1153 Query 209: 0.0744 Query 210: 0.0211 Query 211: 0.3812 Query 212: 0.2536 Query 213: 0.1256 Query 214: 0.0400 Query 215: 0.0069 Query 216: 0.0408 Query 217: 0.0703 Query 218: 0.3932 Query 219: 0.0031 Query 220: 0.0000 Query 221: 0.1980 Query 222: 0.0091 Query 223: 0.0260 Query 224: 0.4989 Query 225: 0.0761 Query 226: 0.0053 Query 227: 0.0094 Query 228: 0.0000 Query 229: 0.0611 Query 230: 0.3109 Query 231: 0.2123 Query 232: 0.1483 Query 233: 0.2478 Query 234: 0.0617 Query 235: 0.1171 Query 236: 0.3113 Query 237: 0.4111 Query 238: 0.3021 Query 239: 0.0096 Query 240: 0.3318 Query 241: 0.0212 Query 242: 0.2510 Query 243: 0.0137 Query 244: 0.0270 Query 245: 0.4491 Query 246: 0.2240 Query 247: 0.2108 Query 248: 0.1569 Query 249: 0.0013 Query 250: 0.0001 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1086 Interpolated Recall - Precision Averages: at 0.00 0.5442 at 0.10 0.3268 at 0.20 0.2511 at 0.30 0.1793 at 0.40 0.1536 at 0.50 0.1236 at 0.60 0.0971 at 0.70 0.0774 at 0.80 0.0477 at 0.90 0.0151 at 1.00 0.0006 Avg. prec. (non-interpolated) for all rel. documents 0.1464 Precision: At 5 docs: 0.2880 At 10 docs: 0.2440 At 15 docs: 0.2267 At 20 docs: 0.2110 At 30 docs: 0.1913 At 100 docs: 0.1178 At 200 docs: 0.0774 At 500 docs: 0.0390 At 1000 docs: 0.0217 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1860

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UCEMULT2

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UCEMULT2

Comparison to median by topic

U Chicago, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.1184 Query 202: 0.4363 Query 203: 0.0239 Query 204: 0.1460 Query 205: 0.2035 Query 206: 0.1706 Query 207: 0.0814 Query 208: 0.2144 Query 209: 0.0707 Query 210: 0.0207 Query 211: 0.3888 Query 212: 0.0925 Query 213: 0.1047 Query 214: 0.1268 Query 215: 0.0043 Query 216: 0.0604 Query 217: 0.1062 Query 218: 0.3594 Query 219: 0.0096 Query 220: 0.0000 Query 221: 0.2667 Query 222: 0.0118 Query 223: 0.0139 Query 224: 0.5565 Query 225: 0.0905 Query 226: 0.0004 Query 227: 0.0556 Query 228: 0.0000 Query 229: 0.1170 Query 230: 0.2702 Query 231: 0.3988 Query 232: 0.1786 Query 233: 0.2515 Query 234: 0.0473 Query 235: 0.1477 Query 236: 0.2844 Query 237: 0.4064 Query 238: 0.3686 Query 239: 0.0225 Query 240: 0.3705 Query 241: 0.0234 Query 242: 0.0641 Query 243: 0.0315 Query 244: 0.1716 Query 245: 0.4619 Query 246: 0.2501 Query 247: 0.3598 Query 248: 0.1631 Query 249: 0.0132 Query 250: 0.0044 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1103 Interpolated Recall - Precision Averages: at 0.00 0.5886 at 0.10 0.3813 at 0.20 0.2679 at 0.30 0.2064 at 0.40 0.1657 at 0.50 0.1350 at 0.60 0.1064 at 0.70 0.0829 at 0.80 0.0554 at 0.90 0.0157 at 1.00 0.0013 Avg. prec. (non-interpolated) for all rel. documents 0.1628 Precision: At 5 docs: 0.3080 At 10 docs: 0.2520 At 15 docs: 0.2400 At 20 docs: 0.2180 At 30 docs: 0.1947 At 100 docs: 0.1204 At 200 docs: 0.0800 At 500 docs: 0.0399 At 1000 docs: 0.0221 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2001

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UCEMULTD1

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UCEMULTD1

Comparison to median by topic

U Hildesheim, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0356 Query 202: 0.4700 Query 203: 0.5531 Query 204: 0.3975 Query 205: 0.1867 Query 206: 0.1973 Query 207: 0.0954 Query 208: 0.1589 Query 209: 0.0013 Query 210: 0.1294 Query 211: 0.7103 Query 212: 0.0290 Query 213: 0.3307 Query 214: 0.0071 Query 215: 0.0435 Query 216: 0.0010 Query 217: 0.0097 Query 218: 0.6347 Query 219: 0.0004 Query 220: 0.0008 Query 221: 0.4649 Query 222: 0.0193 Query 223: 0.0057 Query 224: 0.1194 Query 225: 0.0136 Query 226: 0.0219 Query 227: 0.0312 Query 228: 0.0013 Query 229: 0.1439 Query 230: 0.5822 Query 231: 0.3632 Query 232: 0.0207 Query 233: 0.2105 Query 234: 0.0302 Query 235: 0.0735 Query 236: 0.7007 Query 237: 0.4162 Query 238: 0.2105 Query 239: 0.0272 Query 240: 0.5887 Query 241: 0.0163 Query 242: 0.0661 Query 243: 0.0722 Query 244: 0.3333 Query 245: 0.4739 Query 246: 0.0189 Query 247: 0.3438 Query 248: 0.5044 Query 249: 0.0057 Query 250: 0.0000 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1031 Interpolated Recall - Precision Averages: at 0.00 0.5503 at 0.10 0.3603 at 0.20 0.3170 at 0.30 0.2801 at 0.40 0.2525 at 0.50 0.2201 at 0.60 0.1534 at 0.70 0.1157 at 0.80 0.0739 at 0.90 0.0339 at 1.00 0.0036 Avg. prec. (non-interpolated) for all rel. documents 0.1974 Precision: At 5 docs: 0.3360 At 10 docs: 0.3040 At 15 docs: 0.2760 At 20 docs: 0.2600 At 30 docs: 0.2253 At 100 docs: 0.1260 At 200 docs: 0.0782 At 500 docs: 0.0370 At 1000 docs: 0.0206 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2300

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UHImlt1

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UHImlt1

Comparison to median by topic

U Hildesheim, Multilingual English, Auto, TD

Average precision (individual queries): Query 201: 0.0214 Query 202: 0.4334 Query 203: 0.4682 Query 204: 0.4693 Query 205: 0.1553 Query 206: 0.1681 Query 207: 0.0748 Query 208: 0.0740 Query 209: 0.0023 Query 210: 0.1691 Query 211: 0.6178 Query 212: 0.0387 Query 213: 0.3026 Query 214: 0.0043 Query 215: 0.0216 Query 216: 0.0005 Query 217: 0.0020 Query 218: 0.6689 Query 219: 0.0001 Query 220: 0.0009 Query 221: 0.3834 Query 222: 0.0142 Query 223: 0.0035 Query 224: 0.0643 Query 225: 0.0009 Query 226: 0.0511 Query 227: 0.0082 Query 228: 0.0016 Query 229: 0.1409 Query 230: 0.5320 Query 231: 0.1553 Query 232: 0.0272 Query 233: 0.1679 Query 234: 0.0213 Query 235: 0.0501 Query 236: 0.7600 Query 237: 0.4495 Query 238: 0.1709 Query 239: 0.0184 Query 240: 0.5280 Query 241: 0.0058 Query 242: 0.1205 Query 243: 0.0677 Query 244: 0.6667 Query 245: 0.5054 Query 246: 0.0233 Query 247: 0.2188 Query 248: 0.3930 Query 249: 0.0029 Query 250: 0.0000 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 973 Interpolated Recall - Precision Averages: at 0.00 0.5521 at 0.10 0.3538 at 0.20 0.2990 at 0.30 0.2615 at 0.40 0.2368 at 0.50 0.2085 at 0.60 0.1377 at 0.70 0.0855 at 0.80 0.0620 at 0.90 0.0187 at 1.00 0.0004 Avg. prec. (non-interpolated) for all rel. documents 0.1849 Precision: At 5 docs: 0.3280 At 10 docs: 0.2840 At 15 docs: 0.2613 At 20 docs: 0.2370 At 30 docs: 0.2053 At 100 docs: 0.1174 At 200 docs: 0.0728 At 500 docs: 0.0346 At 1000 docs: 0.0195 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2165

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UHImlt2

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UHImlt2

Comparison to median by topic

U Jaen/SINAI, Multilingual English, Auto, TD Average precision (individual queries): Query 201: 0.0382 Query 202: 0.5970 Query 203: 0.1963 Query 204: 0.5736 Query 205: 0.4829 Query 206: 0.3124 Query 207: 0.1907 Query 208: 0.4907 Query 209: 0.1800 Query 210: 0.1908 Query 211: 0.7902 Query 212: 0.0776 Query 213: 0.5421 Query 214: 0.0309 Query 215: 0.0394 Query 216: 0.0866 Query 217: 0.2215 Query 218: 0.8073 Query 219: 0.0975 Query 220: 0.0442 Query 221: 0.1960 Query 222: 0.0205 Query 223: 0.0302 Query 224: 0.7497 Query 225: 0.2840 Query 226: 0.0240 Query 227: 0.0006 Query 228: 0.0704 Query 229: 0.1638 Query 230: 0.6445 Query 231: 0.3460 Query 232: 0.0304 Query 233: 0.3360 Query 234: 0.1852 Query 235: 0.2943 Query 236: 0.1416 Query 237: 0.8407 Query 238: 0.2339 Query 239: 0.0625 Query 240: 0.6337 Query 241: 0.0200 Query 242: 0.4085 Query 243: 0.1495 Query 244: 0.0236 Query 245: 0.4828 Query 246: 0.1762 Query 247: 0.7140 Query 248: 0.5935 Query 249: 0.0148 Query 250: 0.0061 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 1826 Rel_ret: 1346 Interpolated Recall - Precision Averages: at 0.00 0.6168 at 0.10 0.4768 at 0.20 0.4077 at 0.30 0.3738 at 0.40 0.3205 at 0.50 0.2972 at 0.60 0.2609 at 0.70 0.2138 at 0.80 0.1787 at 0.90 0.1123 at 1.00 0.0242 Avg. prec. (non-interpolated) for all rel. documents 0.2773 Precision: At 5 docs: 0.3720 At 10 docs: 0.3920 At 15 docs: 0.3613 At 20 docs: 0.3360 At 30 docs: 0.2880 At 100 docs: 0.1702 At 200 docs: 0.1064 At 500 docs: 0.0492 At 1000 docs: 0.0269 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3279

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run UJAMLRL

Recall−Precision Values 201210220 Topic number

−1.0

−0.50.0

+0.5

+1.0

Difference (average precision)

Run UJAMLRL

Comparison to median by topic