Appendix

(1)

Run Type: type of experiment (automatic/manual).

Pool: specifies if experiment was used for relevance assessment pooling.

The following eleven pages compare the top entries for every track/task. Shown are the recall/precision curves for at most five groups. Chosen are the best entries by the respective groups, with the topic fields fixed to title+description (mandatory experiment for every participant) and only automatic experiments used. For the multilingual and bilingual experiments, the top entries regardless of topic language are presented.

The remainder of the pages is made up by the individual results for every official experiment. Each experiment is presented on one page, with the following details given:

1. A table providing the following information:

• Average precision figures for every individual query. This allows comparison of system performance for single queries, which is important since variation of performance across queries is often very high and can be significant.

• Overall statistics, giving:

- the total number of documents retrieved by the system

- the total number of overall relevant documents in the collection, and - the total number of relevant documents actually found by the system - interpolated precision averages at specific recall levels

- non-interpolated average precision over all queries

- precision numbers after inspecting a specific number of documents - R-precision: precision after the last relevant document was retrieved.

2. Two graphs consisting of:

• a recall/precision graph, providing a plot of the precision values for various recall levels. This is the standard statistic and is the one most commonly reported in the literature.

• a comparison to median performance. For each query, the difference in average precision, when compared to the median performance for the given task, is plotted. This graph gives valuable insight into which type of queries is handled well by different systems.

The results page for a specific experiment can be most quickly located by using the table of contents. This table is sorted by group, task, topic language and topic fields. The individual pages are sorted by task and run tag.

More information on the interpretation of the standard measures used for scoring experiments (average precision, recall levels, precision/recall graphs, etc.) can be found e.g. in:

Martin Braschler, Carol Peters: CLEF Methodology and Metrics, CLEF 2001, pages 394-404 (Carol Peters, Martin Braschler, Julio Gonzalo, Michael Kluck (Eds.), Evaluation of Cross-Language Information Retrieval Systems, Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, Darmstadt, Germany, September 3-4, 2001. Revised Papers. Lecture Notes in Computer Science, Vol. 2406, Springer, 2002, ISBN 3- 540-44042-9)

(2)

COLE Group Univ. La Coruna Spain coleTDlem02 Mono ES TD Auto

COLE Group Univ. La Coruna Spain coleTDNlem02 Mono ES TDN Auto Y

COLE Group Univ. La Coruna Spain coleTDNpds02 Mono ES TDN Auto Y

COLE Group Univ. La Coruna Spain coleTDNsyn02 Mono ES TDN Auto

CWI/CNLP Netherlands/USA AAbiENNLt Bi-NL EN T Auto

CWI/CNLP Netherlands/USA AAbiENNLtd Bi-NL EN TD Auto Y

CWI/CNLP Netherlands/USA AAmoNLt Mono NL T Auto

CWI/CNLP Netherlands/USA AAmoNLtd Mono NL TD Auto Y

Eurospider IT AG Switzerland EIT2GDM1 Mono DE TD Auto Y

Eurospider IT AG Switzerland EIT2GDL1 Mono DE TD Auto Y

Eurospider IT AG Switzerland EIT2GDB1 Mono DE TD Auto

Eurospider IT AG Switzerland EIT2GNM1 Mono DE TDN Auto

Eurospider IT AG Switzerland EIT2MDF3 Multi DE TD Auto Y

Eurospider IT AG Switzerland EIT2MDC3 Multi DE TD Auto

Eurospider IT AG Switzerland EIT2MNF3 Multi DE TDN Auto Y

Eurospider IT AG Switzerland EIT2MNU1 Multi DE TDN Auto

Eurospider IT AG Switzerland EAN2MDF4 Multi EN TD Auto

Fondazione Ugo Bordini Italy fub02bl Mono IT TD Auto Y

Fondazione Ugo Bordini Italy fub02l Mono IT TD Auto Y

Fondazione Ugo Bordini Italy fub02b Mono IT TD Auto

Fondazione Ugo Bordini Italy fub02 Mono IT TD Auto

Hummingbird Canada humDE02 Mono DE TD Auto Y

Hummingbird Canada humES02 Mono ES TD Auto Y

Hummingbird Canada humFI02 Mono FI TD Auto Y

Hummingbird Canada humFI02n Mono FI TDN Auto Y

Hummingbird Canada humFR02 Mono FR TD Auto Y

Hummingbird Canada humIT02 Mono IT TD Auto Y

Hummingbird Canada humNL02 Mono NL TD Auto Y

Hummingbird Canada humNL02n Mono NL TDN Auto

Hummingbird Canada humSV02 Mono SV TD Auto Y

Hummingbird Canada humSV02n Mono SV TDN Auto Y

IMBIT Inst., Univ. Hildesheim Germany IMBIT Mono DE TDN Manual Y

IMS Univ. Padova Italy PDDS2PLL3 Mono IT TD Auto Y

IMS Univ. Padova Italy PDDP Mono IT TD Auto Y

IMS Univ. Padova Italy PDDS2PL Mono IT TD Auto

IMS Univ. Padova Italy PDDN Mono IT TD Auto

IRIT, Toulouse France iritBFr2En Bi-EN FR TD Auto Y

IRIT, Toulouse France RunMonoSp Mono ES TD Auto Y

IRIT, Toulouse France RunMonFrench Mono FR TD Auto Y

IRIT, Toulouse France RunMonoIt Mono IT TD Auto Y

IRIT, Toulouse France iritMEn2All Multi EN TD Auto Y

ISU, Univ. Hildesheim Germany UHi02r1 GIRT DE TD Auto Y

ISU, Univ. Hildesheim Germany UHi02r2 GIRT DE TD Auto Y

ITC-IRST Italy IRSTen2it1 Bi-IT EN TD Auto Y

ITC-IRST Italy IRSTen2it2 Bi-IT EN TD Auto

ITC-IRST Italy IRSTen2it3 Bi-IT EN TD Auto

ITC-IRST Italy IRSTit1 Mono IT TD Auto Y

JHU/APL USA aplbiende Bi-DE EN TD Auto Y

JHU/APL USA aplbiptena Bi-EN PT TD Auto Y

JHU/APL USA aplbienes Bi-ES EN TD Auto

JHU/APL USA aplbiptesa Bi-ES PT TD Auto

JHU/APL USA aplbiptesb Bi-ES PT TD Auto

JHU/APL USA aplbienfi Bi-FI EN TD Auto Y

(3)

Middlesex Univ. United Kingdom MDXman Bi-EN PT TDN Manual Y

National Taiwan Univ. Taiwan NTUmulti01 Multi EN TD Auto

National Taiwan Univ. Taiwan NTUmulti02 Multi EN TD Auto Y

National Taiwan Univ. Taiwan NTUmulti04 Multi EN TD Auto Y

Océ Netherlands oce02es2enLO Bi-EN ES TD Auto Y

Océ Netherlands oce02es2enBF Bi-EN ES TD Auto

Océ Netherlands oce02nl2enER Bi-EN NL TD Auto Y

Océ Netherlands oce02en2esLO Bi-ES EN TD Auto

Océ Netherlands oce02en2esBF Bi-ES EN TD Auto

Océ Netherlands oce02en2nlER Bi-NL EN TD Auto Y

Océ Netherlands oce02monDEto Mono DE T Auto

Océ Netherlands oce02monDE Mono DE TD Auto Y

Océ Netherlands oce02monESto Mono ES T Auto

Océ Netherlands oce02monES Mono ES TD Auto Y

Océ Netherlands oce02monFRto Mono FR T Auto

Océ Netherlands oce02monFR Mono FR TD Auto Y

Océ Netherlands oce02monITto Mono IT T Auto

Océ Netherlands oce02monIT Mono IT TD Auto Y

Océ Netherlands oce02monNLto Mono NL T Auto

Océ Netherlands oce02monNL Mono NL TD Auto Y

Océ Netherlands oce02mulRRloTO Multi EN T Auto

Océ Netherlands oce02mulRRlo Multi EN TD Auto Y

Océ Netherlands oce02mulMSlo Multi EN TD Auto Y

Océ Netherlands oce02mulRRbf Multi EN TD Auto

Océ Netherlands oce02mulMSbf Multi EN TD Auto

RALI U Montreal Canada run1 Multi EN TDN Auto Y

RALI U Montreal Canada run2 Multi EN TDN Auto Y

RALI U Montreal Canada run3 Multi EN TDN Auto

SICS/Conexor Sweden/Finland sicsFRFR0 Mono FR TD Auto

SICS/Conexor Sweden/Finland sicsFRFRX0 Mono FR TD Auto Y

SICS/Conexor Sweden/Finland sicsITIT Mono IT TD Auto

SICS/Conexor Sweden/Finland sicsITITX Mono IT TD Auto Y

SICS/Conexor Sweden/Finland siteseeker Mono SV T Auto Y

SICS/Conexor Sweden/Finland sicsSVSV Mono SV TD Auto Y

SICS/Conexor Sweden/Finland sicsSVSVX Mono SV TD Auto Y

Tagmatica France runA Mono FR DN Manual Y

TLR Research USA tlren2es Bi-ES EN TD Auto Y

TLR Research USA tlren2fr Bi-FR EN TD Auto

TLR Research USA tlrde Mono DE TD Auto Y

TLR Research USA tlres Mono ES TD Auto Y

TLR Research USA tlrfr Mono FR TD Auto Y

TLR Research USA tlrit Mono IT TD Auto Y

TLR Research USA tlrnl Mono NL TD Auto Y

TLR Research USA tlrsv Mono SV TD Auto Y

TLR Research USA tlren2multi Multi EN TD Auto Y

Univ. Dortmund Germany GIRTsppc GIRT DE TD Auto Y

Univ. Dortmund Germany GIRTstem GIRT DE TD Auto Y

Univ. Dortmund Germany MLstem Mono DE TD Auto Y

Univ. Exeter United Kingdom exespgemgcntbi Bi-ES DE TD Auto

Univ. Exeter United Kingdom exespengorgbi Bi-ES EN TD Auto

(4)

Univ. Jaen/SINAI Spain UJAMLTDRSV2RR Multi EN TD Auto

Univ. Jaen/SINAI Spain UJAMLTDNORM Multi EN TD Auto

Univ. Jaen/SINAI Spain UJAMLTDRR Multi EN TD Auto

Univ. Jaen/SINAI Spain UJAMLTDRSV2 Multi EN TD Auto Y

Univ. Jaen/SINAI Spain UJAMLTD2RSV2 Multi EN TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKAMEF1 Amaryllis EN TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKAMFF2 Amaryllis FR TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKAMFF1 Amaryllis FR TDN Auto Y

Univ. of Ca. at Berkeley 1 USA BKBIEG2 Bi-DE EN TD Auto

Univ. of Ca. at Berkeley 1 USA BKBIEG1 Bi-DE EN TDN Auto

Univ. of Ca. at Berkeley 1 USA BKBIFG2 Bi-DE FR TD Auto

Univ. of Ca. at Berkeley 1 USA BKBIFG1 Bi-DE FR TDN Auto

Univ. of Ca. at Berkeley 1 USA BKBIRG2 Bi-DE RU TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKBIRG1 Bi-DE RU TDN Auto Y

Univ. of Ca. at Berkeley 1 USA BKBIGF1 Bi-FR DE TD Auto

Univ. of Ca. at Berkeley 1 USA BKBIEF1 Bi-FR EN TD Auto

Univ. of Ca. at Berkeley 1 USA BKBIEF2 Bi-FR EN TDN Auto

Univ. of Ca. at Berkeley 1 USA BKBIRF1 Bi-FR RU TD Auto

Univ. of Ca. at Berkeley 1 USA BKGRGG2 GIRT DE TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKGRGG1 GIRT DE TDN Auto Y

Univ. of Ca. at Berkeley 1 USA BKGREG1 GIRT EN TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKGRRG2 GIRT RU TD Auto Y

Univ. of Ca. at Berkeley 1 USA BKGRRG1 GIRT RU TDN Auto Y

Univ. of Ca. at Berkeley 1 USA BKMLGG2 Mono DE TD Auto

Univ. of Ca. at Berkeley 1 USA BKMLGG1 Mono DE TDN Auto Y

Univ. of Ca. at Berkeley 1 USA BKMLFF2 Mono FR TD Auto

Univ. of Ca. at Berkeley 1 USA BKMLFF1 Mono FR TDN Auto Y

Univ. of Ca. at Berkeley 2 USA bky2biende Bi-DE EN TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2bifrde Bi-DE FR TD Auto

Univ. of Ca. at Berkeley 2 USA bky2bienes Bi-ES EN TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2bidefr Bi-FR DE TD Auto

Univ. of Ca. at Berkeley 2 USA bky2bienfr Bi-FR EN TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2bienfr2 Bi-FR EN TD Auto

Univ. of Ca. at Berkeley 2 USA bky2bienit Bi-IT EN TD Auto

Univ. of Ca. at Berkeley 2 USA bky2biennl Bi-NL EN TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2mode Mono DE TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2moes Mono ES TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2mofr Mono FR TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2moit Mono IT TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2monl Mono NL TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2muen1 Multi EN TD Auto Y

Univ. of Ca. at Berkeley 2 USA bky2muen2 Multi EN TD Auto Y

Univ. of Twente Netherlands tnoen1 Bi-NL EN TD Auto

Univ. of Twente Netherlands tnofifi1 Mono FI TD Auto Y

Univ. of Twente Netherlands tnoutn1 Mono NL TDN Manual Y

Univ. Tampere, Dept. IS Finland finbi2 Bi-FI EN TD Auto Y

Univ. Tampere, Dept. IS Finland bifren Bi-FR EN TD Auto Y

Univ. Tampere, Dept. IS Finland dualge Bi-NL EN TD Auto Y

Univ. Tampere, Dept. IS Finland finmo1 Mono FI TD Auto Y

Univ. Tampere, Dept. IS Finland finmo2 Mono FI TD Auto Y

(5)

Université de Neuchâtel Switzerland UniNEestdn Mono ES TDN Auto Y

Université de Neuchâtel Switzerland UniNEfi1 Mono FI TD Auto Y

Université de Neuchâtel Switzerland UniNEfi2 Mono FI TD Auto Y

Université de Neuchâtel Switzerland UniNEfr Mono FR TD Auto

Université de Neuchâtel Switzerland UniNEfrtdn Mono FR TDN Manual Y

Université de Neuchâtel Switzerland UniNEit Mono IT TD Auto Y

Université de Neuchâtel Switzerland UniNEnl Mono NL TD Auto Y

Université de Neuchâtel Switzerland UniNEm1 Multi EN TD Auto Y

Université de Neuchâtel Switzerland UniNEm2 Multi EN TD Auto Y

Université de Neuchâtel Switzerland UniNEm3 Multi EN TD Auto

University of Alicante Spain irn1 Mono ES TD Auto Y

University of Alicante Spain irn3 Mono ES TD Auto

University of Alicante Spain irn2 Mono ES TDN Auto

University of Alicante Spain irn4 Mono ES TDN Auto Y

University of Amsterdam Netherlands UAmsC02EnAmTTiKW Amaryllis EN TD Auto Y University of Amsterdam Netherlands UAmsC02EnAmTTiRR Amaryllis EN TD Auto Y

University of Amsterdam Netherlands UAmsC02FrAmKW Amaryllis FR N Auto Y

University of Amsterdam Netherlands UAmsC02FrAmTT Amaryllis FR TD Auto Y

University of Amsterdam Netherlands UAmsC02FrAmTTiKW Amaryllis FR TDN Auto Y University of Amsterdam Netherlands UAmsC02EnGeNGram Bi-DE EN TD Auto University of Amsterdam Netherlands UAmsC02EnDuMorph Bi-NL EN TD Auto University of Amsterdam Netherlands UAmsC02EnDuNGiMO Bi-NL EN TD Auto University of Amsterdam Netherlands UAmsC02EnDuNGram Bi-NL EN TD Auto

University of Amsterdam Netherlands UAmsC02GeGiTT GIRT DE TD Auto Y

University of Amsterdam Netherlands UAmsC02GeGiTTiKW GIRT DE TD Auto Y

University of Amsterdam Netherlands UAmsC02GeGiTTiRR GIRT DE TD Auto Y

University of Amsterdam Netherlands UAmsC02EnGiTTiKW GIRT EN TD Auto Y

University of Amsterdam Netherlands UAmsC02EnGiTTiRR GIRT EN TD Auto Y

University of Amsterdam Netherlands UAmsC02GeGeLC2F Mono DE TD Auto

University of Amsterdam Netherlands UAmsC02GeGeNGiMO Mono DE TD Auto Y

University of Amsterdam Netherlands UAmsC02GeGeNGram Mono DE TD Auto

University of Amsterdam Netherlands UAmsC02SpSpNGiSt Mono ES TD Auto Y

University of Amsterdam Netherlands UAmsC02FiFiNGram Mono FI TD Auto Y

University of Amsterdam Netherlands UAmsC02FrFrNGiMO Mono FR TD Auto Y

University of Amsterdam Netherlands UAmsC02ItItNGiMO Mono IT TD Auto Y

University of Amsterdam Netherlands UAmsC02DuDuNGiMO Mono NL TD Auto Y

University of Amsterdam Netherlands UAmsC02DuDuNGram Mono NL TD Auto

University of Amsterdam Netherlands UAmsC02SwSwNGram Mono SV TD Auto Y

University of Salamanca Spain usalNST Mono ES T Auto

University of Salamanca Spain usalNAT Mono ES T Auto

University of Salamanca Spain usalNNTDN Mono ES TDN Auto Y

University of Salamanca Spain usalFNTDN Mono ES TDN Auto Y

XRCE Xerox France xrcegirt1 GIRT EN T Auto Y

(6)

0.0 0.2 0.4 0.6 Recall

0.0

0.2

0.4

0.6 0.8 1.0

Precision

0.3783 U Neuchâtel 0.3762 UC Berkeley 2 0.3409 Eurospider 0.2774 U Jaen/SINAI 0.2331 Océ

CLEF 2002 − Multilingual Task (Average Precision) Automatic, Title + Description, Five Best Groups

(7)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.4158 JHU/APL 0.3502 Clairvoyance 0.3235 Océ

0.2449 IRIT

0.2088 Middlesex U

CLEF 2002 − Bilingual to English Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(8)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.2016 U Tampere 0.2003 JHU/APL

CLEF 2002 − Bilingual to Finnish Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(9)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.4090 UC Berkeley 2 0.3994 U Exeter 0.3756 U Neuchâtel 0.3552 ITC−IRST

0.2794 JHU/APL

CLEF 2002 − Bilingual to Italian Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(10)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.3003 JHU/APL

CLEF 2002 − Bilingual to Swedish Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(11)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.5441 U Neuchâtel 0.5338 UC Berkeley 2 0.5192 JHU/APL 0.4993 TLR Research 0.4980 U Alicante

CLEF 2002 − Spanish Monolingual Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(12)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.5191 UC Berkeley 2 0.4841 U Neuchâtel 0.4558 UC Berkeley 1 0.4535 U Amsterdam 0.4509 JHU/APL

CLEF 2002 − French Monolingual Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(13)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.5028 JHU/APL 0.4878 U Neuchâtel 0.4847 UC Berkeley 2 0.4598 U Amsterdam 0.4447 Hummingbird

CLEF 2002 − Dutch Monolingual Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(14)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

(15)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.4802 U Neuchâtel 0.4396 UC Berkeley 1 0.2681 U Amsterdam

CLEF 2002 − Amaryllis Monolingual Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(16)

0.0 0.2 0.4 0.6 0.8 1.0 Recall

0.0 0.2 0.4

Precision

0.0 0.2 0.4 0.6 0.8 1.0

Recall 0.0

0.2 0.4 0.6 0.8 1.0

Precision

0.2587 UC Berkeley 1 0.1906 U Amsterdam 0.1890 U Dortmund 0.1097 U Hildesheim

CLEF 2002 − GIRT Monolingual Task (Average Precision)

Automatic, Title + Description, Five Best Groups

(17)

JHU/APL, Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.0432 Query 92: 0.4077 Query 93: 0.0001 Query 94: 0.4458 Query 95: 0.1565 Query 96: 0.0007 Query 97: 0.1591 Query 98: 0.0322 Query 99: 0.0758 Query 100: 0.1888 Query 101: 0.1633 Query 102: 0.5017 Query 103: 0.1062 Query 104: 0.3265 Query 105: 0.5721 Query 106: 0.0605 Query 107: 0.1172 Query 108: 0.2891 Query 109: 0.1746 Query 110: 0.1680 Query 111: 0.1382 Query 112: 0.1430 Query 113: 0.0214 Query 114: 0.3196 Query 115: 0.1753 Query 116: 0.1621 Query 117: 0.0334 Query 118: 0.0043 Query 119: 0.6915 Query 120: 0.1655 Query 121: 0.2549 Query 122: 0.1637 Query 123: 0.6085 Query 124: 0.1104 Query 125: 0.0921 Query 126: 0.1512 Query 127: 0.0481 Query 128: 0.0401 Query 129: 0.2482 Query 130: 0.6098 Query 131: 0.5279 Query 132: 0.5204 Query 133: 0.0131 Query 134: 0.1615 Query 135: 0.1195 Query 136: 0.1839 Query 137: 0.0800 Query 138: 0.2859 Query 139: 0.0398 Query 140: 0.2495 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 4729 Interpolated Recall - Precision Averages: at 0.00 0.6876 at 0.10 0.4105 at 0.20 0.3359 at 0.30 0.2832 at 0.40 0.2274 at 0.50 0.1755 at 0.60 0.1524 at 0.70 0.1219 at 0.80 0.0903 at 0.90 0.0579 at 1.00 0.0186 Avg. prec. (non-interpolated) for all rel. documents 0.2070 Precision: At 5 docs: 0.4680 At 10 docs: 0.4180 At 15 docs: 0.3973 At 20 docs: 0.3790 At 30 docs: 0.3513 At 100 docs: 0.2792 At 200 docs: 0.2189 At 500 docs: 0.1425 At 1000 docs: 0.0946 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2650

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run aplmuena

Recall−Precision Values 91100110 Topic number

−1.0

−0.50.0

+0.5

+1.0 Difference (average precision)

Run aplmuena

Comparison to median by topic

(18)

JHU/APL, Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.0832 Query 92: 0.3773 Query 93: 0.0001 Query 94: 0.5247 Query 95: 0.1735 Query 96: 0.0018 Query 97: 0.1872 Query 98: 0.0623 Query 99: 0.0733 Query 100: 0.2563 Query 101: 0.3103 Query 102: 0.3980 Query 103: 0.1100 Query 104: 0.2544 Query 105: 0.5276 Query 106: 0.0806 Query 107: 0.1124 Query 108: 0.2181 Query 109: 0.1477 Query 110: 0.1937 Query 111: 0.1157 Query 112: 0.0753 Query 113: 0.0193 Query 114: 0.3674 Query 115: 0.1810 Query 116: 0.1705 Query 117: 0.0533 Query 118: 0.0032 Query 119: 0.6800 Query 120: 0.0827 Query 121: 0.1442 Query 122: 0.1522 Query 123: 0.6079 Query 124: 0.1322 Query 125: 0.1585 Query 126: 0.1748 Query 127: 0.0726 Query 128: 0.0301 Query 129: 0.2650 Query 130: 0.5426 Query 131: 0.5086 Query 132: 0.4081 Query 133: 0.0045 Query 134: 0.2922 Query 135: 0.1567 Query 136: 0.2169 Query 137: 0.0717 Query 138: 0.3366 Query 139: 0.0608 Query 140: 0.2316 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 4660 Interpolated Recall - Precision Averages: at 0.00 0.7118 at 0.10 0.4276 at 0.20 0.3526 at 0.30 0.2969 at 0.40 0.2424 at 0.50 0.1793 at 0.60 0.1421 at 0.70 0.1111 at 0.80 0.0691 at 0.90 0.0375 at 1.00 0.0099 Avg. prec. (non-interpolated) for all rel. documents 0.2082 Precision: At 5 docs: 0.4480 At 10 docs: 0.4420 At 15 docs: 0.4267 At 20 docs: 0.4050 At 30 docs: 0.3847 At 100 docs: 0.2956 At 200 docs: 0.2285 At 500 docs: 0.1434 At 1000 docs: 0.0932 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2765

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run aplmuenb

−1.0

−0.50.0

+0.5

Run aplmuenb

(19)

Univ. of Ca. at Berkeley 2, Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.1832 Query 92: 0.3782 Query 93: 0.6555 Query 94: 0.4686 Query 95: 0.2080 Query 96: 0.2726 Query 97: 0.4967 Query 98: 0.5991 Query 99: 0.2957 Query 100: 0.4329 Query 101: 0.2298 Query 102: 0.5244 Query 103: 0.5829 Query 104: 0.1913 Query 105: 0.5235 Query 106: 0.1210 Query 107: 0.1798 Query 108: 0.3819 Query 109: 0.1130 Query 110: 0.6656 Query 111: 0.0893 Query 112: 0.5905 Query 113: 0.0067 Query 114: 0.2074 Query 115: 0.3806 Query 116: 0.5823 Query 117: 0.2375 Query 118: 0.0340 Query 119: 0.7770 Query 120: 0.1850 Query 121: 0.4959 Query 122: 0.2526 Query 123: 0.7943 Query 124: 0.0272 Query 125: 0.1486 Query 126: 0.2036 Query 127: 0.6207 Query 128: 0.0385 Query 129: 0.3792 Query 130: 0.4976 Query 131: 0.4905 Query 132: 0.4855 Query 133: 0.1314 Query 134: 0.8288 Query 135: 0.6999 Query 136: 0.8458 Query 137: 0.2403 Query 138: 0.3528 Query 139: 0.2913 Query 140: 0.3931 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5880 Interpolated Recall - Precision Averages: at 0.00 0.8646 at 0.10 0.6252 at 0.20 0.5606 at 0.30 0.4992 at 0.40 0.4397 at 0.50 0.3898 at 0.60 0.3364 at 0.70 0.2759 at 0.80 0.2115 at 0.90 0.1229 at 1.00 0.0136 Avg. prec. (non-interpolated) for all rel. documents 0.3762 Precision: At 5 docs: 0.5960 At 10 docs: 0.5880 At 15 docs: 0.5867 At 20 docs: 0.5800 At 30 docs: 0.5587 At 100 docs: 0.4092 At 200 docs: 0.3196 At 500 docs: 0.1961 At 1000 docs: 0.1176 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4153

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run bky2muen1

−1.0

−0.50.0

+0.5

Run bky2muen1

(20)

Univ. of Ca. at Berkeley 2, Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.1036 Query 92: 0.3794 Query 93: 0.5229 Query 94: 0.3986 Query 95: 0.2249 Query 96: 0.2914 Query 97: 0.4919 Query 98: 0.4665 Query 99: 0.3154 Query 100: 0.4484 Query 101: 0.2135 Query 102: 0.5181 Query 103: 0.5901 Query 104: 0.1696 Query 105: 0.5010 Query 106: 0.1230 Query 107: 0.1578 Query 108: 0.4011 Query 109: 0.1191 Query 110: 0.6241 Query 111: 0.0814 Query 112: 0.5816 Query 113: 0.0069 Query 114: 0.2176 Query 115: 0.3373 Query 116: 0.5694 Query 117: 0.2697 Query 118: 0.0379 Query 119: 0.7676 Query 120: 0.1668 Query 121: 0.5218 Query 122: 0.1923 Query 123: 0.7845 Query 124: 0.0263 Query 125: 0.1055 Query 126: 0.1748 Query 127: 0.5701 Query 128: 0.0321 Query 129: 0.3330 Query 130: 0.4799 Query 131: 0.5198 Query 132: 0.3482 Query 133: 0.1085 Query 134: 0.8277 Query 135: 0.6924 Query 136: 0.7821 Query 137: 0.2238 Query 138: 0.4248 Query 139: 0.3171 Query 140: 0.2903 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5765 Interpolated Recall - Precision Averages: at 0.00 0.8159 at 0.10 0.6061 at 0.20 0.5418 at 0.30 0.4890 at 0.40 0.4322 at 0.50 0.3731 at 0.60 0.3106 at 0.70 0.2509 at 0.80 0.1916 at 0.90 0.1110 at 1.00 0.0102 Avg. prec. (non-interpolated) for all rel. documents 0.3570 Precision: At 5 docs: 0.5760 At 10 docs: 0.5540 At 15 docs: 0.5600 At 20 docs: 0.5580 At 30 docs: 0.5387 At 100 docs: 0.4044 At 200 docs: 0.3148 At 500 docs: 0.1915 At 1000 docs: 0.1153 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4055

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run bky2muen2

−1.0

−0.50.0

+0.5

Run bky2muen2

(21)

Eurospider IT AG, Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.2240 Query 92: 0.3764 Query 93: 0.6842 Query 94: 0.5001 Query 95: 0.1830 Query 96: 0.2029 Query 97: 0.2717 Query 98: 0.4926 Query 99: 0.1601 Query 100: 0.4937 Query 101: 0.3323 Query 102: 0.5128 Query 103: 0.4731 Query 104: 0.1513 Query 105: 0.5629 Query 106: 0.1669 Query 107: 0.1734 Query 108: 0.3487 Query 109: 0.1563 Query 110: 0.5965 Query 111: 0.2235 Query 112: 0.5838 Query 113: 0.0061 Query 114: 0.2488 Query 115: 0.2629 Query 116: 0.6334 Query 117: 0.2071 Query 118: 0.0214 Query 119: 0.7728 Query 120: 0.1728 Query 121: 0.1982 Query 122: 0.1219 Query 123: 0.7282 Query 124: 0.1566 Query 125: 0.3115 Query 126: 0.4163 Query 127: 0.4233 Query 128: 0.0903 Query 129: 0.3469 Query 130: 0.1722 Query 131: 0.1369 Query 132: 0.3890 Query 133: 0.2042 Query 134: 0.8435 Query 135: 0.4988 Query 136: 0.8520 Query 137: 0.2787 Query 138: 0.3670 Query 139: 0.2943 Query 140: 0.3754 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5698 Interpolated Recall - Precision Averages: at 0.00 0.8629 at 0.10 0.6314 at 0.20 0.5489 at 0.30 0.4741 at 0.40 0.4122 at 0.50 0.3466 at 0.60 0.2913 at 0.70 0.2326 at 0.80 0.1468 at 0.90 0.0856 at 1.00 0.0105 Avg. prec. (non-interpolated) for all rel. documents 0.3480 Precision: At 5 docs: 0.6480 At 10 docs: 0.6260 At 15 docs: 0.6067 At 20 docs: 0.5700 At 30 docs: 0.5447 At 100 docs: 0.3940 At 200 docs: 0.2992 At 500 docs: 0.1846 At 1000 docs: 0.1140 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3958

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run EAN2MDF4

−1.0

−0.50.0

+0.5

Run EAN2MDF4

(22)

Eurospider IT AG, Multilingual German, Auto, TD Average precision (individual queries): Query 91: 0.0818 Query 92: 0.3786 Query 93: 0.7509 Query 94: 0.6713 Query 95: 0.0315 Query 96: 0.4379 Query 97: 0.3165 Query 98: 0.4991 Query 99: 0.0544 Query 100: 0.4143 Query 101: 0.4902 Query 102: 0.3781 Query 103: 0.4825 Query 104: 0.1027 Query 105: 0.6118 Query 106: 0.1426 Query 107: 0.1767 Query 108: 0.3006 Query 109: 0.1408 Query 110: 0.6960 Query 111: 0.0538 Query 112: 0.5966 Query 113: 0.0350 Query 114: 0.2745 Query 115: 0.1414 Query 116: 0.5209 Query 117: 0.1757 Query 118: 0.0602 Query 119: 0.5612 Query 120: 0.0470 Query 121: 0.3871 Query 122: 0.0761 Query 123: 0.7615 Query 124: 0.2050 Query 125: 0.4068 Query 126: 0.3548 Query 127: 0.5857 Query 128: 0.0562 Query 129: 0.3353 Query 130: 0.3197 Query 131: 0.1067 Query 132: 0.4175 Query 133: 0.4033 Query 134: 0.7047 Query 135: 0.4059 Query 136: 0.9091 Query 137: 0.0303 Query 138: 0.5031 Query 139: 0.2298 Query 140: 0.1761 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5368 Interpolated Recall - Precision Averages: at 0.00 0.7893 at 0.10 0.6033 at 0.20 0.5375 at 0.30 0.4587 at 0.40 0.4050 at 0.50 0.3508 at 0.60 0.2810 at 0.70 0.2310 at 0.80 0.1549 at 0.90 0.0815 at 1.00 0.0129 Avg. prec. (non-interpolated) for all rel. documents 0.3400 Precision: At 5 docs: 0.6160 At 10 docs: 0.5860 At 15 docs: 0.5813 At 20 docs: 0.5520 At 30 docs: 0.5200 At 100 docs: 0.3940 At 200 docs: 0.2886 At 500 docs: 0.1735 At 1000 docs: 0.1074 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3802

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run EIT2MDC3

−1.0

−0.50.0

+0.5

Run EIT2MDC3

(23)

Eurospider IT AG, Multilingual German, Auto, TD Average precision (individual queries): Query 91: 0.0820 Query 92: 0.3775 Query 93: 0.7410 Query 94: 0.6401 Query 95: 0.0423 Query 96: 0.4344 Query 97: 0.3248 Query 98: 0.4954 Query 99: 0.0521 Query 100: 0.4017 Query 101: 0.4538 Query 102: 0.4011 Query 103: 0.4521 Query 104: 0.1017 Query 105: 0.6143 Query 106: 0.1569 Query 107: 0.1793 Query 108: 0.3092 Query 109: 0.1387 Query 110: 0.7302 Query 111: 0.0609 Query 112: 0.6056 Query 113: 0.0347 Query 114: 0.2866 Query 115: 0.1566 Query 116: 0.5285 Query 117: 0.1578 Query 118: 0.0591 Query 119: 0.5937 Query 120: 0.0418 Query 121: 0.3411 Query 122: 0.0807 Query 123: 0.7649 Query 124: 0.2238 Query 125: 0.3958 Query 126: 0.3685 Query 127: 0.5470 Query 128: 0.0532 Query 129: 0.3258 Query 130: 0.3432 Query 131: 0.1302 Query 132: 0.4066 Query 133: 0.4093 Query 134: 0.6917 Query 135: 0.3865 Query 136: 0.9246 Query 137: 0.0283 Query 138: 0.5329 Query 139: 0.2370 Query 140: 0.1999 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5411 Interpolated Recall - Precision Averages: at 0.00 0.7882 at 0.10 0.6047 at 0.20 0.5397 at 0.30 0.4587 at 0.40 0.4120 at 0.50 0.3503 at 0.60 0.2826 at 0.70 0.2308 at 0.80 0.1615 at 0.90 0.0820 at 1.00 0.0137 Avg. prec. (non-interpolated) for all rel. documents 0.3409 Precision: At 5 docs: 0.6160 At 10 docs: 0.6040 At 15 docs: 0.5853 At 20 docs: 0.5520 At 30 docs: 0.5247 At 100 docs: 0.3958 At 200 docs: 0.2911 At 500 docs: 0.1746 At 1000 docs: 0.1082 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3836

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run EIT2MDF3

−1.0

−0.50.0

+0.5

Run EIT2MDF3

(24)

Eurospider IT AG, Multilingual German, Auto, TDN Average precision (individual queries): Query 91: 0.1523 Query 92: 0.3837 Query 93: 0.7426 Query 94: 0.6015 Query 95: 0.1727 Query 96: 0.3906 Query 97: 0.6104 Query 98: 0.5064 Query 99: 0.1594 Query 100: 0.4017 Query 101: 0.4398 Query 102: 0.3223 Query 103: 0.4882 Query 104: 0.0930 Query 105: 0.5976 Query 106: 0.2142 Query 107: 0.1751 Query 108: 0.3019 Query 109: 0.1399 Query 110: 0.6563 Query 111: 0.0681 Query 112: 0.5486 Query 113: 0.0436 Query 114: 0.5335 Query 115: 0.1085 Query 116: 0.4369 Query 117: 0.1359 Query 118: 0.1013 Query 119: 0.6094 Query 120: 0.0580 Query 121: 0.2838 Query 122: 0.1416 Query 123: 0.6191 Query 124: 0.2511 Query 125: 0.4001 Query 126: 0.4946 Query 127: 0.5298 Query 128: 0.0248 Query 129: 0.2733 Query 130: 0.5268 Query 131: 0.1900 Query 132: 0.3862 Query 133: 0.4671 Query 134: 0.6831 Query 135: 0.4539 Query 136: 0.7005 Query 137: 0.0306 Query 138: 0.7074 Query 139: 0.1972 Query 140: 0.2165 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5620 Interpolated Recall - Precision Averages: at 0.00 0.8652 at 0.10 0.6532 at 0.20 0.5573 at 0.30 0.4966 at 0.40 0.4081 at 0.50 0.3532 at 0.60 0.2893 at 0.70 0.2212 at 0.80 0.1523 at 0.90 0.0896 at 1.00 0.0128 Avg. prec. (non-interpolated) for all rel. documents 0.3554 Precision: At 5 docs: 0.6880 At 10 docs: 0.6520 At 15 docs: 0.6107 At 20 docs: 0.5950 At 30 docs: 0.5593 At 100 docs: 0.4160 At 200 docs: 0.3084 At 500 docs: 0.1849 At 1000 docs: 0.1124 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3934

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run EIT2MNF3

−1.0

−0.50.0

+0.5

Run EIT2MNF3

(25)

Eurospider IT AG, Multilingual German, Auto, TDN Average precision (individual queries): Query 91: 0.0996 Query 92: 0.3913 Query 93: 0.7536 Query 94: 0.5366 Query 95: 0.1531 Query 96: 0.4614 Query 97: 0.6221 Query 98: 0.5187 Query 99: 0.0862 Query 100: 0.4378 Query 101: 0.4643 Query 102: 0.2965 Query 103: 0.5177 Query 104: 0.0784 Query 105: 0.6062 Query 106: 0.1576 Query 107: 0.1470 Query 108: 0.2436 Query 109: 0.0655 Query 110: 0.6546 Query 111: 0.0352 Query 112: 0.5817 Query 113: 0.0391 Query 114: 0.4816 Query 115: 0.0159 Query 116: 0.3221 Query 117: 0.0773 Query 118: 0.0763 Query 119: 0.4669 Query 120: 0.0216 Query 121: 0.5401 Query 122: 0.1269 Query 123: 0.8311 Query 124: 0.1584 Query 125: 0.4183 Query 126: 0.4415 Query 127: 0.6387 Query 128: 0.0220 Query 129: 0.2774 Query 130: 0.6096 Query 131: 0.2074 Query 132: 0.3942 Query 133: 0.4417 Query 134: 0.7650 Query 135: 0.3497 Query 136: 0.9119 Query 137: 0.0536 Query 138: 0.7498 Query 139: 0.1964 Query 140: 0.1533 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 5188 Interpolated Recall - Precision Averages: at 0.00 0.8267 at 0.10 0.6371 at 0.20 0.5431 at 0.30 0.4725 at 0.40 0.4071 at 0.50 0.3524 at 0.60 0.2995 at 0.70 0.2402 at 0.80 0.1620 at 0.90 0.1031 at 1.00 0.0081 Avg. prec. (non-interpolated) for all rel. documents 0.3539 Precision: At 5 docs: 0.6680 At 10 docs: 0.6560 At 15 docs: 0.6173 At 20 docs: 0.6010 At 30 docs: 0.5607 At 100 docs: 0.4066 At 200 docs: 0.2994 At 500 docs: 0.1720 At 1000 docs: 0.1038 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3906

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run EIT2MNU1

−1.0

−0.50.0

+0.5

Run EIT2MNU1

(26)

IRIT, Toulouse, Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.0282 Query 92: 0.1103 Query 93: 0.0000 Query 94: 0.0956 Query 95: 0.0244 Query 96: 0.0005 Query 97: 0.0409 Query 98: 0.0000 Query 99: 0.0371 Query 100: 0.1560 Query 101: 0.0000 Query 102: 0.1896 Query 103: 0.0204 Query 104: 0.2053 Query 105: 0.4195 Query 106: 0.0206 Query 107: 0.0974 Query 108: 0.0627 Query 109: 0.0541 Query 110: 0.0665 Query 111: 0.1842 Query 112: 0.1017 Query 113: 0.0088 Query 114: 0.0280 Query 115: 0.0184 Query 116: 0.0480 Query 117: 0.0027 Query 118: 0.0000 Query 119: 0.1446 Query 120: 0.1153 Query 121: 0.0452 Query 122: 0.0407 Query 123: 0.1617 Query 124: 0.0450 Query 125: 0.0186 Query 126: 0.1140 Query 127: 0.0001 Query 128: 0.0015 Query 129: 0.0831 Query 130: 0.3751 Query 131: 0.0631 Query 132: 0.0000 Query 133: 0.0002 Query 134: 0.1574 Query 135: 0.0042 Query 136: 0.1969 Query 137: 0.0313 Query 138: 0.0700 Query 139: 0.0234 Query 140: 0.0667 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 1537 Interpolated Recall - Precision Averages: at 0.00 0.6697 at 0.10 0.2809 at 0.20 0.1402 at 0.30 0.0542 at 0.40 0.0326 at 0.50 0.0197 at 0.60 0.0067 at 0.70 0.0000 at 0.80 0.0000 at 0.90 0.0000 at 1.00 0.0000 Avg. prec. (non-interpolated) for all rel. documents 0.0756 Precision: At 5 docs: 0.5000 At 10 docs: 0.4000 At 15 docs: 0.3400 At 20 docs: 0.3140 At 30 docs: 0.2613 At 100 docs: 0.1378 At 200 docs: 0.0881 At 500 docs: 0.0480 At 1000 docs: 0.0307 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1165

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run iritMEn2All

−1.0

−0.50.0

+0.5

Run iritMEn2All

(27)

National Taiwan Univ., Multilingual English, Auto, TD Average precision (individual queries): Query 91: 0.0006 Query 92: 0.0415 Query 93: 0.0000 Query 94: 0.0358 Query 95: 0.0293 Query 96: 0.0004 Query 97: 0.0037 Query 98: 0.0005 Query 99: 0.0140 Query 100: 0.0044 Query 101: 0.0241 Query 102: 0.0260 Query 103: 0.0077 Query 104: 0.0183 Query 105: 0.0376 Query 106: 0.0101 Query 107: 0.0090 Query 108: 0.0051 Query 109: 0.0135 Query 110: 0.0132 Query 111: 0.0287 Query 112: 0.0224 Query 113: 0.0004 Query 114: 0.0002 Query 115: 0.0296 Query 116: 0.0724 Query 117: 0.0026 Query 118: 0.0000 Query 119: 0.0654 Query 120: 0.0077 Query 121: 0.0060 Query 122: 0.0030 Query 123: 0.0227 Query 124: 0.0084 Query 125: 0.0027 Query 126: 0.0316 Query 127: 0.0010 Query 128: 0.0009 Query 129: 0.0369 Query 130: 0.0083 Query 131: 0.0198 Query 132: 0.0000 Query 133: 0.0072 Query 134: 0.0378 Query 135: 0.0000 Query 136: 0.0046 Query 137: 0.0027 Query 138: 0.0915 Query 139: 0.0097 Query 140: 0.0473 Overall statistics (for 50 queries): Total number of documents over all queries Retrieved: 50000 Relevant: 8068 Rel_ret: 1083 Interpolated Recall - Precision Averages: at 0.00 0.3098 at 0.10 0.0565 at 0.20 0.0161 at 0.30 0.0068 at 0.40 0.0015 at 0.50 0.0013 at 0.60 0.0000 at 0.70 0.0000 at 0.80 0.0000 at 0.90 0.0000 at 1.00 0.0000 Avg. prec. (non-interpolated) for all rel. documents 0.0173 Precision: At 5 docs: 0.1440 At 10 docs: 0.1440 At 15 docs: 0.1320 At 20 docs: 0.1280 At 30 docs: 0.1027 At 100 docs: 0.0664 At 200 docs: 0.0488 At 500 docs: 0.0309 At 1000 docs: 0.0217 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.0535

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run NTUmulti01

−1.0

−0.50.0

+0.5

Run NTUmulti01

(28)

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run NTUmulti02

−1.0

−0.50.0

+0.5

Run NTUmulti02

(29)

0.00.20.40.0

0.2

0.4

0.6

0.81.0

Precision

Run NTUmulti03

−1.0

−0.50.0

+0.5