Chapter 8 Conclusion and Future Work 125
C.2 Table Results for MovieLens
k= 1 k= 3 k= 5
FB 0.46013 0.5329 0.5797
LM 0.5192 0.5231 0.5206
δF B 41/23 p=0.0335(+) 31/33 p=0.9005(=) 24/40 p=0.0607(=)
LMW 0.5294 0.5190 0.5168
δF B 43/21 p=0.0086(+) 32/32 p=1(=) 22/42 p=0.0175(-)
δLM 16/15 p=1(=) 20/19 p=1(=) 22/17 p=0.5218(=)
LM-MF 0.5554 0.6156 0.5606
δF B 42/22 p=0.0175(+) 42/22 p=0.0175(+) 29/34 p=0.6142(=) δLM 38/19 p=0.0171(+) 42/19 p=0.0048(+) 40/21 p=0.0211(+)
LM-MF-Reg 0.5936 0.5801 0.5855
δF B 46/18 p=0.0007(+) 39/25 p=0.1041(=) 31/32 p=1(=) δLM 42/21 p=0.0117(+) 40/21 p=0.0211(+) 45/17 p=0.0006(+) Table C.3: NDCG@k results on meta-mining for the Full Cold Start setting. For each method, we give the comparison results against the full memory-base and LambdaMART methods in the rows denoted byδF B andδLM respectively. The table explanation is as in table C.2.
δU B 247/223 p=0.2887(=) 233/238 p=0.8537(=) 1392/1600 p=0.0001(-) 1545/1467 p=0.1606(=)
LMW 0.6252 0.6241 0.6455 0.6450
δU B 254/216 p=0.0878(=) 243/228 p=0.5188(=) 1559/1421 p=0.0120(+) 1546/1465 p=0.1448(=) δLM 160/153 p=0.7345(=) 136/133 p=0.9029(=) 1750/1021 p=0.0000(+) 690/715 p=0.5219(=)
LM-MF 0.6439 0.6455 0.6694 0.6700
δU B 267/203 p=0.0036(+) 253/218 p=0.1171(=) 1673/1303 p=0.0000(+) 1756/1253 p=0.0000(+) δLM 267/168 p=2e−06(+) 265/170 p=6e−06(+) 1935/950 p=0.0000(+) 1622/1140 p=0.0000(+)
LM-MF-Reg 0.6503 0.6581 0.6694 0.6705
δU B 279/190 p=4e−05(+) 276/194 p=0.0001(+) 1715/1257 p=0.0000(+) 1725/1283 p=0.0000(+) δLM 270/171 p=3e−06(+) 278/159 p=0.0000(+) 1982/862 p=0.0000(+) 1606/1081 p=0.0000(+)
Table C.4: NDCG@k results on the two MovieLens datasets for the User Cold Start setting. For each method, we give the comparison results against the user memory-based and LambdaMART methods in the rows denoted by δU B and δLM respectively. More precisely we report the numbers of wins/losses, thep-values of the McNemar’s test on these values, and denote by (+) a statistically significant improvement, by (=) no performance difference and by (-) a significant loss. In bold, the best method for a givenk.
100K 1M
k= 5 k= 10 k= 5 k= 10
FB 0.5452 0.5723 0.5339 0.5262
LM 0.5486 0.5641 0.5588 0.5597
δF B 238/231 p=0.7817(=) 244/227 p=0.4609(=) 1632/1367 p=1e−06(+) 1614/1403 p=0.0001(+)
LMW 0.5549 0.5622 0.55737 0.5631
δF B 244/225 p=0.4058(=) 221/249 p=0.2129(=) 1620/1377 p=9e−06(+) 1640/1376 p=1e−06(+) δLM 91/58 p=0.0087(+) 91/94 p=0.8830(=) 1105/1170 p=0.1796(=) 655/496 p=3e−06(+)
LM-MF 0.5893 0.5876 0.5733 0.5750
δF B 266/204 p=0.0048(+) 247/223 p=0.2887(=) 1720/1279 p=0.0000(+) 1711/1305 p=0.0000(+) δLM 272/173 p=3e−06(+) 271/188 p=0.0001(+) 1587/1219 p=0.0000(+) 1573/1187 p=0.0000(+)
LM-MF-Reg 0.5699 0.57865 0.5736 0.5683
δF B 261/207 p=0.0142(+) 250/220 p=0.1810(=) 1714/1288 p=0.0000(+) 1647/1368 p=4e−07(+) δLM 251/204 p=0.0310(+) 243/204 p=0.0722(=) 1596/1219 p=0.0000(+) 1520/1281 p=0.0000(+)
Table C.5: NDCG@k results on the two MovieLens datasets for the Full Cold Start set-ting. For each method, we give the comparison results against the full memory-base and LambdaMART methods in the rows denoted by δF B and δLM respectively. The table annotation is as before.
C.2. Table Results for MovieLens
Jacob Abernethy, Francis Bach, Theodoros Evgeniou, and Jean-Philippe Vert. Low-rank matrix factorization with attributes. arXiv preprint cs/0611124, 2006.
Gediminas Adomavicius and Alexander Tuzhilin. Toward the next generation of recom-mender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans.
on Knowl. and Data Eng., 17:734–749, June 2005. ISSN 1041-4347.
Gediminas Adomavicius, Nikos Manouselis, and YoungOk Kwon. Multi-criteria recom-mender systems. In Recommender systems handbook, pages 769–803. Springer, 2011.
Deepak Agarwal and Bee-Chung Chen. Regression-based latent factor models. In Pro-ceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’09, pages 19–28, New York, NY, USA, 2009. ACM. ISBN 978-1-60558-495-9. doi: 10.1145/1557019.1557029.
Deepak Agarwal and Bee-Chung Chen. flda: matrix factorization through latent dirichlet allocation. In Proceedings of the third ACM international conference on Web search and data mining, WSDM ’10, pages 91–100, New York, NY, USA, 2010. ACM. ISBN 978-1-60558-889-6. doi: 10.1145/1718487.1718499.
David W Aha. Generalizing from case studies: A case study. InProc. of the 9th Interna-tional Conference on Machine Learning, pages 1–10, 1992.
David W. Aha, Dennis Kibler, and Marc K. Albert. Instance-based learning algo-rithms. Mach. Learn., 6(1):37–66, January 1991. ISSN 0885-6125. doi: 10.1023/A:
1022689900470.
Shawkat Ali and Kate Smith-Miles. A meta-learning approach to automatic kernel selec-tion for support vector machines. Neurocomputing, 70(1):173–186, 2006.
Bibliography
Shawkat Ali and Kate Smith-Miles. Kernel width selection for svm classification: a meta-learning approach. Mathematical Methods for Knowledge Discovery and Data Mining, page 101, 2007.
Michael L Anderson and Tim Oates. A review of recent research in metareasoning and metalearning. AI Magazine, 28(1):7–16, 2007.
Hiroki Arimura. Efficient algorithms for mining frequent and closed patterns from semi-structured data. In PAKDD’08: Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining, pages 2–13, Berlin, Heidelberg, 2008.
Springer-Verlag. ISBN 3-540-68124-8, 978-3-540-68124-3.
Franz Baader, Diego Calvanese, Deborah L. McGuinness, Daniele Nardi, and Peter F.
Patel-Schneider, editors. The Description Logic Handbook: Theory, Implementation, and Applications, 2003. Cambridge University Press. ISBN 0-521-78176-0.
Robert M Bell and Yehuda Koren. Scalable collaborative filtering with jointly derived neighborhood interpolation weights. InData Mining, 2007. ICDM 2007. Seventh IEEE International Conference on, pages 43–52. IEEE, 2007.
Hilan Bensusan and Alexandros Kalousis. Estimating the predictive accuracy of a classifier.
InProceedings of the 12th European Conference on Machine Learning, EMCL ’01, pages 25–36, London, UK, UK, 2001. Springer-Verlag. ISBN 3-540-42536-5.
Abraham Bernstein, Foster Provost, and Shawndra Hill. Toward intelligent assistance for a data mining process: An ontology-based approach for cost-sensitive classification.
Knowledge and Data Engineering, IEEE Transactions on, 17(4):503–518, 2005.
Christopher M Bishop and Nasser M Nasrabadi.Pattern recognition and machine learning, volume 1. springer New York, 2006.
Jagadeesh Chandra Bose and Wil M. Aalst. Abstractions in process mining: A taxonomy of patterns. In Proceedings of the 7th International Conference on Business Process Management, pages 159–175, Berlin, Heidelberg, 2009. Springer-Verlag. ISBN 978-3-642-03847-1. doi: http://dx.doi.org/10.1007/978-3-642-03848-8\ 12.
Pavel Brazdil, Christophe Giraud-Carrier, Carlos Soares, and Ricardo Vilalta. Metalearn-ing: Applications to Data Mining. Springer Publishing Company, Incorporated, 1 edi-tion, 2008. ISBN 3540732624, 9783540732624.
Pavel B Brazdil, Carlos Soares, and Joaquim Pinto Da Costa. Ranking learning algorithms:
Using ibl and meta-learning on accuracy and time results. Machine Learning, 50(3):
251–277, 2003.
Leo Breiman. Random forests. Machine learning, 45(1):5–32, 2001.
Leo Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regres-sion Trees. Statistics/Probability Series. Wadsworth Publishing Company, Belmont, California, U.S.A., 1984.
Bjrn Bringmann. Matching in frequent tree discovery. InProceedings of the Fourth IEEE International Conference on Data Mining (ICDM04, pages 335–338, 2004.
Christopher JC Burges. From ranknet to lambdarank to lambdamart: An overview.
Learning, 11:23–581, 2010.
Christopher JC Burges, Krysta Marie Svore, Paul N Bennett, Andrzej Pastusiak, and Qiang Wu. Learning to rank using an ensemble of lambda-gradient models. Journal of Machine Learning Research-Proceedings Track, 14:25–35, 2011.
Robin Burke. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction, 12(4):331–370, November 2002. ISSN 0924-1868. doi:
10.1023/A:1021240730564.
Deng Cai, Xiaofei He, Jiawei Han, and Thomas S Huang. Graph regularized nonnegative matrix factorization for data representation.Pattern Analysis and Machine Intelligence, IEEE Transactions on, 33(8):1548–1560, 2011.
Pete Chapman, Julian Clinton, Randy Kerber, Thomas Khabaza, Thomas Reinartz, Colin Shearer, and Rudiger Wirth. Crisp-dm 1.0 step-by-step data mining guide. Technical report, The CRISP-DM consortium, August 2000.
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3):
273–297, 1995.
Steven P Coy, Bruce L Golden, George C Runger, and Edward A Wasil. Using experi-mental design to find effective parameter settings for heuristics. Journal of Heuristics, 7(1):77–97, 2001.
Pedro Domingos. A unified bias-variance decomposition. InProceedings of 17th Interna-tional Conference on Machine Learning. Stanford CA Morgan Kaufmann, pages 231–
238, 2000.
Bibliography
Pinar Donmez, Krysta M Svore, and Christopher JC Burges. On the local optimality of lambdarank. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 460–467. ACM, 2009.
Besik Dundua, Temur Kutsia, and Mircea Marin. Strategies in prholog.EPTCS, 15:32–43, 2010.
Saˇso Dˇzeroski. Relational data mining. Springer, 2010.
Theodoros Evgeniou and Massimiliano Pontil. Regularized multi–task learning. In Pro-ceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 109–117. ACM, 2004.
Ronald Fagin, Ravi Kumar, and D. Sivakumar. Comparing top k lists. InProceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms, SODA ’03, pages 28–36, Philadelphia, PA, USA, 2003. Society for Industrial and Applied Mathematics.
ISBN 0-89871-538-5.
Usama Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth. From data mining to knowledge discovery in databases. AI magazine, 17(3):37, 1996.
Jeffrey Forbes and David Andre. Practical reinforcement learning in continuous domains.
Technical report, Berkeley, CA, USA, 2000.
Jerome H Friedman. Greedy function approximation: a gradient boosting machine.Annals of Statistics, pages 1189–1232, 2001.
Johannes F¨urnkranz and Eyke H¨ullermeier. Preference learning. Springer, 2010.
Yolanda Gil, Ewa Deelman, Mark Ellisman, Thomas Fahringer, Geoffrey Fox, Dennis Gannon, Carole Goble, Miron Livny, Luc Moreau, and Jim Myers. Examining the challenges of scientific workflows. Computer, 40(12):24–32, 2007.
Christophe Giraud-Carrier. Metalearning-a tutorial. InProceedings of the 7th international conference on machine learning and applications, 2008.
Christophe Giraud-Carrier, Ricardo Vilalta, and Pavel Brazdil. Introduction to the special issue on meta-learning. Mach. Learn., 54(3):187–193, March 2004. ISSN 0885-6125. doi:
10.1023/B:MACH.0000015878.60765.42.
Jonathan Gratch and Gerald Dejong. Composer: A probabilistic solution to the utility problem in speed-up learning. 1992.
Gianluigi Greco, Antonella Guzzo, and Luigi Pontieri. Mining taxonomies of process models. Data Knowl. Eng., 67(1):74–102, 2008. ISSN 0169-023X. doi: http://dx.doi.
org/10.1016/j.datak.2008.06.010.
I.M. Guyon, S.R. Gunn, M. Nikravesh, and L. Zadeh, editors. Feature Extraction, Foun-dations and Applications. Springer, 2006.
M. A. Hall. Correlation-based Feature Subset Selection for Machine Learning. PhD thesis, University of Waikato, Hamilton, New Zealand, 1998.
Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. The weka data mining software: An update. SIGKDD Explor. Newsl., 11(1):10–18, November 2009. ISSN 1931-0145. doi: 10.1145/1656274.1656278.
Jiawei Han, Micheline Kamber, and Jian Pei. Data mining: concepts and techniques.
Morgan kaufmann, 2006.
Jiawei Han, Hong Cheng, Dong Xin, and Xifeng Yan. Frequent pattern mining: current status and future directions. Data Mining and Knowledge Discovery, 15, 2007.
Melanie Hilario. Model complexity and algorithm selection in classification. In Proceed-ings of the 5th International Conference on Discovery Science, DS ’02, pages 113–126, London, UK, UK, 2002. Springer-Verlag. ISBN 3-540-00188-3.
Melanie Hilario and Alexandros Kalousis. Fusion of meta-knowledge and meta-data for case-based model selection. InProceedings of the 5th European Conference on Principles of Data Mining and Knowledge Discovery, PKDD ’01, pages 180–191, London, UK, UK, 2001. Springer-Verlag. ISBN 3-540-42534-9.
Melanie Hilario, Alexandros Kalousis, Phong Nguyen, and Adam Woznica. A data min-ing ontology for algorithm selection and meta-learnmin-ing. InProc of the ECML/PKDD09 Workshop on Third Generation Data Mining: Towards Service-oriented Knowledge Dis-covery, 2009.
Melanie Hilario, Phong Nguyen, Huyen Do, Adam Woznica, and Alexandros Kalousis.
Ontology-based meta-mining of knowledge discovery workflows. In N. Jankowski, W. Duch, and K. Grabczewski, editors, Meta-Learning in Computational Intelligence.
Springer, 2011.
Bibliography
T. K. Ho and M. Basu. Complexity measures of supervised classification problems. IEEE Trans. Pattern Anal. Mach. Intell., 24(3):289–300, March 2002. ISSN 0162-8828. doi:
10.1109/34.990132. URL http://dx.doi.org/10.1109/34.990132.
Tin Kam Ho and Mitra Basu. Data complexity in pattern recognition. Springer, 2006.
J¨org Hoffmann. Ff: The fast-forward planning system. AI magazine, 22(3):57, 2001.
Thomas Hofmann, Jan Puzicha, and Michael I Jordan. Learning from dyadic data. Ad-vances in neural information processing systems, pages 466–472, 1999.
Frank Hutter, Holger H Hoos, Kevin Leyton-Brown, and Thomas St¨utzle. Paramils: an au-tomatic algorithm configuration framework. Journal of Artificial Intelligence Research, 36(1):267–306, 2009.
Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Sequential model-based opti-mization for general algorithm configuration. InLearning and Intelligent Optimization, pages 507–523. Springer, 2011.
Wayne Iba and Pat Langley. Induction of one-level decision trees. InML, pages 233–240.
Citeseer, 1992.
Prateek Jain and Inderjit S Dhillon. Provable inductive matrix completion. arXiv preprint arXiv:1306.0626, 2013.
Kalervo J¨arvelin and Jaana Kek¨al¨ainen. Ir evaluation methods for retrieving highly rele-vant documents. InProceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 41–48. ACM, 2000.
George H. John and Pat Langley. Estimating continuous distributions in bayesian classi-fiers. In Eleventh Conference on Uncertainty in Artificial Intelligence, pages 338–345, San Mateo, 1995. Morgan Kaufmann.
Alexandros Kalousis. Algorithm Selection via Metalearning. PhD thesis, University of Geneva, 2002.
Alexandros Kalousis and Melanie Hilario. Model selection via meta-learning: A com-parative study. International Journal on Artificial Intelligence Tools, 10(4):525–554, 2001.
Alexandros Kalousis and Theoharis Theoharis. Noemon: Design, implementation and performance results of an intelligent assistant for classifier selection. Intell. Data Anal., 3(5):319–337, 1999.
Alexandros Kalousis, Joao Gama, and Melanie Hilario. On data and algorithms: Under-standing inductive performance. Machine Learning, 54(3):275–312, 2004.
Gordon V Kass. An exploratory technique for investigating large quantities of categorical data. Applied statistics, pages 119–127, 1980.
C. Maria Keet, Agnieszka Lawrynowicz, Claudia d’Amato, and Melanie Hilario. Modeling issues, choices in the data mining optimization ontology. In OWLED, 2013.
C Maria Keet, Agnieszka Lawrynowicz, Claudia dAmato, Alexandros Kalousis, Phong Nguyen, Raul Palma, Robert Stevens, and Melanie Hilario. The data mining optimiza-tion ontology. Web Semantics: Science, Services and Agents on the World Wide Web, 2015.
J¨org-Uwe Kietz, Floarea Serban, Abraham Bernstein, and Simon Fischer. Towards Co-operative Planning of Data Mining Workflows. In Proc of the ECML/PKDD09 Work-shop on Third Generation Data Mining: Towards Service-oriented Knowledge Discovery (SoKD-09), 2009.
J¨org-Uwe Kietz, Floarea Serban, Abraham Bernstein, and Simon Fischer. Designing kdd-workflows via htn-planning for intelligent discovery assistance. In5th PLANNING TO LEARN WORKSHOP WS28 AT ECAI 2012, page 10, 2012.
R. D. King, C. Feng, and A. Sutherland. Statlog: Comparison of classification algorithms on large real-world problems. Applied Artificial Intelligence, 9(3):289–333, 1995. doi:
10.1080/08839519508945477.
Ralph Klinkenberg, Ingo Mierswa, and Simon Fischer. Free data mining software: Rapid-miner 4.0 (formerly yale). http://www.rapid-i.com/, 2007.
Ron Kohavi, David H Wolpert, et al. Bias plus variance decomposition for zero-one loss functions. InICML, pages 275–283, 1996.
Igor Kononenko. Estimating attributes: Analysis and extensions of relief. In Francesco Bergadano and Luc De Raedt, editors, European Conference on Machine Learning, pages 171–182. Springer, 1994.
Bibliography
Christian K¨opf, Charles Taylor, and J¨org Keller. Meta-analysis: From data characteri-sation for meta-learning to meta-regression. InProceedings of the PKDD-00 Workshop on Data Mining, Decision Support,Meta-Learning and ILP, 2000.
Temur Kutsia. Context sequence matching for XML. Electronic Notes on Theoretical Computer Science, 157(2):47–65, 2006.
Rui Leite and Pavel Brazdil. Predicting relative performance of classifiers from samples.
In Proceedings of the 22Nd International Conference on Machine Learning, ICML ’05, pages 497–503, New York, NY, USA, 2005. ACM. ISBN 1-59593-180-5. doi: 10.1145/
1102351.1102414.
Rui Leite and Pavel Brazdil. Active testing strategy to predict the best classification algo-rithm via sampling and metalearning. In Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence, pages 309–314, Amsterdam, The Netherlands, The Netherlands, 2010. IOS Press. ISBN 978-1-60750-605-8.
Marius Lindauer, Holger H Hoos, Frank Hutter, and Torsten Schaub. Autofolio: an automatically configured algorithm selector. Journal of Artificial Intelligence Research, pages 745–778, 2015.
Mircea Marin and Temur Kutsia. Foundations of the rule-based system RhoLog. Journal of Applied Non-Classical Logics, 16(1–2):151–168, 2006.
Andrew McCallum and Kamal Nigam. A comparison of event models for naive bayes text classification. In IN AAAI-98 WORKSHOP ON LEARNING FOR TEXT CATEGO-RIZATION, pages 41–48. AAAI Press, 1998.
Drew McDermott, Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin Ram, Manuela Veloso, Daniel Weld, and David Wilkins. Pddl-the planning domain definition language.
1998.
Alves De Medeiros, Ana Karla, and Van Der Aalst. Semantic process mining tools: Core building blocks. 2008.
Donald Michie, David J Spiegelhalter, Charles C Taylor, and John Campbell. Machine learning, neural and statistical classification. 1994.
Steven Minton. An analytic learning system for specializing heuristics. In IJCAI, vol-ume 93, pages 922–929, 1993.
Thomas M. Mitchell. Machine Learning. McGraw-Hill, Inc., New York, NY, USA, 1 edition, 1997. ISBN 0070428077, 9780070428072.
Phong Nguyen, Alexandros Kalousis, and Melanie Hilario. A meta-mining infrastructure to support kd workflow optimization. Proc of the ECML/PKDD11 Workshop on Planning to Learn and Service-Oriented Knowledge Discovery, page 1, 2011.
Phong Nguyen, Alexandros Kalousis, and Melanie Hilario. Experimental evaluation of the e-lico meta-miner. In5th PLANNING TO LEARN WORKSHOP WS28 AT ECAI 2012, page 18, 2012.
Eugene Nudelman, Kevin Leyton-Brown, Alex Devkar, Yoav Shoham, and Holger Hoos.
Satzilla: An algorithm portfolio for sat. Solver description, SAT competition, 2004, 2004a.
Eugene Nudelman, Kevin Leyton-Brown, HolgerH. Hoos, Alex Devkar, and Yoav Shoham.
Understanding random sat: Beyond the clauses-to-variables ratio. In Mark Wallace, editor, Principles and Practice of Constraint Programming CP 2004, volume 3258 of Lecture Notes in Computer Science, pages 438–452. Springer Berlin Heidelberg, 2004b.
ISBN 978-3-540-23241-4.
Yonghong Peng, Peter A Flach, Carlos Soares, and Pavel Brazdil. Improved dataset characterisation for meta-learning. InDiscovery Science, pages 141–152. Springer, 2002.
Bernard Pfahringer, Heni Bensusan, and Christophe Giraud-Carrier. Meta-learning by landmarking various learning algorithms. Proc. 17th International Conference on Ma-chine Learning, pages 743–750, 2000.
Artem Polyvyanyy, Sergey Smirnov, and Mathias Weske. Process model abstraction:
A slider approach. In EDOC ’08: Proceedings of the 2008 12th International IEEE Enterprise Distributed Object Computing Conference, pages 325–331, Washington, DC, USA, 2008. IEEE Computer Society. ISBN 978-0-7695-3373-5. doi: http://dx.doi.org/
10.1109/EDOC.2008.17.
J. Ross Quinlan. Induction of decision trees. Machine Learning, 1(1):81–106, 1986.
J. Ross Quinlan. C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993. ISBN 1-55860-238-0.
R Core Team. R: A language and environment for statistical computing.
http://www.R-project.org/, 2013.
Bibliography
John R. Rice. The algorithm selection problem. InAdvances in Computing, page 15:65118.
1976.
Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. Methods and metrics for cold-start recommendations. InProceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, pages 253–260. ACM, 2002.
Floarea Serban, Joaquin Vanschoren, J¨org-Uwe Kietz, and Abraham Bernstein. A survey of intelligent assistants for data analysis. ACM Computing Surveys, forthcoming:1–36, 2012. ISSN 0360-0300.
William D. Smart and Leslie Pack Kaelbling. Practical reinforcement learning in contin-uous spaces. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML ’00, pages 903–910, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc. ISBN 1-55860-707-2.
Kate A Smith-Miles. Cross-disciplinary perspectives on meta-learning for algorithm selec-tion. ACM Computing Surveys (CSUR), 41(1):6, 2008.
Carlos Soares and Pavel Brazdil. Zoomed ranking: Selection of classification algorithms based on relevant performance information. InProceedings of the 4th European Confer-ence on Principles of Data Mining and Knowledge Discovery, PKDD ’00, pages 126–135, London, UK, 2000. Springer-Verlag. ISBN 3-540-41066-X.
Nathan Srebro, Jason D. M. Rennie, and Tommi S. Jaakkola. Maximum-margin matrix factorization. In Lawrence K. Saul, Yair Weiss, and L´eon Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1329–1336. MIT Press, Cambridge, MA, 2005.
Ramakrishnan Srikant and Rakesh Agrawal. Mining sequential patterns: Generalizations and performance improvements. In EDBT ’96: Proceedings of the 5th International Conference on Extending Database Technology, pages 3–17, London, UK, 1996. Springer-Verlag. ISBN 3-540-61057-X.
David H. Stern, Ralf Herbrich, and Thore Graepel. Matchbox: large scale online bayesian recommendations. In Proceedings of the 18th international conference on World wide web, WWW ’09, pages 111–120, New York, NY, USA, 2009. ACM. ISBN 978-1-60558-487-4. doi: 10.1145/1526709.1526725.
David H Stern, Horst Samulowitz, Ralf Herbrich, Thore Graepel, Luca Pulina, and Ar-mando Tacchella. Collaborative expert portfolio management. InAAAI, pages 179–184, 2010.
Xiaoyuan Su and Taghi M. Khoshgoftaar. A survey of collaborative filtering techniques.
Adv. in Artif. Intell., 2009:4:2–4:2, January 2009. ISSN 1687-7470.
Quan Sun and Bernhard Pfahringer. Pairwise meta-rules for better meta-learning-based algorithm ranking. Machine learning, 93(1):141–161, 2013.
R.S. Sutton and A.G. Barto. Reinforcement learning: An introduction. Neural Networks, IEEE Transactions on, 9(5):1054, sep 1998. ISSN 1045-9227. doi: 10.1109/TNN.1998.
712192.
Hugo Terashima-Mar´ın and Peter Ross. Evolution of constraint satisfaction strategies in examination timetabling. InProceedings of the Genetic and Evolutionary Computation Conference (GECCO99. Citeseer, 1999.
Chris Thornton, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Auto-weka:
Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 847–855. ACM, 2013.
Kim-Chuan Toh and Sangwoon Yun. An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems.Pacific Journal of Optimization, 6(615-640):15, 2010.
Wil MP Van der Aalst and CW Giinther. Finding structure in unstructured processes:
The case for process mining. In Application of Concurrency to System Design, 2007.
ACSD 2007. Seventh International Conference on, pages 3–12. IEEE, 2007.
Vladimir N. Vapnik. Statistical Learning Theory. Wiley-Interscience, September 1998.
ISBN 0471030031.
Ricardo Vilalta and Youssef Drissi. A perspective view and survey of meta-learning.
Artificial Intelligence Review, 18(2):77–95, 2002.
Ricardo Vilalta, Christophe Giraud-Carrier, Pavel Brazdil, and Carlos Soares. Using meta-learning to support data mining. International Journal of Computer Science Ap-plications, 1(1):31–45, 2004.
Bibliography
Jun Wang, Alexandros Kalousis, and Adam Woznica. Parametric local metric learning for nearest neighbor classification. InNIPS, pages 1610–1618, 2012.
Markus Weimer, Alexandros Karatzoglou, Quoc Viet Le, and Alex Smola. Maximum margin matrix factorization for collaborative ranking. Advances in neural information processing systems, 2007.
Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. The Journal of Machine Learning Research, 10:207–244, 2009.
Lin Xu, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Satzilla: Portfolio-based algorithm selection for sat. J. Artif. Int. Res., 32(1):565–606, June 2008. ISSN 1076-9757.
Lin Xu, Frank Hutter, Jonathan Shen, Holger H Hoos, and Kevin Leyton-Brown.
Satzilla2012: improved algorithm selection based on cost-sensitive classification models.
Balint et al.(Balint et al., 2012a), pages 57–58, 2012.
Qiang Yang and Xindong Wu. 10 challenging problems in data mining research.
Qiang Yang and Xindong Wu. 10 challenging problems in data mining research.