Complete LWA 2014 Workshops Proceedings

(1)

Thomas Seidl Marwan Hassani Christian Beecks (Eds.)

Proceedings of the LWA 2014

Workshops: KDML, IR and FGWM

Aachen, Germany, September 8-10, 2014

(2)

Preface

The 16^thedition of the conference LWA - “Lernen, Wissen, Adaption” (in Ger- man) which translates to “Learning, Knowledge, Adaptation” brings together researchers which deal with the discovery, the management and the retrieval of knowledge. Following the tradition of previous years, LWA comprises three workshops organized by representatives of the respective special interest groups of the Gesellschaft f¨ur Informatik (GI), which is the German computer science society. These workshops address the following topics:

– KDML - Knowledge Discovery, Data Mining, and Machine Learning – IR - Information Retrieval

– FGWM - Knowledge Management

The papers have been selected by independent program committees from the respective domains. The workshops run in parallel sessions in close vicinity to enable the exchange of ideas. They particularly meet in a joint session which includes flagship contributions of particular interest for all conference participants.

Recent trends in the corresponding research areas are highlighted by distinguished keynote speakers. In his talk on “Interdisciplinary Machine Learning”, Ulf Brefeld from TU Darmstadt points out the inherent interdisciplinarity of machine learning research as an important building block. Michael Kohlhase from Jacobs University Bremen speaks about “Mathematical Knowledge Management and Information Retrieval: Transcending the One-Brain-Barrier” and advocates for exploiting more mathematics in developing and evaluating of knowledge management concepts. Carsten Dolch from Deloitte illustrates how data analytics projects in industrial consulting contexts are conducted. In addition to these keynotes, Mirjam Minor from the Goethe University Frankfurt am Main will give a special keynote about “Case-based Reasoning in the Cloud”.

As a hand-shake of social and technical program, all authors got invited to present their work at the poster reception. A guided city tour in the historical city center of Aachen followed by the conference dinner in the university quarter complements the social program. The data management and data exploration group in the Department of Computer Science at RWTH Aachen University is proud to host the LWA 2014 conference. We hope the participants will keep the venue as an inspiring event with fruitful discussions in mind and the readers will enjoy studying the scientific contributions in this proceedings volume.

September 2014 Thomas Seidl

Marwan Hassani Christian Beecks Editors, LWA’14

(3)

Organization

The LWA conference series traditionally comprises the workshops IR, KDML and FGWM which are organized by the respective special interest groups within the Gesellschaft f¨ur Informatik (German Computer Science Society). LWA 2014 is organized by the Chair of Computer Science 9 (Data Management and Data Exploration) at the Department of Computer Science, RWTH Aachen Univer- sity, Germany.

KDML’14 Workshop Organization

Florian Lemmerich University of W¨urzburg

Eneldo Loza Menc´ıa Darmstadt University of Technology

IR’14 Workshop Organization

Sascha Kriewel University of Duisburg-Essen

Claus-Peter Klas GESIS Leibniz Institute for the Social Sciences

FGWM’14 Workshop Organization

Michael Leyer Frankfurt School of Finance & Management Joachim Baumeister denkbares GmbH

General Coordination

Institution: Chair of Computer Science 9, RWTH Aachen University

General Chair: Thomas Seidl

Local Coordinator: Christina Rensinghof Technical Program: Christian Beecks Social Program: Merih Seran Uysal

Technical Assistance: Sergej Fries, Brigitte Boden and Detlef Wetzeler

Web Setup: Ines F¨arber and Anca Zimmer Proceedings Manager: Marwan Hassani

KDML’14 Program Committee

Martin Atzmueller Christian Bauckhage

Daniel Bengs Wouter Duivesteijn

(4)

Johannes Fürnkranz Stephan Günnemann Andreas Hotho Kristian Kersting Peer Kröger

Florian Lemmerich Eneldo Loza Menc´ıa Emmanuel M¨uller Nico Piatkowski Ute Schmid

Lars Schmidt-Thieme Robin Senge

Albrecht Zimmermann

IR’14 Program Committee

David Elsweiler Reginald Ferber Norbert Fuhr Joachim Griesbaum Daniel Hienert Andreas Henrich Katja Hofmann

Frank Hopfgartner Udo Kruschwitz Johannes Leveling Thomas Mandl Philipp Mayr Henning M¨uller Peter Mutschke

Ralf Schenkel Ingo Schmitt

Hans-Christian Sperker Christian Wolff

Christa Womser-Hacker David Zellhoefer

FGWM’14 Program Committee

Klaus-Dieter Althoff Kerstin Bach Verdande Axel Benjamins Mareike Dornh¨ofer Susanne Durst Michael Fellmann

Martina Freiberg Dimitris Karagiannis Andrea Kohlhase Christoph Lange Ronald Maier Mirjam Minor

Ulrich Reimer Jochen Reutelsh¨ofer Thomas Roth-Berghofer Bodo Rieger

Peter Rossbach

(5)

Keynote Talks

(6)

Interdisciplinary Machine Learning

Ulf Brefeld

Knowledge Mining & Assessment Group TU Darmstadt, Germany

brefeld@kma.informatik.tu-darmstadt.de

Abstract. Interdisciplinary cooperations are sometimes viewed scep- tically as they often involve non-standard problem settings and inter- actions with researchers and practitioners from other domains. Thus, interdisciplinary projects may require more dedication and engagement than working on off-the-shelf problems and downloadable data sets. In this talk, I will argue that machine learning is intrinsically an interdisciplinary discipline. Reaching out to other domains constitutes an important building block to advance the field of machine learning as it is the key to finding interesting and novel challenges and problem settings. Es- tablishing an abstract view on such a novel problem setting often allows to identify surprisingly unrelated tasks that fall into the same equivalence class of problems and can thus be addressed with the same methods. I will present examples from ongoing research projects.

Copyright c 2014by the paper’s authors. Copying permitted only for private and academic purposes. In: T. Seidl, M. Hassani, C. Beecks (Eds.): Proceedings of the LWA 2014 Workshops: KDML, IR, FGWM, Aachen, Germany, 8-10 September 2014, published at http://ceur-ws.org

(7)

Analytics Applied: Current Market Trends and Case Studies

Carsten Dolch

Deloitte GmbH cdolch@deloitte.de

Abstract. Analytics is becoming more and more part of the decision making process for management and operational work. Within this session the Deloitte Analytics Institute wants to provide you with an insight into an user experience based approach, how to engage customers with analytics applications and how analytics becomes the key driver for IT landscape transformation.

(8)

Mathematical Knowledge Management and Information Retrieval:

Transcending the One-Brain-Barrier

Michael Kohlhase

Jacobs University Bremen, Germany m.kohlhase@jacobs-university.de

Abstract. The talk presents the discipline of Mathematical Knowl- edge Management (MKM), which studies the possibility of computer- supporting and even automating the representation, cataloguing, retrieval, refactoring, plausibilization, change propagation and in some cases even application of knowledge. Mathematics is a suitable test domain, as mathematical language is intrinsically rich in structure, rig- orous but diverse in presentation, and non-trivial but sufficiently well- understood in content.

We focus on theory graph technology here, which supports modular and thus space/computation/cognitively-efficient representations of mathematical knowledge and allows MKM systems to achieve a limited mathematical literacy that is necessary to complement the abilities of human mathematicians and thus to enhance their productivity.

For more details see http://www.ems-ph.org/journals/newsletter/pdf/2014- 06-92.pdf.

(9)

The Joint Session

(10)

Landmark Recognition: State-of-the-Art Methods in a Large-Scale Scenario

^?

Magdalena Rischka and Stefan Conrad

Institute of Computer Science Heinrich-Heine-University Duesseldorf

D-40225 Duesseldorf, Germany rischka@cs.uni-duesseldorf.de

conrad@cs.uni-duesseldorf.de

Abstract. The recognition of landmarks in images can help to manage large image collections and thus is desirable for many image retrieval applications. A practical system has to be scalable with an increasing number of landmarks. For the domain of landmark recognition we investigate state-of-the-art CBIR methods on an image dataset of 900 landmarks.

Our experiments show that the kNN classifier outperforms the SVM in a large-scale scenario. The examined visual phrase concept has shown not to be as effective as the classical Bag-of-Words approach although the most landmarks are objects with a relatively fixed composition of their (nearby) parts.

Keywords: Image Retrieval, Large-Scale, Landmark Recognition, Bag- of-Words, Bag-of-Phrases

1 Introduction

The ongoing development of personal electronic devices like digital cameras, mobile phones or tablets with integrated camera and high-capacity memory cards, as well as their decreasing prices enable taking photos everywhere and at any time. Collecting and storing photos as well as sharing photos with others on online social network platforms leads to huge photo collections in personal house- holds and to a much greater extent on the world wide web. To manage and reuse these images in an useful way (e.g. for search purposes) it is necessary to capture the images’ content, i.e. to annotate the images with meaningful textual keys.

A large amount of the collections’ images are photos shot in the photographer’s vacations and trips showing (prominent) places and landmarks the photographer visited. The detection and recognition of landmarks in images offers several advantages regarding applications: the above-mentioned annotation constitutes

?Copyright c 2014by the paper’s authors. Copying permitted only for private and academic purposes. In: T. Seidl, M. Hassani, C. Beecks (Eds.): Proceedings of the LWA 2014 Workshops: KDML, IR, FGWM, Aachen, Germany, 8-10 September 2014, published at http://ceur-ws.org

(11)

a foundation for a search or can be used as a suggestion for a photo description to the user. Another usage is the identification of locations the photogra- phers visited, for example to summarize personal image collections by offering an overview of places. The application of mobile landmark recognition enables tourists to look up sights in real-time to obtain informations on them. Capturing images’ content by manual annotation of images with landmarks however is very time-consuming, in the scale of these collections even inconvertible, therefore an automatic solution is needed. Several systems for automatic landmark recognition have been proposed [2–6] differing in the focus of application scenario, the initial situation referring metadata, problem definition and implemented techniques. For example the authors of [2] create a database from geo-tagged Flickr photos and Wikipedia. Object-level recognition is performed with the aid of an index and candidate images are ranked using a TF-IDF scheme. [3] also creates a dataset from Flickr images and then derives scene maps of landmarks which are retrieved with an inverted index. [4] creates the database by crawling travel guide websites and then builds a matching graph out of the feature matches of the images. For retrieval a kd-tree is used. We concentrate on images without any metadata, thus on content-based methods only. Several state-of-the-art methods in CBIR have been examined and applied successfully on small or average size datasets. Our focus is on the large-scale aspect of a landmark recognition system and the usability in real world scenarios, thus our contribution is the comparison of these methods with reference to scalability.

The remainder of this paper is organized as follows: in the next section we out- line and formalize the problem of landmark recognition by defining the landmark term, describing the characteristics of landmark images, specifying the landmark recognition task and presenting the components of the landmark recognition system evaluated in section 3. In section 4 we summarize our results and discuss future work.

2 Landmark Recognition Problem and System

Alandmark is a physical object, created by man or by nature, with a high recognition value. Usually a landmark is of remarkable size and is located on a fixed position of the earth. Examples of landmarks are buildings, monuments, stat- ues, parks, mountains and other structures and places. Due to their recognition value, landmarks often serve as geographical points for navigation and locali- sation. The largest amount of photos of landmarks contain only one landmark, which in the most cases takes in 80% of the photo area, in very few cases it takes only a small part of the photo (when it is taken from apart). A marginal part of photos show two or more landmarks. A landmark recognition system has to conduct the following task automatically:

Definition 1 (Landmark Recognition Task). Given a set of L landmarks L={l1, ..., lL} and an imagei whose semantic content is unknown. The task is

(12)

to assign a set of landmarks to the image:

i→

(∅ if imageidoes not contain any landmark

{lj₁, ..., lj_n} if imageicontains landmarks{lj₁, ..., lj_n} (1)

This definition implies a multi-label classification problem with a decision refusal.

We simplify the multi-label classification problem defined in (1) by building our system on a single-label classification approach, thus we accept a possible mis- classification of images containing more than one landmark. We focus on the classification step, the decision refusal which is usually performed with a post- processing verification algorithm (like RANSAC) is beyond this work. The main components of our landmark recognition system, which are the image representation and the classifier are discussed in the following paragraphs.

Image Representation To describe images we extract the popular SIFT [7] features. The SIFT algorithm extracts local features by detecting stable points and then describing the (small) surrounding area around each point by an histogram of gradients. An image i is represented by a set of local SIFT points:

SIF T(i) ={p1, ..., pP |p= (x, y, s, d)}withx, yare the coordinates of the point pin the image,sis the scale anddthe 128-dimensional descriptor. We analyse two types of image representation based on the local SIFT features: Bag-of- Words (BoW) and the Bag-of-Phrases (BoP) model based on the visual phrase concept. Although visual phrases have been used in general object recognition applications, they raised less attention in the domain of landmark recognition.

We like to analyse if visual phrases improve the BoW classification results.

TheBag-of-Words model is a classical approach to create a compact image representation based on local features. The idea is to aggregate local features to one global descriptor and thus to avoid the expensive comparison of images by matching local descriptors against each other. The BoW descriptor bases on a dictionary of visual words which is obtained by partitioning the descriptor- space. Then each partition is represented by an instance of this partition, usually the center of the partition, which is called thevisual word. Several methods for partitioning the descriptor-space have been proposed, a simple and most used one is the k-Means clustering algorithm which requires the input parameter k (which onwards is denoted asD to differentiate between the kNN parameterk) for the number of clusters (visual words) to obtain.

Definition 2 (Bag-of-Words Model).

Given a dictionaryD={(w1, c1), ...,(wD, cD)}ofD visual words (wj, cj) (wj is label,cj the center of the partition) and an image in SIFT representation. Each SIFT point pof the image is assigned to its visual wordwp by:

wp:=wj = argmin

(wj,cj)∈D

(EuclideanDistance(d, cj)) (2)

(13)

The Bag-of-Words image representation is given by:

BoW(i) ={f1, ..., fD}with fj = 1 P

P

X

p=1

1, wp=j

0, else (3)

Visual phrases catch spatial relations in local neighborhood by considering pairs of nearby local features or visual words to support more semantic, anal- ogously to phrases in text retrieval. We follow [8] and define the visual phrase and the Bag-of-Phrases model as follows:

Definition 3 (Bag-of-Phrases Model).

Given the visual dictionaryD={(w1, c1), ...,(wD, cD)}. A visual phrase phj,kis a pair of visual words:phj,k =(wj, wk)withj≤k. An image in SIFT representation with its visual wordsSIF T⁰(i) ={p1, ..., pP | p= (x, y, s, d, w)} contains the phrasephj,kif there exist two SIFT pointspmandpnwith their visual words wj and wk and it holds

EuclideanDistance((xm, ym),(xn, yn))≤max(λ·sm, λ·sn) (4) for a fixed scale factor λ. The Bag-of-Phrases image representation is given by:

BoP(i) ={f1, ..., fD2} with D2= D·(D+ 1)

2 (5)

with fj is the relative frequency of the visual phrasephj in imagei.

Classifier For the choice on the classifier, we evaluate three well-known classifiers, the Support-Vector-Machine (SVM), the k-Nearest Neighbor (kNN) and the Nearest Center classifier (NC). The SVM is a popular classifier as it provides better classification results than other standard classifiers in the most (computer vision) classification tasks. However the drawback of the SVM classifier is the long classifier learning time, especially with an increasing training data size. In addition to that [1] has shown that the superiority of the SVM over the kNN classifier (with regard to classification quality) can swap with an increasing number of classes. As our focus is on the large-scale landmark recognition with reference to an increasing number of landmarks, we investigate the landmark recognition on both classifiers. The kNN classifier has no training part, instead the classification time is linear in the number of training examples, which in a scenario of over 100.000 images and a system implementation without the use of appropriate and efficient access structures can put a strain on the user. However the classifier NC can be seen as a lightweight classifier, as both the learning and the classification time is linear in the number of classes. For the SVM we use the RBF kernel and the one-vs-one mode, for the kNN we set k= 5 (as a result of preliminary experiments on different k), for the kNN and the NC classifier we choose the histogram intersection as the similarity function.

(14)

3 Evaluation

Evaluation Dataset For the evaluation we use a self-provided dataset of landmarks. We gathered landmark terms from several websites which lists landmarks from all over the world, including the website of [4]¹. Our dataset consists of 900 landmarks from 449 cities and 228 countries. To get images for the training and test sets, we queried the google image search engine with each landmark term (specified by its region - city or country) and then downloaded the results from the original source. For scalability analysis we derived training sets of four different sizes: 45, 300, 600 and 900 landmarks. For each size three training sets (A, B, C) have been created. To create a challenging test set we have chosen images manually from the results of the google image search: The images show the landmarks in their canonical views, under different perspective changes, dis- tortions and lighting conditions (also at night) as well as indoor shootings and parts of the landmarks. The test set consists of 900 landmark images (45 well- and lesser-known landmarks from Europe with 20 test images per landmark).

The test images have been proofed to be disjoint from the training set.

Evaluation Measures The outcome of a single-label classifier on a test image is the predicted landmark. To retain a fine-grained evaluation of a test image, we are also interested in a ranking of landmarks, as the ranking reveals how far away is the groundtruth landmark from the top ranking position. The SVM delivers us a ranking based on the probability values of the one-vs-one voting, the NC classifier based on the histogram intersection similarity. The kNN returns only the predicted landmark. To evaluate the results of the classifiers we use two (instance-based) evaluation measures: the (instance-based) recall on the predicted landmark and the MAP measure (which is equal to MRR measure in this case) for the landmarks ranking. We finally report the average value over all test images.

Experiments The first experiment evaluates the Bag-of-Words model in combination with the three classifiers and the four training sets of size 45, 300, 600 and 900. The Bag-of-Words model has one parameter which is the visual dictionary size. We examine the following six different visual dictionary sizes: 500, 1000, 2000, 4000, 6000 and 8000. Figure 1 shows the results of this experiment. The recall and MAP values reported are averages over the training sets A,B,C of the corresponding training set size. Table 1 shows the average recognition time for a test image on the Bag-of-Words model with a visual dictionary size of 8000 depending on the classifier and the training set size. The average recognition time does not include the processing time for image representation computation. Ex- periments are performed on an usual Intel i7 960 3.2 GHz (64-bit) architecture with 16 GB memory size. The system (kNN) is implemented without the use of any efficient access structure. Considering the recall values of all classifiers for all

1 http://mingzhao.name/landmark/landmark_html/demo_files/1000_landmarks.

html

(15)

0 0.2 0.4 0.6 0.8 1 1.2

·10⁴ 0.4

0.5 0.6 0.7

Visual dictionary size (D)

MAP&Recall

Training set with 45 landmarks (TR 45)

SVM MAP NC MAP SVM Rec NC Rec 5NN Rec

0 0.2 0.4 0.6 0.8 1 1.2

·10⁴ 0.3

0.4 0.5

MAP&Recall

0 0.2 0.4 0.6 0.8 1 1.2

·10⁴ 0.3

0.4 0.5

MAP&Recall

0 0.2 0.4 0.6 0.8 1 1.2

·10⁴ 0.25

0.3 0.35 0.4 0.45 0.5

MAP&Recall

Fig. 1. Classification results of the BoW model depending on the parameters visual dictionary sizeD (x-axis), classifier with evaluation measure (plots) and training set size (subimages)

training set sizes, we can see that the best values are achieved by the SVM and the 5NN with a visual dictionary size of 6000 and 8000. On the training set TR 45 the SVM gets the best recall value with 0.65 (D= 6000). From the training set TR 300 on the 5NN outperforms slightly the SVM resulting in a difference of 4% on TR 900 and D = 8000. Furthermore the 5NN shows the tendency to achieve higher results with a growing visual dictionary. These results confirm the observation of the superiority of kNN over the SVM in large-scale problems stated in [1]. The NC classifier achieves best results onD= 2000 for all training set sizes, however its best values are on average 13% lower than the best system (SVM or kNN). The MAP values of the SVM and the NC reveal that there is potential to improve these classifiers when involving the next to top ranking positions in the classification decision. In general the recognition accuracy de- creases with an increasing number of landmarks which is not surprising. A recall value of 0.66 on the TR 45 (SVM,D= 6000) can be somewhat satisfying, however the best result of 0.43 on TR 900 (5NN, D= 8000) is less delightful. The

(16)

second experiment concentrates on the Bag-of-Phrases model. Again we report experiments in combination with the three classifiers and the four training set sizes. The BoP model requires two parameters to be set: the visual dictionary sizeD and the scale factorλ. As the dimension of the image’s descriptor in this representation becomes very large on already small visual dictionary sizes, we examined the two sizes 500 and 1000 resulting in the descriptor dimension of 5050 and 125250, respectively. For the scale factor we choose the values 1, 2, 4 and 6. The BoP results (Figure 2) for all classifiers and all training set sizes are on average 10% lower than the BoW results. The larger visual dictionary (500, λ) achieves better results than the smaller one (100, λ), especially on the SVM, whereas the scale factor influences the results slightly. In the most cases the scale factor of 2 gets best results. Due to the high-dimensional descriptor (D≥500) the recognition time is many times higher than of the BoW model.

(100,1) (100,2) (100,4) (100,6) (500,1) (500,2) (500,4) (500,6)

0.3 0.4 0.5 0.6

(Visual dictionary size (D), Scale factor (λ))

MAP&Recall

(100,1) (100,2) (100,4) (100,6) (500,1) (500,2) (500,4) (500,6)

0.2 0.3 0.4

MAP&Recall

(100,1) (100,2) (100,4) (100,6) (500,1) (500,2) (500,4) (500,6)

0.2 0.3 0.4

MAP&Recall

(100,1) (100,2) (100,4) (100,6) (500,1) (500,2) (500,4) (500,6)

0.2 0.3 0.4

MAP&Recall

Fig. 2. Classification results of the BoP model depending on the parameters visual dictionary sizeD and scale factorλ(x-axis), classifier with evaluation measure (plots) and training set size (subimages)

(17)

TR 45 TR 300 TR 600 TR 900 SVM 0.0577 0.1016 0.2710 0.5081

NC 0.0039 0.0239 0.0424 0.0612 5NN 0.1129 0.4637 0.8444 1.2232

Table 1.Average recognition time (in seconds) for the BoW model with a dictionary of size 8000 dependent on the three classifiers and the four training set sizes.

4 Summary and Future Work

To build a landmark recognition system with large number of landmarks (TR 900) supported, the Bag-of-Words model together with the kNN classifier offers a higher recognition accuracy than the SVM but on the cost of a relatively high recognition time of about 1.2 seconds per image. A solution to use kNN and to reduce the recognition time is to integrate an appropriate and efficient access structure into the system and to try to reduce the number of training images per landmark (by a compressed representation) without loosing too much relevant informations. The BoP model alone does not convince, therefore the question arises, if this model returns additional knowledge to the BoW model. In fact, some few landmarks (33% of the tested landmarks) benefit from the BoP model, others not. A detailed analysis of this and a suitable combination of both models are matters for further research beyond this work. Furthermore it would be interesting to compare our state-of-the-art approach with a system which bases on an inverted file index working directly on local features.

References

1. Deng, J., Berg, A. C., Li, K., Fei-Fei, L.: What does classifying more than 10,000 image categories tell us? In: Proceedings of the 11th European conference on Com- puter vision (ECCV’10), 2010

2. Gammeter, S., Bossard, L., Quack, T., Gool, L.V.: I know what you did last summer:

object-level auto-annotation of holiday snaps. In: IEEE international conference on computer vision (ICCV), 2009

3. Avrithis, Y., Kalantidis, Y., Tolias, G., Spyrou, E.: Retrieving landmark and non- landmark images from community photo collections. In: Proceedings of the international conference on Multimedia (MM ’10). ACM, New York, 2010

4. Zheng, Y., Zhao, M., Song, Y., Adam, H., Buddemeier, U., Bissacco, A., Brucher, F., Tat-Seng Chua, Neven, H.: Tour the world: Building a web-scale landmark recognition engine. In: Computer Vision and Pattern Recognition (CVPR), 2009 5. Philbin, J., Zisserman, A.: Object Mining Using a Matching Graph on Very Large

Image Collections. In: ICVGIP, 2008

6. Crandall, D. J., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In Proceedings of the 18th international conference on World wide web (WWW ’09). ACM, New York, 2009

7. Lowe, D. G.: Distinctive Image Features from Scale-Invariant Keypoints. In: Inter- national Journal of Computer Vision, 2004

8. Zheng, Q. F., Gao, W.: Constructing visual phrases for effective and efficient object- based image retrieval. In ACM Trans. Multimedia Comput. Commun. Appl. 5, October 2008

(18)

Evaluating Assumptions about Social Tagging

^?

A Study of User Behavior in BibSonomy

Stephan Doerfel¹, Daniel Zoller², Philipp Singer³, Thomas Niebler², Andreas Hotho², and Markus Strohmaier^3,4

1 ITeG & Knowledge and Data Engineering Group, University of Kassel (Germany) doerfel@cs.uni-kassel.de

2 Data Mining and Information Retrieval Group, University of W¨urzburg (Germany) {zoller, niebler, hotho}@informatik.uni-wuerzburg.de

3 GESIS (Germany){philipp.singer, markus.strohmaier}@gesis.org

4 University of Koblenz (Germany)

Abstract. Social tagging systems have established themselves as an important part in today’s web and have attracted the interest of our research community in a variety of investigations. Henceforth, several assumptions about social tagging systems have emerged on which our community also builds their work. Yet, testing such assumptions has been difficult due to the absence of suitable usage data in the past. In this work, we investigate and evaluate four assumptions about tagging systems by examining live server log data gathered from the public social tagging system BibSonomy. Our empirical results indicate that while some of these assumptions hold to a certain extent, other assumptions need to be reflected in a very critical light.

1 Introduction

Social tagging systems such as BibSonomy, Delicious or Flickr have attracted the interest of our research community for almost a decade. While previous research has significantly expanded our expertise to describe [4] and model [2], social tagging systems, the community has also built their work on certain assumptions about usage patterns in these systems, which have emerged over time. For such assumptions, arguments and evidence have been discussed, though it is not clear to which degree they remain valid in actual tagging systems. Only a few studies have analyzed user behavior in social tagging systems to better understand such assumptions, either by (i) conducting user surveys (e.g., [5]) or by (ii) tapping into the rich corpus of tagging data (i.e., the posts) that is available on the web (e.g., [2]). However, such studies lack of detailed data how users actually

?Extended Abstract for Work-in-Progress.

(19)

request information. In this paper we overcome these drawbacks by presenting and thoroughly investigating a detailed usage log of the real-world, open social tagging system BibSonomy.⁵

2 Assumptions and Results

The Social Assumption. Assuming that social tagging systems are social, we measure to which degree users collaboratively share resources and we discuss evidence for the interest of users in the content of others. Details of this analysis can be found in [3].

The Retrieval Assumption.For the retrieval assumption we investigate whether users store resources in BibSonomy for later retrieval. We discover that while users post a large number of resources and tags to BibSonomy, they only retrieve a rather small fraction of them later.

The Equality Assumption.The equality assumption claims that the three sets of entities in a tagging system – users, tags, and resources – are equally important for navigation and retrieval. However, we find a stronginequality in the use of these entity sets: in BibSonomy, requests to user pages dominate the number of requests to tags and to resources.

The Popularity Assumption. Finally, we test whether the popularity of users, tags, and resources in posts is matched by their popularity in retrieval. We ob- serve common usage patterns in posting and requesting behavior on an aggregate level. The patterns are less pronounced on an individual level.

Acknowledgments.This work is in part funded by the DFG through the PoSTs II project.

References

1. Benz, D., Hotho, A., J¨aschke, R., Krause, B., Mitzlaff, F., Schmitz, C., Stumme, G.:

The social bookmark and publication management system BibSonomy. The VLDB Journal 19(6), 849–875 (Dec 2010)

2. Cattuto, C., Schmitz, C., Baldassarri, A., Servedio, V.D.P., Loreto, V., Hotho, A., Grahl, M., Stumme, G.: Network properties of folksonomies. AI Communications Journal, Special Issue on “Network Analysis in Natural Sciences and Engineering”

20(4), 245–262 (2007)

3. Doerfel, S., Zoller, D., Singer, P., Niebler, T., Strohmaier, M., Hotho, A.: How social is social tagging? In: Proceedings of the 23rd International World Wide Web Conference. WWW 2014, ACM, New York, NY, USA (2014)

4. Golder, S.A., Huberman, B.A.: Usage patterns of collaborative tagging systems.

Journal of information science 32(2), 198–208 (April 2006)

5. Heckner, M., Heilemann, M., Wolff, C.: Personal information management vs. re- source sharing: Towards a model of information behaviour in social tagging systems.

In: Proceedings of the 3rd International Conference on Weblogs and Social Media.

ICWSM ’09, San Jose, CA, USA (May 2009)

5 http://www.bibsonomy.org/, see [1] for a detailed description and various analyses.

(20)

Maintenance of distributed case-based reasoning systems in a multi-agent system

^∗

Pascal Reuss and Klaus-Dieter Althoff pascal.reuss@dfki.de

klaus-dieter.althoff@dfki.de

Intelligent Information Systems Lab, University of Hildesheim

Competence Center for Case Based Reasoning, German Center for Artificial Intelligence, Kaiserslautern

Abstract. In many knowledge-based systems the used knowledge is distributed among several knowledge sources. Knowledge maintenance of such systems has several challenges to be met. This paper gives a short overview of a maintenance approach using so-called Case Factories to maintain knowledge sources and considering the dependencies between these sources. Furthermore we present a concept how our maintenance approach can be applied to a multi-agent system with several case-based reasoning systems.

1 Introduction

When maintaining the knowledge among distributed case-based reasoning (CBR) systems the dependencies between the knowledge sources are of crucial importance. For maintaining a single CBR system there are also several approaches that deal with maintaining the case base, the similarity, or the adaptation knowledge. In general all the knowledge sources belonging to a knowledge-based system have to be considered, too.

This paper describes a multi-agent system, based on the SEASALT architecture, that is extended with several agents to apply the Case Factory approach. We describe the tasks of every required agent and the communication between them. In addition we present the required agents for the explanation capabilities. Section 2 describes related work to knowledge maintenance. In Section 3 the agents required for applying the Case Factory approach to a multi-agent system are described. In Section 4 a short conclusion is given.

1.1 SEASALT architecture

The SEASALT (Shared Experience using an Agent-based System Architecture Lay- out) architecture is a domain-independent architecture for extracting, analyzing, sharing, and providing experiences [[5]]. The architecture is based on the Collaborative Multi-Expert-System approach [1][2] and combines several software engineering and

∗Copyrightc 2014by the paper’s authors. Copying permitted only for private and academic purposes.In: T. Seidl, M. Hassani, C. Beecks (Eds.): Proceedings of the LWA 2014 Work- shops: KDML, IR, FGWM, Aachen, Germany, 8-10 September 2014, published at http://ceur- ws.org

(21)

artificial intelligence technologies to identify relevant information, process the experience and provide them via an interface. The knowledge modularization allows the compilation of comprehensive solutions and offers the ability of reusing partial case information in form of snippets. Figure 1 gives an overview over the SEASALT architecture.

Fig. 1.Overview of the SEASALT architecture

The SEASALT architecture consists of five components: the knowledge sources, the knowledge formalization, the knowledge provision, the knowledge representation, and the individualized knowledge. The knowledge sources component is responsible for extracting knowledge from external knowledge sources like databases or web pages and especially Web 2.0 platforms. These knowledge sources are analyzed by so-called Collector Agents, which are assigned to specific Topic Agents. The Collector Agents collect all contributions that are relevant for the respective Topic Agent’s topic [5].

The knowledge formalization component is responsible for formalizing the extracted knowledge from the Collector Agents into a modular, structural representation. This formalization is done by a knowledge engineer with the help of a so-called Apprentice Agent. This agent is trained by the knowledge engineer and can reduce the workload for the knowledge engineer [5]. The knowledge provision component contains the so called Knowledge Line. The basic idea is a modularization of knowledge analogous to the modularization of software in product lines. The modularization is done among the

(22)

individual topics that are represented within the knowledge domain. In this component a Coordination Agent is responsible for dividing a given query into several sub queries and pass them to the according Topic Agent. The agent combines the individual solutions to an overall solution, which is presented to the user. The Topic Agents can be any kind of information system or service. If a Topic Agent has a CBR system as knowledge source, the SEASALT architecture provides a Case Factory for the individual case maintenance. [5][4] The knowledge representation component contains the underlying knowledge models of the different agents and knowledge sources. The synchronization and matching of the individualized knowledge models improves the knowledge maintenance and the interoperability between the components. The individualized knowledge component contains the web-based user interfaces to enter a query and present the solution to the user.[5]

2 Related work

This section contains related work from other authors with focus on the maintenance of the knowledge containers of CBR systems and the maintenance of distributed knowledge in CBR systems. There exist several approaches to maintain the knowledge containers of a CBR system. For the maintenance of a case base various strategies were developed for example by [9], [10], [12], [13], [18], [17], [19] and [22]. [20] and [11]

describe approaches to maintain the similarity measures within a CBR system. All this approaches are set up to maintain knowledge containers of a single CBR system. They neither consider the use of multiple CBR systems nor the dependencies between the knowledge containers of different CBR systems. All mentioned maintenance strategies could be applied within a Case Factory, but have to be embedded in an overall maintenance strategy managed by the Case Factory Organization.

Geissbuhler and Miller describe in their paper an approach for maintaining distributed knowledge bases in a clinical decision support system called WizOrder. Con- trary to our approach, the maintenance in the WizOrder system is not done by one knowledge engineer, but by many different users of the system, like house staff, physi- cians, and nurses. The knowledge sources in the decision support system are heteroge- neous and not homogenous as intended in our approach. Therefore many different tools for maintenance are used, each one with a specific interface for the respective user. The local knowledge bases are maintained by the users and an expert integrates the maintenance actions into the central knowledge base called knowledge library. From this knowledge library the cumulative changes are provided to the local knowledge bases.

While this is done by human users and experts within the WizOrder system, in our approach we use software agents to suggest maintenance actions and central planning and supervising agents to generate a maintenance plan. This plan has still to be checked by a human knowledge engineer. [8]

Ferrario and Smyth described an approach for collaborative maintenance of a case base.

The feedback of several users is evaluated and an appropriate maintenance action derived. When we compare our approach to theirs, our agents could be seen as users, that gives feedback and suggest maintenance actions. A Case Factory, maintaining one

(23)

CBR system could be compared to the collaborative maintenance. One difference between the approaches is that our approach is extended with maintenance capabilities for several CBR systems.[6][7]

3 Maintenance of distributed case-based reasoning systems

This section gives a short overview over the idea of the Case Factory (CF) and the Case Factory Organization (CF). Then the software agents for the realization of the CF and CFO are described. At last the software agents for the explanation capabilities are described.

3.1 Agents of the Case Factory

Three types of software maintenance can be distinguished: corrective, adaptive and perfective maintenance. Corrective maintenance deals with processing failures, performance failures and implementation failures. Processing failures are situations like ab- normal termination of an application. Performance failures deals with situations where the application violates defined performance constraints like to long response time. Im- plementation failure can lead to processing and performance failures, but may also be have no effect on the system. Adaptive maintenance deals with changes in the environment of an application and aims at avoiding failures caused by the change of an application environment. Perfective maintenance cover all actions that are performed to eliminate processing inefficiencies, enhance performance or improve the maintainabil- ity. This type of maintenance aims at keeping an application running at less expense or running to better serve the users needs [21]. [16] defines the knowledge maintenance of CBR systems as the combination of technical and associated administrative actions that are required to preserve the knowledge of a CBR system, or to restore the knowledge of the system to provide the intended functionality. This maintenance actions include also actions to adapt an CBR system to environment changes and enhance the performance.

The SEASALT architecture supports the maintenance of a CBR system with the help of a Case Factory. The original idea is from Althoff, Hanft and Schaaf [3] and the concept was extended by Reuss and Althoff [14]. The CF approach and the SEASALT architecture support the maintenance of distributed knowledge sources in multi-agent systems and the CF is intended to perform corrective maintenance as well as adaptive and perfective maintenance. Feedback from users about false solutions may lead to corrective maintenance actions, while the evaluation of the knowledge in a CBR system may lead to adaptive or perfective maintenance. The extended CF supports the maintenance of a single CBR system. It contains several software agents responsible for the evaluation and the maintenance of the case-based reasoning knowledge containers. This knowledge containers were introduced by [15].

The idea behind the Case Factory approach is maintenance of knowledge sources should consider the dependencies between the knowledge containers in a CBR system and the dependencies between the knowledge containers of different CBR systems.

(24)

There are dependencies between the vocabulary and the case base in a single CBR system or between case bases in different CBR systems. Changing the knowledge in one knowledge container may cause inconsistencies. Therefore additional maintenance actions may be necessary to restore the consistency of the knowledge.

To apply the CF approach to the a multi-agent system with CBR system nine agents are required: four monitoring and evaluation agents, each one responsible for one knowledge container (case base, vocabulary, similarity, and adaptation), and four maintenance agents, each one responsible for processing individual maintenance actions for the respective knowledge container. We propose an individual monitoring and evaluation agent for each knowledge container to process the monitoring and evaluation tasks in parallel. In addition it will be possible to activate and deactivate the monitoring and evaluation of a knowledge container during runtime by starting or shutting down the associated agent without affecting the monitoring and evaluation of the other knowledge containers. The last new agent is a supervising agent that coordinates the monitoring and evaluation of the knowledge containers and the processing of maintenance actions.

In addition the agent communicates with the high-level Case Factory Organization. Fig- ure 2 shows these agents in a multi-agent system.

Fig. 2.Multi-agent system with Case Factory agents

In the following the tasks, permissions and responsibilities of the agent roles are described in GAIA notation [23]. Protocols and activities define the communication with other roles and the tasks a role can perform. The permissions are used to describe the knowledge a role has access to and the knowledge a role can change or generate. At last the responsibilities are used to describe the life cycle of a role. It is defined in which

(25)

order the protocols and activities are performed and if there are repetitions of protocols or activities. A (*) means, that a protocol or activity is performed 0 to n times, a (+) that a protocol or activity is performed 1 to n times. The exponent at the end of the liveness responsibilities describes the times the the whole process is performed.ω means it is repeated endlessly.

Fig. 3.Role schema Evaluator in Gaia notation

Fig. 4.Role schema Maintainer in GAIA notation

Both generic roles are specialized for the specific agents and generic terms are substituted with the concrete knowledge container. Both roles have access to a local maintenance map, which contains information about available and preferred evaluation strategies and maintenance actions as well as evaluation metrics to compare the results to the maintenance goals. This way several evaluation strategies can be defined and applied to an agent.

(26)

Fig. 5.Role schema Supervisor in GAIA notation

3.2 Agents of the Case Factory Organization

While a Case Factory is able to maintain a single CBR system a high-level Case Fac- tory Organization is required to coordinate the actions of all Case Factories and take the dependencies between the single CBR systems into account. This CFO consists of several additional software agents to supervise the communication between the Case Factories and the adherence of high level maintenance goals. Additionally, agents collect the maintenance suggestions from the Case Factories and derive a maintenance plan from all single maintenance suggestions. The agents are also responsible for checking constraints or solving conflicts between individual maintenance suggestions. In addition, a maintenance suggestion may trigger follow-up maintenance actions based on the dependencies between the CBR systems. The concept of the CFO allows to realize as many CFs and layers of CFOs as required. A multi-agent system can be divided into layers and each layer can have its own Case Factory Organization. This way a hierarchy of CFOs can be established that is scalable and supports multi-agent systems with many agents and layers. [14]

Each required Case Factory Organization consists of four software agents. A Col- lector Agent, a Maintenance Planning Agent, a Goal Monitoring Agent and a Team Supervisor Agent. For the assumed MAS only one Case Factory Organization level is required. Figure 6 shows the multi-agent system with the the additional agents for the Case Factory Organization.

Inside the CF agents evaluating the knowledge containers and derive maintenance suggestions from the result with the help of the local maintenance map (1). The results and the derived maintenance actions are send to the supervisor (2). The supervisor passed the maintenance actions to the collector (3). This collector gets the derived maintenance actions from all Case Factories and sends them to the goal monitoring agent.

The goal monitoring agent is responsible for checking the maintenance actions against constraints from the team maintenance map. If no constraints are violated the maintenance actions are sent to the maintenance planner (5). This agent generates a plan from the maintenance actions. During the planning process it is possible to generate new maintenance actions based on the dependencies between different CBR systems. The

(27)

Fig. 6.docQuery multi-agent system with Case Factory Organization agents

maintenance plan is sent to the team supervisor (6). This agent checks the plan against constraint violation like the goal monitor does for individual actions. The checked plan is sent to the maintenance communicator and shown to a knowledge engineer (7). The knowledge engineer checks the plan and confirms the maintenance actions to be performed. He can also eliminate actions from the plan. The confirmed plan is sent back to the team supervisor in the CFO, the supervisor in the CF and the single maintenance actions to the maintaining agents.

Our concept for the Case Factory Organization includes explanation capabilities of the maintenance actions and the maintenance plan. The idea is to provide a set of explanations to support the knowledge engineer’s understanding of the suggested maintenance plan and single actions. The idea is to use explanation templates that are filled with logging information. These templates consists of several text modules in human natural language. This way we try to use the systems logging information to generate human readable explanations.

To achieve this goal, the multi-agent system has to log all communication and actions of all agents, as well as evaluation results, feedback, constraint checks, and denied maintenance actions. From this logged information explanations should be extracted and combined for each maintenance action and the maintenance plan itself. Three additional roles are required to provide simple explanations: Logger, Logging Supervisor, Explainer. For each role at least one agent in the docQuery multi-agent system will be implemented. For several roles like the Logger or the Evaluator more than one instance is required. Some of the described roles and the respective agents can be combined in agent teams. For example, for a Case Factory a team of four Evaluators, four Main-

(28)

tainers and one Supervisor is requiresd. Adding a new Case Factory will require the creation of nine software agents. Other roles like the Logger or the Explainer and its respective agents can be added as single agents. This way the multi-agent system has a high scalability and agents can be created and removed based on the tasks the single agents or the agent team are designed for.

Figure 7 shows the multi-agent system with all agents for the CF, CFO, and explanations. In the figure only the communication with the new agents is illustrated.

Fig. 7.MAS with CF, CFO and explanation agents

Several agents are responsible for logging the communication and performed tasks of the agent in the multi-agent system (0a) and the logged information are send to the logging supervisor (0b). These information are used to generate explanations for suggested maintenance actions. Steps 1 till 6 are the same as described above. In addition, the checked plan is sent to the explanation agent (7). This agent uses the logged information to enrich the maintenance plan with explanations. The enriched plan is sent to the maintenance communicator and shown to a knowledge engineer (8). The knowledge engineer checks the plan and confirms the maintenance actions to be performed.

He can also eliminate actions from the plan. The confirmed plan is sent back to the team supervisor in the CFO, the supervisor in the CF and the single maintenance actions to the maintaining agents (9).

(29)

4 Summary and Outlook

In this paper we presented the concept for a multi-agent system in the travel medicine domain with software agents for distributed maintenance with explanation capabilities.

We gave an short overview of the Case Factory and Case Factory Organization and described the required tasks of the individual agent roles. The roles do not describe any concrete implementation of tasks or communications. The realization of the described concept and the implementation of the agents within a multi-agent system is the next step in our research. We will implement the single agents and agent teams as well as evaluation and maintenance strategies and evaluate the extended multi-agent system.

References

1. Althoff, K.D.: Collaborative multi-expert-systems. In: Proceedings of the 16th UK Workshop on Case-Based Reasoning (UKCBR-2012), located at SGAI International Conference on Artificial Intelligence, December 13, Cambride, United Kingdom. pp. 1–1 (2012)

2. Althoff, K.D., Bach, K., Deutsch, J.O., Hanft, A., Mänz, J., Müller, T., Newo, R., Reichle, M., Schaaf, M., Weis, K.H.: Collaborative multi-expert-systems – realizing knowledge-product- lines with case factories and distributed learning systems. In: Baumeister, J., Seipel, D. (eds.) KESE @ KI 2007. Osnabrück (Sep 2007)

3. Althoff, K.D., Hanft, A., Schaaf, M.: Case factory: Maintaining experience to learn. In: Pro- ceedings of the 8th European conference on Advances in Case-Based Reasoning. pp. 429–

442 (2006)

4. Althoff, K.D., Reichle, M., Bach, K., Hanft, A., Newo, R.: Agent based maintenance for modularised case bases in collaborative mulit-expert systems. In: Proceedings of the AI2007, 12th UK Workshop on Case-Based Reasoning (2007)

5. Bach, K.: Knowledge Acquisition for Case-Based Reasoning Systems. Ph.D. thesis, Univer- sity of Hildesheim (2013), dr. Hut Verlag Mnchen

6. Ferrario, M.A., Smyth, B.: A user-driven distributed maintenance strategy for large-scale case-based reasoning systems. In: ECAI Workshop Notes. pp. 55–63 (2000)

7. Ferrario, M.A., Smyth, B.: Distributing case-based maintenance: The collaborative maintenance approach. Computational Intelligence 17(2), 315–330 (2001)

8. Geissenbuhler, A., Miller, R.A.: Distributing knowledge maintenance for clinical decision- support systems: The ”knowledge library” approach. In: Proceedings of the AMIA Sympo- sium. pp. 770–774 (1999)

9. Iglezakis, I.: The conflict graph for maintaining case-based reasoning systems. In: Case- Based Reasoning Research and Development: Proceedings of the Fourth International Con- ference on Case-Based Reasoning (2001)

10. Iglezakis, I., Roth-Berghofer, T.: A survey regarding the central role of the case base for maintenance in case-based reasoning. In: ECAI Workshop Notes. pp. 22–28 (2000) 11. Patterson, D., Anand, S., Hughes, J.: A knowledge light approach to similarity maintenance

for improving case-base competence. In: ECAI Workshop Notes. pp. 65–78 (2000) 12. Racine, K., Yang, Q.: Maintaining unstructured case bases. In: Case-Based Reasoning and

Development. pp. 553–564 (1997)

13. Racine, K., Yang, Q.: Redundancy and inconsistency detection in large and semi-structured case bases. IEEE Transactions on Knowledge and Data Engineering (1998)

14. Reuss, P., Althoff, K.D.: Explanation-aware maintenance of distributed case-based reasoning systems. In: LWA 2013. Learning, Knowledge, Adaptation. Workshop Proceedings. pp. 231–

325 (2013)

(30)

15. Richter, M.M.: Introduction. chapter 1 in case-based reasoning technology - from founda- tions to applications. lnai 1400, springer (1998)

16. Roth-Berghofer, T.: Knowledge maintenance of case-based reasoning systems. The SIAM methodology. Akademische Verlagsgesellschaft Aka GmbH (2003)

17. Smyth, B.: Case-based maintenance. In: Proceedings of the 11th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems (1998) 18. Smyth, B., Keane, M.: Remembering to forget: A competence-preserving case deletion pol- icy for case-based reasoning systems. In: Proceedings of the 13th International Joint Confer- ence on Artificial Intelligence. pp. 377–382 (1995)

19. Smyth, B., McKenna, E.: Competence models and the maintenance problem. Computational Intelligence (2001)

20. Stahl, A.: Learning feature weights from case order feedback. In: Case-Based Reasoning Research and Development: Proceedings of the Fourth International Conference on Case- Based Reasoning (2001)

21. Swanson, E.B.: The dimensions of maintenance. In: Proceedings of the 2nd International Conference on Software Engineering. pp. 492–497 (1976)

22. Wilson, D.: Case-Based Maintenance: The Husbandry of Experience. Ph.D. thesis, Faculty of the University Graduate School, Department of Computer Science, University of Indiana (2001)

23. Wooldridge, M., Jennings, N., Kinney, D.: The gaia methodology for agent-oriented analysis and design. Autonomous Agents and Multi-Agent Systems 3, 285–312 (2000)

(31)

KDML: Workshop on Knowledge

Discovery, Data Mining and Machine

Learning

(32)

Dyad Ranking Using a Bilinear Plackett-Luce Model (Abstract)

^?

Dirk Sch¨afer¹ and Eyke H¨ullermeier²

1 University of Marburg, Germany dirk.schaefer@uni-marburg.de

2 Department of Computer Science University of Paderborn, Germany

eyke@upb.de

Preference learning is an emerging subfield of machine learning, which deals with the induction of preference models from observed or revealed preference information [2]. Such models are typically used for prediction purposes, for example, to predict context-dependent preferences of individuals on various choice alternatives. Depending on the representation of preferences, individuals, alternatives, and contexts, a large variety of preference models are conceivable, and many such models have already been studied in the literature.

A specific type of preference learning problem is the problem oflabel ranking, namely the problem of learning a model that maps instances to rankings (total orders) over a finite set of predefined alternatives [3]. An instance, which defines the context of the preference relation, is typically characterized in terms of a set of attributes or features; for example, an instance could be a person described by properties such as sex, age, income, etc. As opposed to this, the alternatives to be ranked, e.g., the political parties of a country, are only identified by their name (label), while not being characterized in terms of any properties or features.

In practice, however, information about properties of the alternatives is often available, too, and such information could obviously be useful from a learning point of view. Motivated by this observation, we introduce dyad ranking as a generalization of the label ranking problem. In dyad ranking, not only the instances but also the alternatives are represented in terms of attributes. For learning in the setting of dyad ranking, we propose an extension of an existing label ranking method based on the Plackett-Luce model, a statistical model for rank data [1]. First experimental studies with real and synthetic data confirm the usefulness of the additional feature information of alternatives.

References

1. W. Cheng, K. Dembczy´nski, and E. H¨ullermeier. Label ranking methods based on the Plackett-Luce model. In Proceedings ICML, 27th International Conference on Machine Learning, pages 215–222, Haifa, Israel, 2010.

?Copyright c 2014by the paper’s authors. Copying permitted only for private and academic purposes. In: T. Seidl, M. Hassani, C. Beecks (Eds.): Proceedings of the LWA 2014 Workshops: KDML, IR, FGWM, Aachen, Germany, 8-10 September 2014, published at http://ceur-ws.org

(33)

2. J. F¨urnkranz and E. H¨ullermeier, editors. Preference Learning. Springer, 2011.

3. S. Vembu and T. Gärtner. Label ranking: A survey. In J. Fürnkranz and E. Hüllermeier, editors,Preference Learning. Springer, 2011.

(34)

Improved Questionnaire Trees for Active Learning in Recommender Systems

Rasoul Karimi¹, Alexandros Nanopoulos², Lars Schmidt-Thieme¹

1 Information Systems and Machine Learning Lab Marienburger Platz 22 University of Hildesheim 31141 Hildesheim Germany

karimi, schmidt-thieme@ismll.uni-hildesheim.de

2 Department of Business Informatics., Schanz 49, University of Eichstatt-Ingolstadt , 85049 Ingolstadt, Germany

nanopoulos@ku.de

Abstract. A key challenge in recommender systems is how to profile new-users. This problem is called cold-start problem or new-user problem. A well-known solution for this problem is to use active learning techniques and ask new users to rate a few items in order to reveal their preferences. Recently, questionnaire trees (tree structures) have been proposed to build such adaptive questionnaires. In this paper, we improve the questionnaire trees by splitting the nodes of the trees in a finer-grained fashion. Specifically, the nodes are split in a 6-way manner instead of 3-way split. Furthermore, we compare our approach to online updating and show that our method outperforms online updating in order to fold-in the new user into recommendation model. Finally, we develop three simple baselines based on the questionnaire trees and compare them against the state-of-the-art baseline to show that the new- user problem in recommender systems is tough and demands a mature solution.

1 Introduction

Recommender systems help web users to address information overload in a large space of possible options [1]. Collaborative filtering is the traditional technique for recommender systems. Evidently, the performance of collaborative filtering depends on the amount of information that users provide regarding items, most often in the form of ratings. This problem is amplified for new users because they have not provided any rating which impacts negatively on the quality of gen- erated recommendations. A simple and effective way to overcome this problem, is by posing queries to new users in order that they express their preferences about selected items, e.g., by rating them. Nevertheless, the selection of items must take into consideration that users are not willing to answer a lot of such Copyright c 2014by the paper’s authors. Copying permitted only for private and academic purposes. In: T. Seidl, M. Hassani, C. Beecks (Eds.): Proceedings of the LWA 2014 Workshops: KDML, IR, FGWM, Aachen, Germany, 8-10 September 2014, published at http://ceur-ws.org

(35)

queries. To address this problem,active learning methods have been proposed to acquire the most informative ratings, i.e ratings from users that will help most in determining their interests [3, 4].

Recently, active learning based on tree structures have been proposed by (Golbandi et al. [2]). In [2], the tree structures are ternary because there are three possible answers for queries: ”Like”, ”Dislike”, or ”Unknown”. In datasets like Netflix and MovieLens that the range of the ratings is from 1 to 5, ratings from 1 to 3 are considered as ”Dislike” and ratings 4 and 5 are treated as ”Like”.

Moreover, the missing ratings are considered as ”Unknown”, meaning users do not know the queried item, so they can not rate it.

Nevertheless, [2] was a breakthrough in the literature of active learning for recommender systems. In our previous paper [5], we improved [2] by incorpo- rating matrix factorization into the tree structures and proposing a sampling method to speed up the tree construction algorithm. In this paper, we improve it one step further by upgrading the ternary trees to 6-way trees, meaning the nodes are split in a 6-way fashion. In the 6-way split, there is one child node per each rating from 1 to 5 and one child node for the ”Unknown” response. As the 6-way split distinguishes users tastes more precisely, it is expected that the accuracy of the rating prediction also improves. On the other hand, the 6-way split might lead to overfitting, which affects adversely on the accuracy. There- fore, we need a rating prediction model that handles the overfitting issue very well. We apply the 6-way split to two prediction models and show the effect of the overfitting on the accuracy of the 6-way split.

2 Related Work

The idea of using decision trees for the cold-start recommendation was proposed by (Rashid et al. [7]). They tried to formalize the cold-start problem in a su- pervised learning context and solve it through decision trees. However, they face challenges that force them not to use standard decision tree learning algorithms such as ID3 and C4.5. (Golbandi et al. [2]) improved [7] by advocating a specialized version of decision trees to adapt the preference elicitation process to the new user’s responses. As our method relies on [2], we briefly explain it in this section.

Here, each interior node is labeled with an itemi ∈ I and each edge with the user’s response to itemi. The new user preference elicitation corresponds to following a path starting at the root by asking the user to rate items associated with the tree nodes along the path and traversing the edges labeled by the users response until a leaf node is reached. Here, decision trees are ternary. Each internal tree node represents a single item on which the user is queried. After answering the query, the user proceeds to one of the three subtrees, according to her answer. The answer is either Like, Dislike, or Unknown. The Unknown means users are not able to rate the queried item because they do not know it.

Letting users not to rate the queried items in case they do not know it, is crucial because it happens frequently in recommender systems.