Relational networks of conditional preferences

DOI 10.1007/s10994-012-5309-4

Relational networks of conditional preferences

Frédéric Koriche

Received: 30 January 2012 / Revised: 6 May 2012 / Accepted: 14 June 2012 / Published online: 18 July 2012

© The Author(s) 2012

Abstract Much like relational probabilistic models, the need for relational preference mod- els naturally arises in real-world applications involving multiple, heterogeneous, and richly interconnected objects. On the one hand, relational preferences should be represented into statements which are natural for human users to express. On the other hand, relational pref- erence models should be endowed with a structure that supports tractable forms of reasoning and learning. Based on these criteria, this paper introduces the framework of relational con- ditional preference networks (RCP-nets), that maintains the spirit of the popular “CP-nets”

by expressing relational preferences in a natural way using the ceteris paribus semantics. We show that acyclic RCP-nets support tractable inference for optimization and ranking tasks.

In addition, we show that in the online learning model, tree-structured RCP-nets (with bi- partite orderings) are efficiently learnable from both optimization tasks and ranking tasks, using linear loss functions. Our results are corroborated by experiments on a large-scale movie recommendation dataset.

Keywords Conditional preferences·Relational networks·Preference optimization· Preference ranking·Online learning

1 Introduction

A recurrent issue in AI is the development of rational agents capable of tailoring their actions and recommendations to the preferences of human users. The spectrum of applications that depend on this ability is extremely wide, ranging from adaptive interfaces and configuration software, to recommender systems and group decision-making (Brafman and Domshlak 2009). In a nutshell, the crucial ingredients for addressing this issue are representation, reasoning and learning. In complex domains, we need a representation that offers a compact encoding of preference relations defined over large outcome spaces. We also need to be

Editors: S. Muggleton and J. Chen.

F. Koriche (

LIRMM, CNRS UMR 5506, Université Montpellier II, Montpellier Cedex 5, France e-mail:[email protected]

able to use this representation effectively in order to answer a broad range of queries. And, since the performance of decision makers is dependent on their aptitude to reflect users’

preferences, we need to be able to predict and extract such preferences in an automatic way.

Among the different preference models that have been devised in the literature, condi- tional preference networks (CP-nets) have attracted a lot of attention by providing a compact and intuitive representation of qualitative preferences (Boutilier et al.2004b). By analogy with Bayesian networks, CP-nets are graphical models in which nodes describe variables of interest and edges capture preferential dependencies between variables. Each node is labeled with a table expressing the preference over alternative values of the node given dif- ferent values of the parent nodes under a ceteris paribus (“all else being equal”) assumption.

For example, in a CP-net for movie recommendation, the rule:

Genre:^comedydrama|^Date=^fifties

might state that, for a film released in the fifties I prefer a comedy to a drama, provided that all other properties are the same. The semantics of a CP-net is a preference ordering on outcomes derived from such reading of entries in the conditional preference tables.

Despite their popularity, CP-nets are intrinsically limited to “attribute-value” domains.

Many applications, however, are richly structured, involving objects of multiple types that are related to each other through a network of different types of relations. Such applications pose new challenges for devising relational preference models endowed with expressive representations, efficient inference engines, and fast learning algorithms.

In this paper, we introduce the framework of relational conditional preference networks (RCP-nets) that extends ceteris paribus preferences to relational domains. Briefly, an RCP- net is a template over a relational schema which specifies a ground CP-net for each particular set of objects. Based on the ceteris paribus paradigm, the representations provided by RCP- nets are transparent, in that a human expert can easily capture their meaning. For example, in an RCP-net for movie recommendation, the entry:

Movie.Genre:actiondrama|mode(Movie.Audience.User.Age)=teen

might capture the stereotype that, all other things being equal, action movies are preferable to dramas if the majority of people in the audience are teenagers.

In essence, the interest of RCP-nets lies in their ability to compare and order relational outcomes. Semantically, a relational outcome consists in a set of objects interconnected by the functional dependencies of the database schema. Two outcomes are “comparable” if they are defined over the same set of objects, but differ in the values assigned to the attributes of objects. For example, in configuration software (Junker2006), the overall goal of the deci- sion maker is to assemble from an available catalog of generic objects a customized product that meets user’s preferences. Customized products, such as computers, cars, insurance prod- ucts and travel packages, can be described as relational outcomes. In recommender systems (Jannach et al.2010), a fundamental task is to predict what degree of desire a user would give to a new, unrated, item. Modeling the interaction of users and items as relational outcomes can help the system in making a better personalized recommendation by incorporating rela- tional information about the user, such as her community in social networks (Golbeck2006), or about the item, such as the actors, directors and critics of a movie (Melville et al.2002;

Newton and Greiner2004).

From a computational viewpoint, a key feature of our framework is that the class of acyclic RCP-nets supports efficient inference for two well-studied inference tasks in pref- erence handling: outcome optimization and outcome ranking. In outcome optimization, the

decision maker is given a partial outcome in which some object attributes are left unspec- ified; the task is to find a maximally preferred completion of this outcome. For an acyclic CPR-net, (unconstrained) outcome optimization can be solved in polynomial time using a greedy algorithm that finds a maximally preferred completion of a partial outcome accord- ing to some topological order induced by the preference network. In outcome ranking, the decision maker is given a set of outcomes defined over the same collection of objects, but which differ in the values assigned to the object attributes; the task is to rank these outcomes in some non-decreasing order of preference. Again, such a task can be solved in polynomial time for acyclic RCP-nets, by compiling the network into a utility function that assigns a score to each outcome under consideration.

The learnability of RCP-nets is analyzed within the online learning setting (Cesa-Bianchi and Lugosi2006), a well-studied theoretical model for devising algorithms capable of mak- ing and updating recommendations in real-time. In this setting, the decision maker observes instances of a reasoning task in a sequential manner. On roundt, after observing thetth instance, the decision maker attempts to predict the solution associated with this instance.

The prediction is formed by a hypothesis chosen from a predefined classN of RCP-nets.

The decision maker can use this information to choose another hypothesis from the class N before proceeding to the next round. As a common thread in online learning, we make no assumption regarding the sequence of instance-solution pairs. This setting is thus general enough to capture agnostic situations in which the “true” preference model is not necessarily an element of the predefined class_N.

To measure the performance of the decision maker, we consider two standard metrics.

The first, called regret, measures the difference in cumulative loss between the decision maker and the optimal hypothesis inN. The second metric is computational complexity, i.e.

the amount of computer resources required to choose hypotheses and to predict solutions.

Based on these metrics, we show that the class of tree-structured RCP-nets (with bipartite or- derings) is efficiently learnable from both optimization tasks and ranking tasks, using linear loss functions. Our online learning algorithm is an extension of the Hedge algorithm (Fre- und and Schapire1997) that exploits the Matrix-Tree Theorem (Tutte1984) for generating directed spanning trees at random.

The paper is organized as follows. After introducing the necessary background in graph theory (Sect.2), we examine the syntax and semantics of RCP-nets in Sect.3. The theoretical results concerning reasoning with acyclic RCP-nets and learning with tree-structured RCP- nets are presented in Sects.4and5, respectively. In Sect.6we illustrate the learning potential of our framework with experiments on a large dataset. In Sect.7, we compare our framework with related work and, in Sect.8, we conclude by mentioning some perspectives of further research.

2 Preliminaries

Before delving into the representation of relational preference networks, we review the basic concepts from graph theory used in this paper. A digraph is a pairG=(X,E), whereX is a nonempty, finite set, andE is a binary relation onX. The elements ofX are the nodes ofG, and the elements ofE are the (directed) edges ofG. The size ofG, denoted |G|, is given by the number of its edges. Undirected graphs are represented here as digraphs for which the binary relation on nodes is symmetric. Notably, the underlying graph of a digraph G=(X,E)is the pair formed byX and the symmetric closure ofE.

For a digraph G and a pair of nodes X, Y, a walk of lengthkin G fromXto Y is a sequence of nodes(X1, . . . , Xk+1)such thatX=X1,Y=Xk+1and(Xi, Xi+1)is an edge

in_Gfor 1≤i≤k. The walk is a path if all nodes are distinct, and a cycle ifX1=X_k+1and all intermediate nodes are distinct. For any pair of nodesX, Y, if there is a path of lengthk fromXtoY thenY is an ancestor ofX, andXis a descendant ofY. In the specific case wherek=1,Yis called a parent ofX, andXis called a child ofY. A root (or source) is a node with no parents, and dually, a leaf (or sink) is a node with no children. The in-degree (resp. out-degree) of a digraph _G is the maximum number of parents (resp. children) per node inG.

The deletion of an edge (X, Y ) from a digraph G is the digraph obtained by simply removing(X, Y )fromG. The contraction of(X, Y )inGis the digraph obtained by merging (X, Y )with a new nodeZand redefining any edge(X, X)(resp.(Y, Y )) to(Z, X)(resp.

(Y, Z)).

A digraph is acyclic if it contains no cycles. An acyclic digraphG=(X,E)is complete bipartite ifX can be partitioned into two sets X1and X2 such thatE =X1×X2. In the particular case where|X1| =1,Gis a star. A forest is an acyclic digraph of in-degree one, and a tree is a forest with exactly one root node. A spanning tree (resp. spanning forest) of a digraph_G=(X,E)is a tree (resp. forest) for which the node set is_X and the edge set is contained inE. LetKndenote the complete digraph of ordern. Then, by Cayley’s formula, the number of spanning trees ofKnrooted at a fixed node isnⁿ⁻², and hence, the number of spanning forests ofKnis(n+1)ⁿ⁻¹.

For a digraphGwith node set{X1, . . . , Xn}, a linear extension ofGis a permutationπ over{1, . . . , n}such that, if there is an edge fromXi toXj, thenπ(i) < π(j ). It is well- known that ifG is acyclic, then a linear extension ofGcan be constructed inO(|G|)time using a topological sort algorithm.

Finally, a weighted digraph is a tripleG=(X,E, w), where(X,E)is a digraph andw is a map fromX×X to the set of nonnegative reals, such thatw(X, Y ) >0 if and only if (X, Y )∈E. For any subgraph_Gof_G, the weight of_Gis given by the product of weights of its edges. The Laplacian ofGis the real matrixΛ(G)overX×X for which each entry λ(X, Y )is given by:

λ(X, Y )=

Z∈Xw(Z, Y ) ifX=Y

−w(X, Y ) otherwise

LetΛ_X(G)be the matrix obtained by deleting the row ofXand the column ofXfromΛ(G).

By the Matrix-Tree Theorem (Tutte1984, Theorem 6.27), the determinant ofΛX(G)is equal to the sum of weights of all spanning trees ofGrooted atX. Based on this property, various algorithms have been proposed in the literature for generating in polynomial time random spanning trees of digraphs (see e.g. Kulkarni1990; Colbourn et al.1996).

3 RCP-nets

On the surface, our representation for relational preferences is similar to a probabilistic relational model (Getoor et al.2002): the representation is structured in a graphical way by exploiting conditional independencies. However, the nature of connections between nodes in the graph is different: whereas conditional probabilities are quantitative and specify a probability measure over the outcome space, conditional preferences are qualitative and specify a strict partial order between outcomes.

3.1 Language

The basic building block of our framework is a relational schema that specifies a database structure. In order to clarify dependencies among the attributes of interconnected objects, the schema is represented as a digraphS.

The nodes ofS are separated into class names and attribute names. Intuitively, a class name denotes a type of objects, and an attribute name captures an elementary property that can be attached to a class name. Each attribute nameAis associated with two predefined components: a finite domainDAand a finite setΓAof aggregate functions or aggregators.

EachγA∈ΓAmaps any vector of values inDAinto a single value ofDA. Common aggrega- tors include the mode (most frequently occurring value), the median, maximum or minimum (if values are ordinals), and the mean (if values are cardinals). The domain size ofSis given by the maximum of the sizes of its domains.

The edges ofS capture functional constraints between nodes: they are separated into attributes and references. An attribute is an edge of the form (X, A), also denotedX.A, whereXis a class name andAan attribute name. A reference is an edge of the formX.Y whereXandYare class names.

There is a natural correspondence between our representation and that of relational databases. Each class name X is associated with a table and each of its adjacent nodes is associated with a column in the table. For an attributeX.A, the entries in the correspond- ing column are values in D_A, and for a referenceX.Y, the entries are foreign keys, each identifying an object inY. We note in passing that this representation does not prevent us from having complex relationships between entities: using a standard reification technique, eachk-ary relationship can be captured by introducing a new class name associated withk references, in which each object corresponds to a row in the relationship. These notions are illustrated in the following example.

Example 1 Suppose we would like to design a movie recommender system that periodically suggests a list of movies to each subscriber. The relational schema, described in Fig.1, is composed of movies, actors, directors, critics, and users. Each box specifies a class name with its adjacent attribute names (in roman style) and reference names (in italic style); the dotted lines indicate the types of objects referenced. For instance, the class nameCastspeci- fies the rank of actors playing in a movie; this class is associated with the attributeCast.Rank and the referencesCast.ActorandCast.Movie.

Fig. 1 A relational schema for the movie domain

A chain is a path in the underlying graph of _S of the formX.Rwhere X is a class name and R is a (possibly empty) sequence of class names. A slot is an expression of the form X.R.AwhereX.R is a chain andA is an attribute name connected to the last class name of X.R. Intuitively,X.R.Adenotes a binary relation between objects of type X and values of type A. Note that because any slot is a path in the symmetric closure ofS, the relation captured byX.R.Ais not necessarily functional. For example, the slot User.Audience.Movie.Genrerefers to all genres of movies watched by a user. A term is an aggregated slot, that is, an expression of the formγA(X.R.A)whereX.R.Ais a slot andγA

is an aggregator inΓA. The set of all attributes occurring inSis denotedA(S)and the set of all possible terms that can be generated fromSis denotedT(S).

In this study, preference relations are modeled as strict partial orders. Formally, given an arbitrary setX, a preference orderingonX is a binary relation onX that is irreflexive, antisymmetric and transitive.

Definition 1 For a setA⊆A(S)of attributes and a setT ⊆T(S)of terms, a relational conditional preference network (RCP-net) is a pairN=(par,cpt)such that:

− par associates to each attributeX.A∈Aa parent set par(X.A), which is a collection {γA₁(X.R1.A1), . . . , γAp(X.Rp.Ap)}of terms inT rooted atX.

− cpt associates to each attributeX.A∈A a conditional preference table cpt_X.A, which maps each vectoruinDA₁× · · · ×DAp into a preference ordering cpt_X.A(u)overDA. By_N[A,T]we denote the class of all RCP-nets defined over the set_Aof attributes taking parents in the setT of terms. Each attribute in A is said to be controllable. For example, in a movie recommender system, it is legitimate to consider that movie attributes, including the genre, the release date and the duration of a film, are controllable. On the other hand, user attributes such as the age, the gender and the occupation of a person, are typically uncontrollable.

Given an RCP-netN∈N[A,T], the dependency graph ofNis the digraph_G(N )with node setAand such that there is an edge fromX.AtoX.Aif and only ifX.Ais the suffix of some slot in par(X.A). Based on this notion, an RCP-net is acyclic if its dependency graph is acyclic, and tree-structured if its dependency graph is a forest. Finally, an RCP-net is bipartite-ordered (resp. star-ordered) if each of the entries of its conditional preference tables is a complete bipartite digraph (resp. a star).

The parent size of N is the maximum numberp of parents per attribute in N. It is important to keep in mind that the parent size of an RCP-net does not necessarily coincide with the in-degree of its dependency graph. In particular, a tree-structured RCP-netNcan have parent sets composed of multiple terms, provided that at most one suffix is inA. Example 2 Consider a restricted view of our movie recommendation domain, described in Fig. 2. The RCP-net N is defined over the setA of controllable attributes including Critics.Rating,Movie.DurationandMovie.Genre. We assume here that all attributes are as- sociated with the mode aggregator. The dependency graph ofNis depicted in the left part of the figure, while the parent sets and preference tables ofNare presented in the right part.

For instance, the first entry of the table associated toMovie.Durationstates that a long movie is preferred over a short one if the aggregated reviews for this film are positive. Based on the above terminology, we can observe thatNis both tree-structured and bipartite-ordered.

In particular, the parents ofMovie.Genreare defined over uncontrollable attributes.

Fig. 2 A tree-structured RCP-net for the movie domain

3.2 Semantics

Given a schemaS, a skeleton forS is a mapκ that assigns to each class nameXa finite setJXKκ of objects, and to each reference X.Y a functionJX.YKκ fromJXKκ intoJYKκ. We assume that each object is associated to a unique class, i.e.JXKκ∩JYKκ=∅whenever X=Y. Based on the standard semantics of inverse and composition operations, a skeleton assigns a binary relation to any chain in the schema. A ground chain is an expression of the formo.RwhereX.Ris a chain andois an object inJXKκ. The notions of ground attribute and ground term are defined similarly. For a ground chaino.R, we denote byJo.RKκthe set of objectsosuch that(o, o)∈JX.RKκ.

Given a skeletonκ and a collection of attributesA, we denote byAκ the set of ground attributes{o.A:X.A∈A, o∈JXKκ}. For an RCP-netN, the ground dependency graph of N with respect toκ is the digraphGκ(N )with node setAκ and such that there is an edge fromo.Atoo.Aif and only if there is an edge fromX.AtoX.AinG(N ), whereXis the class ofoandXis the class ofoinκ. Clearly, ifG(N )is acyclic thenGκ(N )is acyclic.

A relational outcome or interpretation is a mapI that extends a skeletonκ by assign- ing to each attributeX.A a functionJX.AKI fromJXKκ intoDA. By Iκ, we denote the space formed by all interpretations extendingκ. Given a ground attribute o.A, the value assigned byItoo.Ais denotedJo.AKI. More generally, given a ground sloto.R.Awhere Jo.RKκ= {o1, . . . , on}, we denote byJo.R.AKI the vectorvinD_Aⁿ such thatvi=Joi.AKI

for 1≤i≤n.

With these notions in hand, we are now ready to examine the ceteris paribus semantics of relational preference networks. Consider an RCP-netN defined over a set of attributes A, and a skeletonκ. A pair(I, J )of interpretations extendingκis called a flip on a ground attributeo.A∈Aκ if they are everywhere identical onAκ, excepted foro.A. Given an at- tributeX.Awith parent set{γ_A1(X.R1.A1), . . . , γ_Ap(X.R_p, A_p)}and a flip(I, J ) ono.A, we say thatIdominatesJ inNif the valueJo.AKI is preferred to the valueJo.AKJ in the entry of the table cpt_X.Aassociated to the vector:

qpar(o.A)y

I= γA₁

Jo.R1.A1KI

, . . . , γAp

Jo.Rp.ApKI

By extension, given any pair(I, J )of interpretations inIκ, we say thatIdominatesJ inN, and writeINJ, if there is a sequence(I1, . . . , In)of flips such thatI1=I,In=J, andIi

dominatesIi+1inNκfor 1≤i < n.

Table 1 Two relational outcomes for the movie domain

User Critics Movie

Name Age Gender Pub. Title Rating Title Dur. Genre

I₁ Mary mid female IMDb Big Fish high Big Fish long romance

I₂ GoodFellas GoodFellas action

Example 3 Consider two relational outcomes I1 and I2 for the schema given in Fig.2, specified in the Table1. We remark that(I1, I2)is a flip on the movie genre. Thus, using the conditional preference table ofMovie.Genreit follows thatI1dominatesI2.

Finally, we say that an RCP-netNis coherent if, for any skeletonκ, the binary relation N overIκis a preference ordering.

Theorem 1 Any acyclic RCP-net is coherent.

Proof LetN be an acyclic RCP-net defined over a set of attributes_A, letκ be a skeleton, and suppose thatINI holds for someI∈Iκ. By definition of the dominance relation, there is a sequence(I1, . . . , In)of flips such thatI1=I=Inand,IiNI_i+1for 1≤i < n.

LetSbe the set of all ground attributeso.AinAκ such thatJo.AKIi=Jo.AKIi+1 for some flip (Ii, I_i+1). Consider any linear extension of S according to Gκ(N ), and take the first elemento.Ain the ordering. Leti be the first index in the sequence such thatJo.AKIi= Jo.AKIi+1. It follows thatJo.AKI=Jo.AKIi+1. However, becauseGκ(N )is acyclic, there is no parent ofo.AinS, and hence the ground conditionJpar(o.A)KI remains unchanged during all the sequence of flips. It follows thatJo.AKI_i+1=Jo.AKI_i+2= · · · =Jo.AKIn, and henceJo.AKI=Jo.AKIn, which contradicts the assumption thatINI. For the problems under consideration in the remaining sections, we will assume a pre- fixed and known schemaS of domain sized, in which any aggregator can be implemented by a procedure that runs inO(nlogd)time, wherenis the maximum number of objects per class name assigned by a skeleton. Because the size of preference tables grows exponen- tially with the number of parents, we will also assume that the parent sizepof any RCP-net is constant.

4 Preference reasoning

For a class of RCP-nets, a reasoning task consists in a set of instances and a set of solutions.

Of particular interest here is the classNacy[A,T]of acyclic RCP-nets. The size ofAis denoteda, and the maximum of the sizes of terms inT is denotedk. The next lemma states that the parent values of an attribute can be retrieved in quasi-linear time using standard join and projection operations.

Lemma 1 LetN be an RCP-net inNacy[A,T]. Then, for any interpretationI and any ground attributeo.A, the vector of parent valuesJpar(o.A)KI can be constructed in_O(t ) time, wheret=knlog₂(n)+nlog₂(d).

Proof Consider any referenceX.Y in the schema _S. The inverseJY.XKI of the relation JX.YKI can be computed inO(n)time. In addition, the tuples inJX.YKI(or its inverse) can be ordered lexicographically, which requiresO(nlog₂n)steps. Based on this ordering, the compositionJX.YKI◦JY.ZKI can be performed inO(n)time, in the following way: project JX.YKI onto the objects shared byJY.ZKIand prune any tuple inJY.ZKI that has no match in that projection. Thus, theith valueγAi(Jo.Ri.AiKI)of the tupleJpar(o.A)KIcan be found in_O(knlog₂n+nlog₂d)time using at mostkjoin operations and one aggregate operation.

Since the number of parents per attribute is constant, the result follows.

4.1 Preference optimization

A partial outcome is an interpretationIthat assigns the value “∗” (unknown) to some ground attributes. A completion ofIwith respect to a set of attributesA, is a map that extendsI by replacing the unknown value of each attribute inAκwith a value of appropriate type.J is optimal for an RCP-netNif there is no distinct completionJofI such thatJNJ.

Based on these considerations, an outcome optimization task forN[A,T]is a reasoning task in which instances are partial outcomes and solutions are outcome completions with respect toA. Given an RCP-netNand an instanceI, the task is to find a completionJ ofI that is optimal forN.

For acyclic RCP-nets, such a completion can be found in polynomial time using the forward sweep algorithm (Boutilier et al.2004b), adapted to relational domains. Given an RCP-netN defined over an attribute set Aand a partial outcome I, we first construct a topological ordering of_Aκ according to the ground dependency graph_Gκ(N ), whereκis the skeleton of I. Then, starting from J =I, we instantiate each o.A∈Aκ in turn to a maximally preferred value in the preference ordering cpt_X.A(Jpar(o.A)KI).

Example 4 Suppose thatAnnis a young woman. Based on the tree-structured RCP-net in Fig. 2, what would be her favorite romance movies? Starting from the partial outcomeI in which onlyAnn’s attributes are known, we can derive two optimal completionsJ1and J2ofI. For both completions, the value ofCritics.Ratingis set tohigh, and the value of Movie.Durationis set tolong. The value ofMovie.GenreisactionforJ1andromanceforJ2. Theorem 2 LetNbe an RCP-net in_Nacy[A,T]. Then, for any partial outcomeI, finding an optimal completion ofI forNcan be done inO(ant )time.

Proof We first establish the correctness of the forward sweep algorithm. LetI be a partial outcome, andJ be the completion ofI returned by the algorithm. Suppose thatJNJ for some distinct completionJofI. In this case, there is a sequence of flips fromJtoJ inN. LetSbe the set of ground attributes for which the value is switched in the sequence.

BecauseGκ(N )is acyclic, there is at least oneo.A∈Sfor which the values of all parents are fixed byI. SinceJ andJare both extensions ofI, we haveJpar(o.A)KI=Jpar(o.A)KJ= Jpar(o.A)KJ. However, becauseJo.AKJ is optimal, the value ofo.Acannot be switched in the sequence, which contradicts the assumption thatJNJ.

Now, consider the computational cost of the algorithm. A linear extension ofG(N )can be found inO(a)time. Based on this ordering, a linear extension ofGκ(N )can be found in O(an)time, by replacing the attribute at positioniwith itsnground instances at positions i1, . . . , in, wherei1> i−1 andin< i+1. Since by Lemma1the parent assignment of each attribute can be found in_O(t )time, the algorithm returns a completion ofIin_O(ant )

time.

4.2 Preference ranking

For a skeletonκ, an outcome set is any finite collectionS= {I1, . . . , Im}of interpretations inIκ. Note that all interpretations in an outcome set are defined over the same set of objects, but differ in the values assigned to object attributes. A ranking ofSis a permutationπover [m] = {1, . . . , m}. We say thatπis consistent with an RCP-netNif, for any pairI_i, I_j inS, IiNIj implies thatIi occurs beforeIj inπ, i.e.π(i) < π(j ). An outcome ranking task is a reasoning task for which any instance is an outcome set of sizemand any solution is a permutation over[m]. Given an RCP-netN and an instanceS, the problem is to find a rankingπofSthat is consistent withN.

This problem can be solved in polynomial time for acyclic RCP-nets using a compilation technique inspired from Boutilier et al. (2001) and Brafman and Domshlak (2008). For a skeletonκ, a utility function is a mapφ:Iκ→R. Any utility functionφinduces a prefer- ence orderingφoverIκ such thatIφJ if and only ifφ (I ) > φ (J ). The functionφis consistent with an RCP-netN ifN impliesφ, that is, every linear extension ofφ is a linear extension ofN. Based on these notions, the overall idea of the compilation technique is to map any acyclic RCP-netNand any skeletonκinto a utility functionφoverIκthat is consistent withN. The functionφcan then be exploited for solving multiple ranking tasks defined overκ.

We now embark on technical aspects. The basic ingredients ofφ are two weight map- pings f and g, where f is used to quantify local preferences in the tables of N, while g is used to quantify global dependencies between attributes. Consider an attributeX.A with parent set {X1.R1.A1, . . . , Xp.Rp.Ap}. Then, for each vectoru of parent values in DA₁× · · · ×DAp and each valuevinDA,f (u, v)is the number of descendants ofvin the preference ordering cpt_X.A(u). Note thatf (u, v)≤ |DA| −1, andf (u, v)=0 ifvis a leaf.

The weight mappingg is constructed in a recursive way using a topological ordering ofG(N ). IfX.Ais the first attribute in the ordering, then we setg(X.A)=1. Assuming by induction hypothesis thatgis fixed for the firstk−1 elements in the ordering, if thekth elementX.Ahas no parents, theng(X.A)=1. Otherwise,

g(X.A)= 1

|DA| min

i=1,...,p

g(Xi.Ai)

Xj.Aj∈ch(X_i.Ai)|JXjKκ| (1) where{X1.A1, . . . , Xp.Ap}is the set of all parents ofX.AinG(N ), and ch(Xi.Ai)is the set of all children ofXi.AiinG(N ). The aim ofgis thus to distribute the weight of an attribute evenly between its ground children.

With these ingredients in hand, the utility function φ is specified as a sum of sub- utilities φX.A, each defined for a controllable attributeX.A in N. Namely, if X.Ais an attribute with parent set par(X.A)= {X1.R1.A1, . . . , Xp.Rp.Ap}, then φX.A associates to each vector u in DA₁ × · · · ×DAp and to each value v in DA the utility φX.A(u, v)= g(X.A)f (u, v). Based on these sub-utility functions, the value of any interpretationI is simply given by:

φ (I )=

X.A∈A

o∈JXKκ

φX.A

Jpar(o.A)KI,Jo.AKI

(2)

Example 5 Consider again the RCP-netN in Fig.2, and suppose that three long movies o1,o2and o3are presented to John, a middle-aged male user. Specifically, o1and o2are romance movies, while o3is an action movie. The corresponding reviews, denoted r1,r2

andr3, are positive foro1ando3, but quite negative foro2. We denote byI1(resp.I2and I3) the interpretation associated to the entitiesJohnand{r1, o1}(resp.{r2, o2}and{r3, o3}).

Now, according to our utility functionφ, we haveφ (I1)=1+1/2+0=3/2,φ (I2)=1/2 andφ (I3)=5/2. Therefore, the ranking(3,1,2)is consistent withN.

Theorem 3 LetNbe an RCP-net in_Nacy[A,T]andκ be a skeleton. Then, compilingN into a consistent utility functionφonIκcan be done inO(ad^p+1)time. Based on this utility functionφ, finding a consistent ranking of any outcome setS⊆Iκof sizemcan be done in O(ant mlog₂m)time.

Proof LetNκ be the CP-net that associates to each instanceo.Aof an attributeX.AinN a parent set par(o.A)and a preference table cpt_o.A. Specifically, ifX.Ais an attribute with parent set{X.R1.A1, . . . , X.Rp.Ap}, then:

− par(o.A)is the set of all attributesoij.Ai, such that 1≤i≤p, 1≤j≤ni, andJo.RiKκ= {oi1, . . . , o_ini},

− cpt_o.A is the map that assigns to each vectorw=(v₁, . . . ,v_p)in the spaceD(A1)ⁿ¹×

· · · ×D(Ap)^npthe preference ordering cpt_o.A(w)=cpt_X.A(v), wherevis the aggregated vector(γA₁(v1), . . . , γAp(vp)).

We can observe that the dependency graph ofN_κ isGκ(N ). By a direct application of Brafman and Domshlak (2008, Theorem 3), the utility functionφdefined above is consistent with the CP-netNκ. SinceNκ=N, it follows thatφis consistent with the RCP-netN.

Now, consider the computational cost required for constructingφ. For each of the values of each preference ordering inN, the weightf (u, v)can be computed inO(d)time. Since there are at mostaattributes andd^p table entries per attribute, the weight mapf can be constructed inO(ad^p+1)time. The weight mapgcan be obtained inO(a)time by simply constructing a topological ordering ofG(N )and memorizing the number of ground children per attribute during the traversal.

Finally, the utilityφ (I )of any interpretationI in the outcome setScan be computed in O(ant )time, as specified by equality (2). Thus, any ranking of S can be found in O(ant mlog₂m)time by labeling eachI∈Swithφand breaking ties arbitrarily.

5 Preference learning

By extending our previous considerations, a prediction task for a class of RCP-netsN is a triple(X,Y, ), whereXis a space of instances,Yis a space of solutions, and:N×X× Y→ [0, λ]is aλ-bounded loss function.

In the online learning model, the decision maker is a learning algorithm that observes instances of a prediction task in a sequence of rounds. At trialt, the algorithm receives an instance x_t∈X and is required to predict a corresponding solution N_t(x_t)∈Y using its current hypothesisNt∈N. Once the algorithm has predicted, the true solutionyt∈Y is revealed and the algorithm incurs the loss (Nt;xt, yt) that measures the discrepancy between the predicted solutionNt(xt)and the correct responseyt. The ultimate goal of the decision maker is to minimize the cumulative loss it suffers along its run. To achieve this goal, the algorithm is allowed to choose a new hypothesis in N at the end of each trial, possibly using a randomized strategy.

As mentioned in the introduction, we make no assumptions regarding the sequence of ex- amples. In this general setting, the performance of the decision maker is measured relatively

Parameters:

− a finite classN of RCP-nets with its component setC(N),

− a prediction task(X,Y, ), such thatis linear forN Initialization: Setθ1(C)=0 for eachC∈C(N) Trials: for each roundt=1,2, . . .

(1) chooseNtaccording to the distributionPθt(N )∼exp[

C∈Nθt(C)]

(2) receive instancext∈X (3) predict solutionNt(xt)∈Y (4) receive responseyt∈Y

(5) chooseη_tand setθ_t+1(C)=θ_t(C)−η_t(C;x_t, y_t)for eachC∈C(N)

Fig. 3 The expanded Hedge algorithm

to the performance of the best hypothesis inN. Namely, the regret of an online learning algorithmLwith respect to a sequence{(xt, yt)}ofT examples is given by the difference between the expected cumulative loss of the algorithm and the cumulative loss of the best hypothesis chosen with the benefit of hindsight:

regret L,

(x_t, y_t)

=E

t

(N_t;x_t, y_t)

−min

N∈N

t

(N;x_t, y_t)

A class of hypothesesN is online learnable with respect to a prediction task(X,Y, ) if there exists an online learning algorithmLsuch that, for any sequence ofT examples, the regret ofLis sublinear as a function of T. This condition implies that “on average”

the algorithm performs as good as the best fixed hypothesis in hindsight. If, in addition, the computational complexity ofLis polynomial in the dimension parameters associated toN, X, andY, thenN is efficiently learnable.

In this section, any RCP-net is viewed as a set of components, whose data structure will be clarified shortly. LetC(N)be the set of all distinct components generated from a class of hypotheses_N. A loss functionis linear for_N if(N;x, y)=

C∈N(C;x, y)for any N∈N, where (C;x, y)denotes the loss incurred by the componentC on the example (x, y).

Our learning algorithm is a variant of the Hedge algorithm (Freund and Schapire1997) adapted to structured models. Following Koolen et al. (2010), we call this algorithm Ex- panded Hedge. As indicated in Fig.3, the algorithm maintains a parameter vectorθt over C(N). On roundt, the algorithm predicts with a hypothesisNt chosen at random accord- ing to the exponential distribution Pθt induced byθ_t. Then,θ_t is updated using the rule θt+1(C)=θt(C)−ηt(C;xt, yt), whereηtis an adaptive learning rate.

The following result can be derived by a simple adaptation of Hedge’s amortized analysis (Cesa-Bianchi and Lugosi2006, Theorem 2.2).

Lemma 2 LetNbe a finite class of RCP-nets, and(X,Y, )be a predication task whereis aλ-bounded linear loss function forN. Then, for any sequence{(xt, yt)}ofTexamples, the regret of the Expanded Hedge algorithm with adaptive learning rateηt=(2/λ)√

ln|N|/t satisfies:

regret L,

(xt, yt)

≤λ Tln|N|

The rest of this section is devoted to the learnability issue of the class_Ntree of tree- structured and bipartite-ordered RCP-nets. In order to obtain an efficient online learning algorithm for this class, we need a polynomial time procedure for generating hypotheses at random according to an exponential distribution. To this end, we first present a general procedure for generating random subsets; this procedure is the backbone for constructing RCP-nets. Next, we investigate the problem of learning bipartite orderings, and then, we turn to the problem of learning tree structures. We conclude by applying the results to opti- mization and ranking tasks.

5.1 Generating random subsets

Let B be an arbitrary non-empty set of subsets of [b] = {1, . . . , b}, and letw be a map that associates to each subsetB∈Ba positive realw(B)that captures the weight ofB. In this section, we present a simple algorithm due to Kulkarni (1990) that generates random subsets of[b]which belong toBand such that the probability of generating a givenB∈B is proportional tow(B).

Before presenting the algorithm, we introduce the following notations. LetU andV be subsets of[b], not necessarily in B, such thatU⊆V. By [U, V], we denote the interval {B∈B:U⊆B⊆V}. The weight of[U, V], denotedw[U, V], is defined as the sum of weights of sets in[U, V], i.e.

w[U, V] =

B∈[U,V]

w(B) (3)

The algorithm, called Random Subset and described in Fig.4, starts from the largest in- terval[U, V]withU =∅andV = [b], and iteratively shrinks [U, V]until it contains a unique set inB. Namely, for each elementi∈ [b], the algorithm insertsi inU with proba- bilitypand removesifromV with probability 1−p, wherepis the ratio ofw[U∪ {i}, V] tow[U, V]. By a reformulation of Theorem 2.1. in Kulkarni (1990), we get the following result.

Lemma 3 For any nonempty setBof subsets of[b]and any positive weight functionwover B, the Random Subset algorithm produces a random variableU taking values in Bwith distribution proportional tow.

Parameters:

− a nonempty set_Bof subsets of[b]

− a positive weight functionw:B→R+

Initialization: SetU=∅andV= [b]

Fori=1 tob (1) letp=^w[_w[U,V]^U∪{i},V]

(2) generatequniformly at random in the interval[0,1] (3) ifq≤pthenU=U∪ {i}, elseV=V− {i}

(4) i=i+1 ReturnU

Fig. 4 The random subset algorithm

5.2 Parameter learning

After a digression on generating random subsets, let us investigate the problem of learn- ing the conditional preference tables of RCP-nets. More precisely, we examine the class Ntree[par]where the parent set par is prefixed and known. In this setting, any component of the preference network is specified as a triplet of the formC=(X.A,u, v)whereX.Ais an attribute with parent set{X1.R1, A1, . . . , X_p, R_p.A_p},uis a vector overD_A1× · · · ×D_Ap andvis a value inDA. Based on this notation, cpt_X.A(u)is encoded by the set of components (X.A,u, v), wherevis maximally preferred value inDA.

Given a schema of domain sized involvingaattributes, the number of components is bounded byad^p+1, and hence, remains polynomial inaanddwhenever the parent sizep is taken as constant. By contrast, the cardinality ofNtree[par]is bounded by(2^d−1)^adp, where(2^d−1)is the number of complete bipartite digraphs over d values. Because this cardinality grows exponentially withaandd, a key computational issue is to generate hy- potheses at random according to a given distribution overNtree[par]. Fortunately, this issue can be circumvented by exploiting the bipartite structure of preference orderings.

Lemma 4 For the class_Ntree[par], letθ be a parameter vector over the component set C(Ntree[par]). Then, generating a hypothesisN∈Ntree[par]at random according toPθ

can be done inO(ad^p+2)time.

Proof Consider the mapwthat assigns to eachN∈Ntree[par]the weight w(N )=

C∈N

w(C) wherew(C)=e^{θ (C)} (4)

By construction,wis proportional toPθ.

Now, consider any attributeX.Awith parents{X1.R1.A1, . . . , Xp.Rp.Ap}, and letube a vector inDA₁× · · · ×DAp. Without loss of generality, assume thatDA= {1, . . . , d}, and let {C1, . . . , Cd}be the set of components of the form Ci=(X.A,u, i). Recall that any bipartite preference orderingBoverDAis encoded by the set of components(X.A,u, i), whereiis maximally preferred value inDA. Based on equality (4), the weight ofBis given byw(B)=

i∈Bwi, wherewi=w(Ci). Let[U, V]be the set of all nonempty subsetsB ofDAsuch thatU⊆B⊆V. Based on the definition ofw[U, V]given in (3) and using the specification ofwin (4), we get that:

w[U, V] =

−1+

j∈V(1+wj) ifU=∅

i∈Uwi

j∈V\U(1+wj) otherwise (5)

The first equality in (5) is obtained by taking the sum of weights of all subsets ofV and removing the weight of the empty subset. The second equality follows by taking the sum of weights of all subsets ofV which includeU. Thus, using the Random Subset algorithm, we can generate inO(d²)time a random bipartite ordering for the entry cpt_X.A(u). Since there are at mostaattributes andd^ptuples of values per attribute, the result follows.

5.3 Structure learning

Let us turn to the more general glassNtree[A,T]of tree-structured RCP-nets with attribute setAand term setT. Since the parent structure is unknown, any component is now defined

as a quadrupletC=(X.A,par(X.A),u, v), whereX.Ais an attribute, par(X.A)is a parent set,uis a vector over the domain of par(X.A)andvis a value inDA. As specified above, any entry cpt_X.A(u)is encoded by the set of components(X.A,par(X.A),u, v), wherevis maximally preferred value inDA.

For a schema of domain size d, the number of components is bounded byat^pd^p+1, where |A| =a and |T| = t. By contrast, the size of Ntree[A,T] is bounded by (a+1)^a−1(t^p)^a(2^d−1)^adp, where (a+1)^a−1 is the number of spanning forests of the complete digraph of ordera. Again, this combinatorial barrier can be handled by exploiting the Matrix-Tree Theorem.

Lemma 5 For the classNtree[A,T], letθ be a parameter vector over the component set C(Ntree[A,T]). Then, generating a hypothesisN∈Ntree[A,T]at random according to Pθ can be done inO(a⁵+a²t^pd^p+ad^p+2)time.

Proof By analogy with the proof of Lemma 4, let w be the map that assigns to each N∈Ntree[A,T]the weightw(N )specified by equality (4). Again,wis by construction proportional toPθ.

Let X.A. be an attribute with parent set par(X.A), andu be a vector in the domain of par(X.A)= {X1.R1, A1, . . . , X_p, R_p.A_p}. We also assume without loss of generality thatDA= {1, . . . , d}. Let{C1, . . . , Cd}be the set of components of the formCi=(X.A, par(X.A),u, i), each associated with its weightwi=w(Ci). Byw[X.A,par(X.A),u], we denote the sum of weights of all bipartite orderings overDA. From equality (5), we get that

w

X.A,par(X.A),u

= −1+ d i=1

(1+w_i) (6)

By extension, letw[X.A,par(X.A)]be the sum of weights of all possible conditional pref- erence tables defined over the attributeX.Awith parent set par(X.A). Applying the dis- tributive property,

w

X.A,par(X.A)

=

u∈DA1×···×DAp

w

X.A,par(X.A),u

(7)

Now, let_Gbe the weighted digraph with node set_A∪ {}, whereis a “dummy” node denoting the absence of parent. The edge set is formed by all pairs of distinct attributes in A, together with all pairs of the form(X.A,)whereX.A∈A. Ifeis an edge of the form (X.A, X.A), thenw(e)is given by the sum of weightsw[X.A,par(X.A)]of all possible parent sets par(X.A)including one term inT with suffixX.Aand mostp−1 terms inT with uncontrollable suffix. Analogously, ifeis of the form(X.A,), thenw(e)is the sum of weightsw[X.A,par(X.A)]of all parent sets par(X.A)formed by at mostpterms inT with uncontrollable suffix.

Based on these specifications, the weighted digraphGcan be constructed inO(a²t^pd^p) time. Namely, there are 2_a

2

+aedges in the graph, and the weight of each edge is the sum of at mostt^p weights of the formw[X.A,par(X.A)], each being computed inO(d^p)time using equalities (6) and (7).

Finally, let span(G)denote the set of all spanning trees ofGrooted at. For any spanning treeSin span(_G), letw(S)be the product of weights of its edges. By construction,w(S)is equal to the sum of weightsw(N )of all RCP-netsN∈Ntree[A,T]with parent structureS.

Therefore,

Pθ(N )= w(N )

S∈span(_G)w(S)

With this property in hand, letU, V be two subsets of edges ofGsuch thatU⊆V, and letw[U, V]be the sum of weights of all spanning treesS∈span(G)for whichU⊆S⊆V. ByG[U, V]we denote the weighted digraph obtained fromGby deleting the edges which are not inV, and contracting the edges which are inU. By Lemma 5.4 in Kulkarni (1990),

w[U, V] =

e∈U

w(e) det Λ

G[U, V]

Thus, based on the Random Subset algorithm, we can generate at random a spanning treeS ofGrooted atinO(a⁵)time. This, together with the fact that any RCP-net overScan be

generated in_O(ad^p+2)time, yields the result.

5.4 Applications

We now have all ingredients in hand to examine the learnability of tree-structured RCP-nets.

For optimization tasks, each example supplied to the decision maker consists in a partial outcomex and a completiony ofx. Here,opt(C, x, y)indicates whetherC has made a prediction mistake on(x, y). Specifically, a componentCof the form(X.A,par(X.A),u, v) is charged one mistake on(x, y)if there is at least one objectoof typeXfor which the value ofo.Ais incorrectly predicted, i.e.Jpar(o.A)Ky=u,Jo.AKx= ∗, andJo.AKy=v. Based on this notion,opt(N;x, y)is equal to the number of componentsCinNwhich have made a mistake on(x, y). Thus,opt is an upper bound on the number of mistakes made by the decision maker. Together with the fact thatopt is bounded byλ=a, the composition of Lemmas2and5yields the following result.

Theorem 4 The classNtree[A,T]is efficiently online learnable from outcome optimiza- tion tasks with regret bound

a^3/2

ln(a+1)+plnt+d^p+1ln 2 T

For ranking tasks, each example consists in an outcome setx= {I1, . . . , Im}and a per- mutation y over[m] = {1, . . . , m}. Here, the lossrank(C, x, y)is recursively defined as follows. Form=2, any componentCof the form(X.A,par(X.A),u, v)is charged one mis- take on({I1, I2}, (2,1))if there is at least one objectoof typeXsuch thatJpar(o.A)KI₁= Jpar(o.A)KI₂ =u,Jo.AKI₁=v and Jo.AKI₂ =v. Now, form≥2,rank(C;x, y) is the number of pairs inxfor whichChas made a prediction mistake:

rank(C;x, y)=

i,j∈[m]:y(j )<y(i)

rank

C; {Ii, Ij}, (j, i)

Based on this definition,rank(N;x, y)is an upper bound on the Kendall’s tau distance (1938) that calculates the number of pairs inxfor which the permutationsN (x)andyhave opposite orderings (called discordant pairs). Sincerankis bounded byλ=a_m

2

, we get the following result.