A Bidirectional Propagation Extension of DPOP

In DPOP, any UTIL message from an agent to its parent summarizes the utility information from all the subtree rooted at the respective agent. Therefore, the bottom-up UTIL propagation gives the root global utility information, but all other agents have accurateUTILinformation only about their subtrees.

Similar to BTE [107], we extend theUTILpropagation by making itbidirectional, in the sense that it traverses the DFS tree in both directions: not only bottom to top, as in DPOP, but also top to bottom, from each agent to its children. A message from a parent to its child summarizes the utility information from all the problem except the subtree of that child. This new message can be joined together with all

1Each data point is an average over 10 instances

10

Figure 4.3:DPOP vs ADOPT - evaluation on meeting scheduling problems.

the messages received by an agent from its children. The result is a summary of the utility information from the whole problem, which gives each agent a global view of the system, logically making each agent in the system equivalent to the root.

Notice that a similar effect can be obtained by running DPOPntimes, once with each variable as the root. However, this approach clearly would require spending more effort than the bidirectional utility propagation we propose here: roughly speaking,ntimes the effort spent by DPOP, vs. twice this effort.

The process is initiated by the root when it has received theUTILmessages from its children. Each agentXi(including the root) computes for each of its childrenXjaU T IL^j_imessage. To do so,Xifirst builds the join of the messages received from its other neighbors thanXj, plus therelationit shares withXj:J OIN_i^j=R^j_i⊕

c∈{Pi∪Ci\Xj}U T ILⁱc .

The set of dimensions of the joined message is always a superset of the dimensions that have to be passed down to the children. Subsequently, agentXiapplies a projection step to the outgoing message forXj, such that only therelevantdimensions are kept. This is done by projecting out in principle all dimensions not present inSepj, with two exceptions:

1. the dimension ofXjitself

2. the dimension of the sending agentXi,ifXihas a pseudochild in the subtree rooted atXj; this information is a byproduct of the DFS algorithm.

OnceXihas determined the relevant dimensions, it projects out everything else:

U T IL^j_i=J OIN_i^j⊥_{Xk∈{dim(JOINi}^j)\dim(U T IL^j_i)}

Example 9 (Bidirectional UTIL propagation) Let us consider the problem from Figure4.4(same DFS as in Figure4.1). As a result of the normal bottom-up UTIL propagation,X0receives theU T IL⁰₂ message from its childX2and can now compute its UTIL message forX1:J OIN₀¹=U T IL⁰₂⊕R¹₀. X₀has a pseudochild (X₄) in the subtree rooted atX₁, therefore it cannot project itself out of the UTIL message it sends toX1. Therefore,X0sends toX1U T IL¹₀=J OIN₀¹.

Subsequently,X1buildsJ OIN₁³=R³₁⊕U T IL¹₀⊕U T IL¹₄. AsU T IL¹₃previously received by X₁fromX₃does not containX₀as a dimension,X₁will projectX₀out of the UTIL message it will send toX3. Similarly toX0,X1also identifies a backedge to itself originating from the subtree rooted atX₃. Therefore, it cannot project itself out of the message forX₃:U T IL³₁=JOIN₁³⊥X0.

X1then prepares its message for its other child,X4: J OIN₁⁴ =R⁴₁⊕U T IL¹₀⊕U T IL¹₃. As U T IL¹₄previously received byX1fromX4does containX0as a dimension,X1will not projectX0

out of the UTIL message it will send toX₄. Furthermore,X₁does not have any backedge with any agent in the subtree rooted atX4, so it can project itself out. Thus,X1sendsX4U T IL⁴₁=J OIN₁⁴⊥X₁.

Figure 4.4: An example of a problem where bidirectional propagation is performed. Each arrow represents an UTIL message, and the numbers in brackets above represent the dimen-sions of the UTIL message. For example,U T IL¹₀has two dimensions:X₁andX₀(because of the backedgeR⁴₀,X₀cannot project itself out from the message going toX₁).

H-DPOP: compacting UTIL messages with consistency techniques

DPOP groups many valuations together in fewer (and also larger) messages, thus producing small communication overheads. However, the maximum message size is always exponential in the in-duced width of the constraint graph, leading to excessive memory and communication requirements for problems with large width.

Many real problems contain hard constraints that significantly reduce the space of feasible assign-ments. However, dynamic programming does not take advantage of the pruning power of these hard constraints; thus, DPOP sends messages that explicitly represent all value combinations, including many infeasible ones. Search algorithms mitigate this problem by various methods for pruning (partial assignments that have lead to an inconsistency are not further explored). Further pruning is achieved through consistency techniques, as well as the branch-and-bound principle.

This chapter brings two contributions: the first is H-DPOP, a hybrid algorithm that is based on DPOP. H-DPOP uses Constraint Decision Diagrams (CDDs, see [34]) to rule out infeasible combinations, and thus compactly represent UTIL messages. For highly constrained problems, CDDs prove to be extremely space-efficient when compared to the extensional representation used by DPOP: experimental results show space reductions of more than 99% for some instances. H-DPOP is an orthogonal technique, which can nicely complement other improvements to H-DPOP like MB-DPOP, LS-DPOP, A-DPOP, etc.

The second contribution of this chapter is a detailed comparison between search with caching [42, 32,132] and dynamic programming with CDDs. H-DPOP outperforms the search algorithm by a large margin on the number of messages exchanged while exploring a similar search space and thus is better suited for distributed environments.

In this chapter, we consider how to apply known hard constraints on feasible value combinations to prune such combinations, so that only information that actually corresponds to feasible solutions is transmitted. We do this by encoding value combinations usingconstrained decision diagrams(CDDs)

61

( [34]). CDDs eliminate all inconsistent values and only include costs or utilities for value combinations that reach a consistent leaf node. In experiments on several practical problems, we show that this cuts message size by up to 99%, putting problems of practical size within reach of H-DPOP.

A technique that explores hard constraints in a similar way is to cache partial results during the search [42], as implemented in the NCBB algorithm ( [33]). Similar to dynamic programming with CDDs, the caches contain only utility values for value combinations that are actually consistent. How-ever, the pruning carried out by CDDs is very different from that achieved by backtrack search: while backtrack search prunes all value combinations that are inconsistent with variables that arehigherin the ordering, CDDs do the pruning from the bottom up and prune value combinations that are inconsistent with variableslowerin the ordering.

To compare the pruning achieved by the two methods, we have modified the NCBB search algo-rithm ( [33]) to obtain another version that (a) maintains a complete cache and (b) does not use the branch-and-bound heuristic which we cannot reproduce in CDDs. We compare the space explored in dynamic programming with CDDs to that explored in backtrack search by comparing the size of the cache that has been used. We evaluate our CDD-based algorithm against different versions of NCBB, and show on several example domains that CDDs achieve essentially the same pruning achievable by search with the added advantage that only a linear number of messages are required. Thus, dynamic programming with CDDs achieves similar benefits but is more suitable for distributed settings.

The rest of this chapter is structured as follows: Section5.1presents an example problem which contains hard constraints, and introduces constraint decision diagrams (Section5.1.1). Section5.2 in-troduces the H-DPOP algorithm. Section5.3discusses search in general, and compares H-DPOP with the NCBB algorithm with caching from a theoretical point of view. Section5.4contains a comprehen-sive experimental evaluation of H-DPOP against DPOP and NCBB. Section5.5places H-DPOP in the context of existing work, and Section5.6concludes.

5.1 Preliminaries

Without loss of generality, hard constraints can be simulated using soft constraints by assigning utility

−∞to disallowed tuples, and utility 0 to allowed tuples. Then, simply using any utility maximization algorithm such as DPOP avoids infeasible assignments and finds the optimal solution. However, by doing so one does not take advantage of the pruning power of hard constraints. This drawback becomes severe for difficult problems (high induced width).

We introduce below one such real world problem and show the space reduction ability of hard constraints.

Optimal query placement: Recall from Section2.3.3the problem of optimally placing a set of query operators in an overlay network. Each user wants a set of services to be performed by servers in the network. Servers are able to perform services with distinct network and computational character-istics. Each server receives hosting requests from its users (together with the associated utilities). We model the resulting DCOP with servers as variables (agents) and the possible service combinations as the domains.

To avoid accounting the utility from the same service being placed simultaneously on two servers we introduce hard constraints between server pairs. These constraints disallow the same service to be executed by two servers at a time. Although this constraint is simple, it makes the problem highly constrained and computationally difficult.

Note that the above model may not be an exactly equivalent model for optimal query placement but it helps to make the problem tractable. The optimal solution may include running a service on more than one server but the problem would become much more complex in its originality.

Figure5.1(a)shows a DFS tree arrangement for servers in an overlay network. The services each server can execute are listed adjacent to nodes. During the utility propagation phase of DPOP nodeX4

will send a hypercube withX₁,X₂andX₃as context variables to its parentX₃(see figure5.1(b)).

However such a message scheme will send combinations which will never appear in a valid solution.

For example combinations likeX1=a, X₂=a, X₃=bwhich share a common service are infeasi-ble. The total size of this hypercube will be 64 (4³) with only 24 (4!) valid combinations. Eliminating these combinations using hard constraints can provide significant savings.

Consider an instance of a server problem with 9 variables (servers) with the same domain of size 9. The resulting network will be a chain with constraints between every server pair. The maximum size of hypercube in DPOP will be9⁹and the number of valid combinations will be only9!. So we are wasting 99.9% of the space in the message by sending irrelevant combinations. With the help of hard constraints we can prune such infeasible combinations and get extreme savings.