Asynchronous search algorithms

The vast majority of algorithms developed so far for DisCSP/DCOP are asynchronous algorithms.

Asynchrony is appealing for distributed algorithms for a number of reasons. First, asynchrony can offer in principle a better distribution of the computation between the agents involved (all agents can execute in parallel, and do not necessarily have to wait for messages from their peers). Second, asynchronous algorithms are in principle less sensitive to message delays and message loss, as agents execute even without necessarily having received the most up to date messages from their peers.

We start this section with a short review of asynchronous algorithms for distributed constraint satisfaction. Next, we move to algorithms for distributed constraintoptimizationand describe ADOPT, NCBB and AFB.

3.1.2.1 Asynchronous search for DisCSP

This section describes existing asynchronous approaches for Distributed ConstraintSatisfaction Prob-lems. This is by far the area which has received the most attention since the beginnings of the dis-tributed constrain reasoning field, in the early nineties. Undoubtedly, the most influential piece of work is Yokoo’s Asynchronous Backtracking (ABT) algorithm, which represented the basis for many subsequent developments. We describe this algorithm in the following.

Asynchronous Backtracking (ABT) Asynchronous Backtracking (ABT [225]) is the first asyn-chronous algorithm that has been proposed for DisCSP. ABT laid the foundations of DisCSP, being the first algorithm to allow agents to execute concurrently and asynchronously.

In ABT, agents are ordered linearly. They assign values to their variables concurrently and asyn-chronously, and announce the assignments to their lower-priority neighbors viaok?messages. When

an agent receives anok?message, it updates itsagent view⁵and tries to find a compatible value for its variable. If it can, it announces this to lower priority agents with anok? message, otherwise, it backtracks with anogoodmessage. When receiving anogoodmessage, an agent tries to find an-other value for its value, compatible with its own agent view. If it cannot, it backtracks with anogood, and so on. The algorithm terminates if an empty nogood is discovered (the problem has no solution), or if quiescence is reached, in case a solution is found. Note that detecting that a solution was found requires an additional termination detection algorithm, which may introduce some overhead.

ABT is sound and complete, and its complexity is polynomial amount of memory, and exponential number of messages in the worst case. ABT has been extensively studied since its original publication by Yokoo in ’92 [225], and much of the later work in DisCSP is based on it.

Asynchronous Weak Commitment (AWC): Asynchronous Weak Commitment (AWC [224]) is an alternative to ABT which was proposed in order to simulate the dynamic variable ordering heuristics from the centralized case, which have been shown to offer important performance improvements in some cases. Specifically, whenever an agent initiates a backtrack, it takes the first position in the ordering. This step is designed to refocus the algorithm on the newly discovered difficult part of the search space. AWC is shown to be more efficient than ABT on difficult problems [224,86]. However, in this case, AWC must store all nogoods discovered during search to guarantee completeness, which makes it space-exponential in the size of the problem in the worst case. On a side note, Yokoo and Hirayama [228] introduce a modification of AWC which deals with complex local problems, i.e. an agent owns several variables as opposed to a single one.

Dynamic Variable Reordering: In order to allow distributed search to benefit from dynamic vari-able ordering heuristics like AWC, but without AWC’s exponential space problems, varivari-able reorder-ing techniques have been developed for the ABT algorithm in [199,202,193], and then also in [242].

These techniques work by allowing only for a limited type of reordering, namely each agent can im-pose new orderings for agents below itself in the ordering, and inform these lower priority agents of the new ordering. Upon being announced of a change in the ordering, a lower priority agent updates itsagent view, and discards obsolete nogoods. A more advanced reordering protocol is introduced by Silaghi et al. [201]. This protocol allows for general reorderings, thus being able to simulate AWC with polynomial space requirements.

Asynchronous Aggregations in Search (AAS): AAS (asynchronous aggregations in search) [197] is an algorithm that operates on the dual model of the CSP, i.e. where agents own and control the constraints, not the variables, which are public. The domains are now tuples of assignments of variables from the original CSP formulation, and can be large. AAS exploits the fact that in cases

5A data structure holding the agent’s view of the current assignment of agents of higher priority

where variables have large domains, it could happen that several values in the domain behave similarly with respect to constraints. Thus, it can be beneficial to group several values in equivalent sets, and perform ABT on such a modified problem, and managing the grouping into sets dynamically during search.

Asynchronous Consistency Techniques: Consistency techniques have been shown to be very effective in centralized CSP, and have been also implemented in distributed settings [95,138,26,139, 200,195]. Asynchronous Forward-Checking introduced by Meisels and Zivan [139] works by hav-ing agents perform backtrackhav-ing sequentially, and announchav-ing their assignments in parallel to all other agents lower in the ordering, which perform forward checking in parallel. Hamadi proposes a dis-tributed arc consistency algorithm in [95]. Silaghi introduces MHDC [195], an algorithm which main-tains arc consistency during search in AAS, which is shown to improve AAS’s performance signifi-cantly.

Concurrent Search: Multiple search processes operating concurrently and exchanging informa-tion have also been investigated. The idea is to launch parallel search processes that explore different parts of the search space, and let them communicate relevant nogoods between themselves, such that they avoid exploring the same dead ends. [241,176] report promising results.

The vast majority of algorithms that do not operate on a DFS require communication between non-neighbors. This is also the case of ABT (requested viaadd-linkmessages) and derivatives, AWC, DisFC, ConcBT, etc. All these algorithms violate our assumption from Section2.2. An extension to ABT proposed by Bessi`ere and colleagues [22] eliminates the need to add links, but incurs a perfor-mance hit for doing so.

Distributed Local Search: A local search method called the Distributed Breakout Algorithm [227]

has also been developed. DBA is not complete, and works only for satisfaction problems, but in some cases it can find solutions very fast, and it also exhibits anytime behaviour for overconstrained prob-lems. In DBA, agents execute a hillclimbing algorithm in parallel, and try to escape fromquasi-local minima⁶using the breakout method [143]. In DBA, agents initially choose arbitrary values for their variables, and announce their choices to their neighbors withok?messages. Subsequently, when re-ceivingok? messages, each agent evaluates the number of conflicts its current assignment produces with the assignments of its neighbors. The agent (internally) evaluates what reduction in the number of conflicts it could make by changing its value, and advertises this possible improvement to its neighbors with animprovemessage. Neighboring agents thus exchangeimprovemessages, and the one with the highest improvement wins and actually changes its value (ties are broken according to agent ID).

The cycle then repeats, withok?andimprovemessages. In case a solution is found, the algorithm

6In [143], the breakout method is used to escape fromgloballocal minima, but in a distributed setting it is difficult for the agents as a group to realize they are stuck in a global minima; thus quasi-local-minima is used as a loose, cheaper alternative.

reaches quiescence (a termination detection is provided). If a solution is not found, or none exists, DBA cycles forever.

As with ABT, DBA has been the object of many subsequent improvements [236,238,155,157,12].

An improvement to DBA appears in [155] which usesinterchangeabilities[77] to try to contain con-flicts, and keep them localized. This works by using neighborhood interchangeability and neighbor-hood partial interchangeability to select new values for the variables that already are in conflict with other variables, such that we do not risk creating new conflicts by switching to the new values. Ex-perimental results show that the new algorithms are able to solve more problems, and with less effort, especially for difficult problems, close to the phase transition. Another improvement of DBA consist-ing in a value-orderconsist-ing heuristic appears in [157]. This heuristic is developed in the context of resource allocation problems (e.g. sensor networks), and it works by trying to allocate the least contended re-sources first. This tends to produce good allocations from the beginning of the execution of DBA, and thus requires less subsequent effort. Another extension to DBA that identifies hard subproblems and solves them with a complete search algorithm,thus guaranteeing completenesshas been proposed in [62].

Alternatively, distributed stochastic search algorithms have been proposed [75,235,6].

In the next sections we focus on algorithms that were specifically designed for DCOP.

3.1.2.2 ADOPT

ADOPT by Modi et al. ( [141]) is a backtracking based bound propagation mechanism. ADOPT was the first decentralized algorithm to guarantee optimality, while at the same time allowing the agents to operate asynchronously.

The algorithm works as follows: first, the DFS structure is created. Then, backtrack search is performed top-down, using the following value ordering heuristic: at each point in time, each agent chooses the value with the smallest lower bound. It announces its descendants of its choice viaVALUE messages, and waits forCOST messages to come back from the children (please refer to Figure3.1 for a diagram that shows the message flow in ADOPT). Each agent adds the costs received from its children to the lower bound of the current value taken by the agent. If there is another value in the domain that has a smaller lower bound, the agent switches its value, and the process repeats, refining the lower bounds iteratively.

One of the innovative ideas behind ADOPT is that it achieves asynchrony by allowing the agents to change their variable values whenever they detect thepossibilitythat some other values are better than the current ones (i.e. they have smallerlower bounds). Notice that this does not mean that the new values are guaranteed to be better. This strategy allows for asynchronous operation since the agents do not have to wait for achieving global information aboutupper boundson cost to take their local

Figure 3.1: ADOPT: (a) a simple problem graph, arranged as a DFS tree; (b) diagram showing the flow of messages: VALUE assignments are sent by agents to all their descendants in the tree; children respond to their parents with COST messages; parents send their children THRESHOLD messages.

decisions, as would normally happen in classical branch and bound.

However, abandoning partial solutions before proving their suboptimality makes it sometimes nec-essary to revisit several times some of the previously explored partial solutions. One solution to this problem would be to store all these partial results, and retrieve them later on, without any more search effort. The drawback of this approach is that the amount of memory required to do so is exponential in the width of the DFS ordering chosen. ADOPT tries to mitigate this problem by using abacktrack thresholdwhich is an allowance on solution cost intended to reduce the need for backtracking, while maintaining a low memory profile (polynomial).

3.1.2.3 Non-Commitment Branch and Bound

Chechetka and Sycara propose in [33] another DCOP algorithm that operates on a DFS: NCBB (Non-Commitment Branch and Bound). This algorithm is a variant of AOBB, with the important difference that NCBB includes a parallelization technique where an agent advertises different values of itself to different children at the same time. This parallelization technique ensures that all the subtrees of any agent are working in non-intersecting parts of the search space and we do not need to worry about the solution costs between them.

Similar to dAOBB with i-bounded caching (Section3.1.1.3), NCBB was also extended with a caching mechanism in [32].

3.1.2.4 Asynchronous Forward Bounding (AFB)

AFB [81] is also based on branch and bound, and works on a linear ordering of the variables. AFB is similar to SynchBB: agents assign their variables and generate a partial solution sequentially and

synchronously. As in classic B&B, agents extend a partial solution as long as the lower bound on its cost does not exceed the global bound, which is the cost of the best solution found so far. The current partial assignment is propagated together with the cost of the best solution found so far. Each agent which receives the CPA, extends it with its local assignment, if an assignment with a lower bound smaller than the current global upper bound can be found. Otherwise, it backtracks by sending the CPA to a former agent to revise its assignment. An agent that succeeds to extend the assignment on the CPA sends forward copies of the updated CPA, requesting all unassigned agents to compute lower bound estimations on the cost of the partial assignment. The assigning agent will receive these estimations asynchronously over time and use them to update the lower bound of the CPA. Gathering updated lower bounds from future assigning agents, may enable an agent to discover that the lower bound of the CPA it sent forward is higher than the current upper bound (i.e. inconsistent). This discovery triggers the creation of a new CPA which is a copy of the CP A it sent forward. The agent resumes the search by trying to replace its inconsistent assignment. The authors provide an experimental evaluation of AFB against SynchBB, and show that it performs better.