Synchronization protocols for sharing resources in grid
2.3 Mutual exclusion algorithms for large configurations
Several works found in the literature have proposed to adapt or compose existing mutual exclusion algorithms. By exploiting the logical or physical topological architectures of clusters (groups), they aim at reducing the number of messages or the waiting time delay to enter the CS. Basically, they adopt one of the two following approaches: in the first one [Bertier et al., 2006 ], [Mueller, 1998 ], the order of the pending requests of the waiting queue is changed in order to satisfy requests with higher priority first. Thus, priority of requests can dynamically change based on their locality which result in satisfying requests from the same cluster before those from distant ones. In the second approach, authors compose mutual exclusion algorithms and map them to the hierarchy of the architecture (physical or logical): one at the intra-cluster or intra-group level and a second one at the inter-intra-cluster or inter-group level.
2.3.1 Priority-based approach
Bertier et al. [Bertier et al., 2004 ] adapt Naimi-Tr´ehel’s algorithm by chang-ing the order of the pendchang-ing requests of theN EXT queue in order to satisfy intra-cluster requests before inter-cluster ones. Local cluster preemptions of the token take place since higher priority is given to requests of local clus-ter nodes. A threshold for the maximum number of per clusclus-ter preemption which characterizes the degree of locality of the cluster is defined for avoiding starvation. While the number of local requests is below such a threshold, theN EXT queue is modified in order to satisfy local requests first. Naimi-Tr´ehel’s algorithm was modified such that a request for entering a critical section (CS) follows theLAST tree until it reaches the last node of the clus-ter which has currently requested the critical section, denoted the local root.
Hence, if the N EXT of thelocal root exists, it points to a site of a remote cluster. In this case, if the local root receives a request from a node of its own cluster, and the number of preemptions is below the threshold, a local preemption of the token is performed and thelocal root’sN EXT is set to the local requester. The requesting node becomes the newlocal root. In [Loallemi et al., 2006 ], Moallemi et al. propose to apply Bertier et al. approach to the fault tolerant Naimi-Tr´ehel [Sopena et al., 2005 ] in order to provide a fault tolerant hierarchical token-based mutual exclusion algorithm. They consider that at least one node of the cluster where the token is present does not crash.
Broadcast was added between clusters for token and requests transmissions.
No performance evaluation experiments were conducted.
In Mueller [Mueller, 1998 ], the author presents an extension to Naimi-Tr´ehel’s algorithm where he adds the concept of priority in it. Local queues are introduced in each node. They form a virtual global queue ordered by priority
Synchronization protocols for sharing resources in grid environments 37 and FIFO order is used when priorities are equal. A request is associated with a priority and the algorithm first satisfies the requests with higher priority.
The basic idea is to accumulate priority information on intermediate nodes during request forwards. Thus, the N EXT queue is replaced by a set of local queues. When a node issues a request with same priority, this request propagates along theLAST tree until it reaches a node whose pending request is equal or greater than the former. The request is then locally queued at this node. When a node grants the token to another one it appends its local queue at the token. The receiving node then merges its token with the received one.
In the author’s approach, clustering is not considered. However, if priorities were assigned to requests based on their cluster locality, requests of the same cluster could be satisfied in sequence, preempting those of other clusters.
We should point out that the approach of changing the priority of pending requests could be applied to any other token-based algorithm. However, if such an approach allows to reduce the delay for a process to obtain the critical section, it does not reduce the complexity in terms of number of messages, which can become a drawback for large scale Grid systems.
2.3.2 Composition-based approach
Several authors [Chang et al., 1990a ], [Housni and Tr´ehel, 2001 ], [Bertier et al., 2006 ], [Erciyes, 2004 ], [Madhuram and Kumar, 1994 ], [Omara and Nabil, 2002 ] propose to compose mutual exclusion algorithms: one at inter cluster/group level and a second at intra cluster/group level in order to mini-mize both message traffic and the time delay to enter the CS. In the majority of these works, the algorithm between clusters (groups) is different from the algorithm inside a group.
Chang et al. [Chang et al., 1990a ] present in their article a hybrid ap-proach which applies diffusion-based algorithms at both levels: Singhal’s al-gorithm [Singhal, 1992 ] locally and Maekawa’s alal-gorithm [Maekawa, 1985 ] between groups. The former uses a dynamic information structure while the latter is based on a voting approach. The authors argue that since no algo-rithm can minimize both message traffic and time delay at the same time, such a combination is ideal since message complexity of Maekawa algorithm is low and Singhal’s algortihm presents shorter delay time in successive execu-tions of CS when compared to the former. Simulation studies show that when compared to flat Maekawa’s algorithm, the proposed hybrid algorithm signifi-cantly reduces message traffic and time delay, specially if the system presents cluster locality of requests. Similarly, Omara et al.’s solution [Omara and Nabil, 2002 ] is a hybrid of Maekawa’s algorithm at inter cluster level and Singhal’s modified algorithm, which provides more fairness than the latter, at intra cluster level.
In Housni et al. [Housni and Tr´ehel, 2001 ], sites with the same priority are gathered within the same group. The algorithm between groups is different from the algorithm inside a group. They consider that routers ensure the
in-38 Fundamentals of Grid Computing
terface between the algorithms of the two levels. Raymond’s tree-based token algorithm [Raymond, 1989 ] is used inside a group, while Ricart-Agrawala [Ri-cart and Agrawala, 1981 ] diffusion-based algorithm is used between groups.
If a router receives a permission from all the other routers, it generates the token in its group and gives it to the root node of its group. When all the pending requests of the group are satisfied, the token is restored to the router and destroyed. Routers can have different priorities. Hence, nodes of a given group, whose router has generated the token, can be preempted if its router receives a request from another one with higher priority. The former then interrupts the root of its group for requesting it the token. As soon as it gets it, the router sends its permission to the priority router.
Erciyes [Erciyes, 2004 ] proposes an architecture that consists of a logical ring of clusters, i.e., each node on the ring represents a cluster of nodes. Each of this node is a coordinator node, which performs the required critical section requests and releases on behalf of the nodes of the cluster it represents. How-ever, inside the cluster, a central-based mutual exclusion algorithm is used, i.e., a node requests to its coordinator to enter the CS and will enter it upon reception of a reply message from the coordinator. When exiting the CS, the node sends a release message to the coordinator. The author proposes two al-gorithms for the inter-cluster level: the permission-based Ricart-Agrawala [Ri-cart and Agrawala, 1981 ] algorithm and Lan’s token-based algorithm [Lann, 1978 ]. However, in both cases, messages are exchanged based on the inter cluster logical ring structure. Theoretical studies of the composed algorithms compared to their flat original algorithms show: (1) a gain of an order of mag-nitude of improvement in terms of message complexity but at the expense of larger obtaining time; (2) a reduced number of messages to enter a CS when the number of clusters increases. However, with a large number of clusters and number of messages (low level parallel application) coordinators become a bottleneck and token obtaining time increases.
Similarly to Erciyes [Erciyes, 2004 ], Madhuram et al. present in [Madhuram and Kumar, 1994 ] a two level algorithm where the centralized-based mutual exclusion algorithm is used at intra cluster level. However, coordinators at inter cluster level are not organized in a logical ring, but every coordinator can communicate with the other. The authors argue that the centralized algorithm is a good choice because of its low message complexity: 3 messages per CS invocation. All message requests are timestamped. The coordinator queues up the requests it receives from processes of its cluster in a queue in timestamped order (InternalQ). A second queue is used to store requests from the other coordinators (GlobalQ). A coordinator acts on behalf of all sites of its group. Upon receiving a request, the coordinator will broadcast a request provided it did not already do it (i.e., the request is registered in theGlobalQ).
Using theInternalQ, the authors also propose a local preemption mechanism in order to satisfy local requests (imposing an upper limit to avoid starvation) before giving permission to a remote coordinator. They argue that in case of high load (low parallel application) such preemption mechanism increases
Synchronization protocols for sharing resources in grid environments 39 performance of the application. The hierarchical algorithm presented in the paper adapts Ricart-Agrawala at inter-cluster level, but the authors advocate that any timestamp-based algorithms could also be used as well.
In [Bertier et al., 2006 ], Bertier et al. propose a composition approach which consists of Naimi-Tr´ehel algorithm at both level. The authors consider that messages exchanged by nodes of different clusters always pass along spe-cial nodes, called proxys, that behave like routers. There is one proxy per cluster. Therefore, every message that is sent from a node to a remote node is routed to the proxy of the cluster’s sender which forwards the message to receiver’s proxy node. The latter then sends the message to the receiver node.
The sender’s and receiver’s proxies can then gather information about token transfers and requests at cluster level, taking decisions based on such informa-tion. Therefore, some inter-cluster messages can be managed at the proxies’
level, without being necessary to be redirected to the other nodes inside clus-ters. Furthermore, by changing theN EXT queue, intra-cluster CS requests are satisfied before remote ones. To avoid starvation, a threshold value limits the maximum number of CS that can be successively executed by nodes of the same cluster. Experiments were performed on a dedicated cluster where a Grid environment with multilevel network latencies was emulated by injec-tion of network delays. The authors show that due to the proxy and cluster locality exploitation, both the time delay to obtain the token and the number of inter-cluster messages were reduced on their approach when compared to flat Naimi-Tr´ehel’s algorithm, specially for low parallel applications.
All the above mentioned works hierarchically compose mutual exclusion al-gorithms but the choice for the composed alal-gorithms can not change. One could argue that one of these fixed compositions could be more suitable for some type of application. However, no comparative evaluation performance study exists in the literature for such a conclusion. Another point is that the concept of group that the majority of these works exploit is logical, i.e., groups are not necessarily mapped to the physical topology and do not con-sider latency heterogeneity. Hence, whenever some performance results are presented in their respective articles, the experiments were conducted on top of simulators or emulated platform and not on top of a Grid.