Snapshots in a causal delivery system

Two global snapshot recording algorithms, namely, Acharya–Badrinath [1]

and Alagar–Venkatesan [2] assume that the underlying system supports causal message delivery. The causal message delivery propertyCOprovides a built-in message synchronization to control and computation messages. Conse-quently, snapshot algorithms for such systems are considerably simplified.

For example, these algorithms do not send control messages (i.e., markers) on every channel and are simpler than the snapshot algorithms for a FIFO system.

Several protocols exist for implementing causal ordering [5,6,26,28].

107 4.6 Snapshots in a causal delivery system

4.6.1 Process state recording

Both these algorithms use an identical principle to record the state of pro-cesses. An initiator process broadcasts a token, denoted as token, to every process including itself. Let the copy of the token received by processp_ibe denotedtoken_i. A processp_irecords its local snapshotLS_i when it receives token_i and sends the recorded snapshot to the initiator. The algorithm termi-nates when the initiator receives the snapshot recorded by each process.

These algorithms do not require each process to send markers on each channel, and the processes do not coordinate their local snapshot recordings with other processes. Nonetheless, for any two processes p_i and p_j, the following property (called propertyP1) is satisfied:

sendm_ij ∈LS_i ⇒recm_ij ∈LS_j

This is due to the causal ordering property of the underlying system as explained next. Let a messagem_ij be such that rectoken_i−→sendm_ij. Thensendtoken_j−→sendm_ijand the underlying causal ordering prop-erty ensures thatrectoken_j, at which instant processp_jrecordsLS_j, happens beforerecm_ij. Thus,m_ij, whose send is not recorded inLS_i, is not recorded as received inLS_j.

Methods of channel state recording are different in these two algorithms and are discussed next.

4.6.2 Channel state recording in Acharya–Badrinath algorithm

Each process p_i maintains arrays SENT_i1 N and RECD_i1 N.

SENT_ij is the number of messages sent by process p_i to process p_j and RECD_ij is the number of messages received by process p_i from process p_j. The arrays may not contribute to the storage complexity of the algorithm because the underlying causal ordering protocol may require these arrays to enforce causal ordering.

Channel states are recorded as follows: when a processp_irecords its local snapshotLS_ion the receipt oftoken_i, it includes arraysRECD_iandSENT_iin its local state before sending the snapshot to the initiator. When the algorithm terminates, the initiator determines the state of channels in the global snapshot being assembled as follows:

1. The state of each channel from the initiator to each process is empty.

2. The state of channel from processp_i to processp_j is the set of messages whose sequence numbers are given by RECD_ji+1 SENT_ij.

We will now show that the algorithm satisfies conditionsC1andC2.

Let a message m_ij be such that rectoken_i −→ sendm_ij. Clearly, sendtoken_j −→ sendm_ij and the sequence number of m_ij is greater than SENT_i[j]. Therefore, m_ij is not recorded in SC_ij. Thus,

send(m_ij)∈LS_i⇒m_ij∈SC_ij. This in conjunction with propertyP1implies that the algorithm satisfies conditionC2.

Consider a message m_ij which is the k^th message from process p_i to processp_jbeforep_itakes its snapshot. The two possibilities below imply that conditionC1is satisfied:

• Process p_j receives m_ij before taking its snapshot. In this case, m_ij is recorded inp_j’s snapshot.

• Otherwise,RECD_j[i]≤k≤SENT_i[j] and the messagem_ijwill be included in the state of channelC_ij.

This algorithm requires 2n messages and 2 time units for recording and assembling the snapshot, where one time unit is required for the delivery of a message. If the contents of messages in channels state are required, the algorithm requires 2nmessages and 2 time units additionally.

4.6.3 Channel state recording in Alagar–Venkatesan algorithm

A message is referred to asoldif the send of the message causally precedes the send of the token. Otherwise, the message is referred to asnew. Whether a message is new or old can be determined by examining the vector timestamp in the message, which is needed to enforce causal ordering among messages.

In the Alagar–Venkatesan algorithm [2], channel states are recorded as follows:

1. When a process receives the token, it takes its snapshot, initializes the state of all channels to empty, and returnsDonemessage to the initiator.

Now onwards, a process includes a message received on a channel in the channel state only if it is an old message.

2. After the initiator has receivedDonemessage from all processes, it broad-casts aTerminatemessage.

3. A process stops the snapshot algorithm after receiving a Terminate message.

An interesting observation is that a process receives all the old messages in its incoming channels before it receives the Terminate message. This is ensured by the underlying causal message delivery property.

The causal ordering property ensures that no new message is delivered to a process prior to the token and only old messages are recorded in the channel states. Thus, send(m_ij)∈LS_i⇒m_ij∈SC_ij. This together with property P1implies that condition C2is satisfied. Condition C1 is satisfied because each old message m_ij is delivered either before the token is delivered or before theTerminateis delivered to a process and thus gets recorded inLS_i orSC_ij, respectively.

A comparison of the salient features of the various snapshot recording algorithms discused is given in Table4.1.

109 4.7 Monitoring global state

Table 4.1 A comparison of snapshot algorithms.

Algorithms Features

Chandy–Lamport [7] Baseline algorithm. Requires FIFO channels. Oe messages to record snapshot andOdtime.

Spezialetti–Kearns [29] Improvements over [7]: supports concurrent initiators, efficient assembly and distribution of a snapshot. Assumes bidirectional channels.Oemessages to record,Orn² messages to assemble and distribute snapshot.

Venkatesan [32] Based on [7]. Selective sending of markers. Provides message-optimal incremental snapshots.n+u messages to record snapshot.

Helary [12] Based on [7]. Uses wave synchronization. Evaluates function over recorded global state. Adaptable to non-FIFO systems but requires inhibition.

Lai–Yang [18] Works for non-FIFO channels. Markers piggybacked on computation messages. Message history required to compute channel states.

Liet al.[20] Similar to [18]. Small message history needed as channel states are computed incrementally.

Mattern [23] Similar to [18]. No message history required. Termination detection (e.g., a message counter per channel) required to compute channel states.

Acharya–Badrinath [1] Requires causal delivery support. Centralized computation of channel states. Channel message contents need not be known. Requires 2nmessages, 2 time units.

Alagar-Venkatesan [2] Requires causal delivery support. Distributed computation of channel states. Requires 3nmessages, 3 time units, small messages.

n=# processes,u=# edges on which messages were sent after previous snapshot,e=# channels,d= diameter of the network,r=# concurrent initiators.