Transparent Access

The key to two-case delivery is that the direct and buffered modes of operation must appear identical to user software. This principle of transparent access must apply to both the messaging and atomicity primitives of the UDM model. Given transparent access, the runtime system is free to switch to and from buffered mode at any time. As a consequence, the buffered mode provides a unified means of dealing with all exceptional circumstances that prevent a user-level application from proceeding immediately.

Transparency Mechanisms. Transparency of the messaging primitives is achieved through a com-bination of hardware mechanisms and software conventions. First,injectoperations are always directed at hardware queues. As a consequence, only the extractand atomicity manipulation operations must be virtualized.

As discussed in Section 4.1, anextractoperation is decomposed into memory-mapped reads from the network interface, followed by adispose instruction. Access to receive data is made transparent by employing a software convention of using a known base register to point to the input message buffer. In the fast case, this register points at the hardware queue. When delivery must be shifted from fast to buffered mode, the base register is altered to point to the buffered copy of the message (if any) in main memory. Thedisposeinstruction is made transparent by causing it to be trapped and emulated whenever the the user is in buffered mode.

Atomicity control is made transparent by switching seamlessly between modes. In the fast mode, the atomicity bits control message interrupts directly. In the buffered mode (divert-mode set), all interrupts are diverted to the operating system and atomicity is emulated by manipulating user threads as described in Section 4.2.

Mode Transition. Transitions to buffered mode take place when the user cannot or will not make forward progress. InFUGU, there are three reasons to switch the active task from fast to buffered mode (these are demanded for protection and context-switching): page faults in the handler, atomicity timeouts, and scheduler quantum expirations. All three of these events are “soft” in that they merely cause a transparent switch to buffered mode. The user observes an increase in the cost of messaging, but no change in program semantics.

Transparency is important at the beginning of a scheduler quantum, since it allows the scheduler to start a user thread in buffered mode, letting the thread process messages that were received while other threads were scheduled. When the buffered messages have been exhausted, transparency can again be invoked to switch back to the fast mode of reception.

4.4 Discussion

The two-case delivery technique raises a number of research questions. The major question is whether two-case delivery can provide a performance benefit over a system that simply always buffers messages. We have shown in this chapter (Table 4-4) that the implementation of two-case delivery inFUGU achieves low-level message overheads near those of unprotected hardware. The primary experiment in Chapter 7 will show that fast-case performance is achievable over a useful range of mixed, multiprogrammed workloads expected in a scalable workstation.

Two-case delivery raises several other miscellaneous but interesting questions that are not further pursued in this thesis. Three of these questions are discussed briefly below: whether to use upcalls or to buffer on mismatched messages, how the message handling timeout should be tuned for performance and how the two-case delivery buffering system might be integrated with an application-specific buffering system. Finally, some of the design decisions inFUGUmight be somewhat different if the relative costs of operations were different.

Upcall on Mismatch. FUGUoffers both upcall-based and buffer-based modes of delivery of messages that arrive for the wrong application. It is unclear which mode, if either, is better in general. Upcalls give the lowest overhead and certainly the lowest latency for the application receiving the upcall.

Low latency may be crucial for user-level shared memory implementations. On the other hand, buffering probably gives the least disruption to the application that happens to be running when the mismatched message arrives. The buffer insertion handler is known to be fast and to have a limited cache and TLB footprint.

Handler Timeout. The timeout on handler/atomic section execution time is a free parameter that the operating system can adjust as required for performance. Since the difference between direct and buffered modes is transparent to the application, the operating system can cause a switch to buffered mode at any time. There might be different timeouts for each application or different timeouts for same-domain and cross-domain upcalls. Changing the timeout is a scheduling issue: long timeouts are presumably good for the application that is receiving messages. Short timeouts are good for the rest of the system because the short timeout will limit the amount of congestion in the network or the amount of time taken away from an interrupted application.

Further, the timeout might be useful as a means of introducing buffer to benefit even a single application. Mukherjee notes in [57] that judicious use of buffering can improve the performance of some applications running standalone by reducing sender stall time and network congestion. The timeout in message handlers in FUGU could be used as a heuristic for detecting opportunities to improve performance through buffering. Other cues such as whether the input hardware FIFO is full or hybrid heuristics are possible. The ramifications of the timeout are not examined in this thesis.

Buffering Systems. Programmers using active messages apparently often build ad-hoc buffering systems. It is not clear the extent to which two-case delivery can obviate the need for ad-hoc solutions. Alternatively, if the application benefits from some kind of specialized buffering system, it might be useful to merge the system’s two-case delivery with the applications. The exokernel-based implementation of buffering inFUGUis amenable to such specialization but we have not tried to take advantage of it.

As a separate buffering issue, the software buffer cleanup thread currently executes the handler

for every buffered message before returning to direct delivery mode. This policy allows us to preserve the ordering of messages in the network. However, this policy can also cause the system to remain in buffered mode if the send rate exceeds the receive rate even if the receiver is making progress. An alternative policy would be to forgo ordering and process direct and buffered messages alternately, using the atomicity timeout to limit the time spend in the handlers of buffered messages.¹

Relative Costs. The buffer enqueue handler overhead is important to system performance, as will be shown in detail in Chapter 7. The minimum buffer enqueue handler overhead of 180 cycles listed in Table 4-5 includes a user interrupt time. In another implementation of two-case delivery, the relative costs of operations could be quite different. Here are the basic tradeoffs which we will revisit in Chapter 7:

Cross-domain upcalls inFUGUare only slightly more expensive than ordinary user interrupt upcalls because of a tagged TLB and the fact that Sparcle’s multiple register sets can be protected from one another. More expensive domain crossings would make buffering on mismatches significantly more attractive.

The buffer enqueue handler time could be improved by tail-polling in the interrupt handler, or by downloading into the kernel the most common case of the buffer enqueue code. If a small amount of extra hardware were to be invested inFUGU, a good use would be a limited automatic DMA function to accelerate the most common case of buffer enqueue.

The thread scheduling overheads inFUGUare abysmal, as revealed in the cost to receive an isolated message via the buffered path (Table 4-5). Support for threads could be made as fast as 10s of cycles (as in the “featherweight” threads proposed in [39]). With fast threads, the cost a single message handled on the buffered path would be dominated by the cost of servicing the inevitable cache miss.

Applicability. Two-case delivery is the primary architectural technique that gives the direct VNI performance given the UDM model and the requirements for protection. Two-case delivery is usable in any system with a direct interface. However, two-case delivery is particularly attractive if the system can also guaranteed delivery. Guaranteed delivery allows the fast path to avoid all buffer management overhead (pointer manipulation and acknowledgment messages). Guaranteed delivery requires a reliable network and unlimited buffering. While some systems have provided effectively unlimited buffering by providing large amounts of physical buffering (e.g., the SP-2 [71]), pinning down physical memory is inconvenient in general, particularly in a multiprogrammed system where it is desirable to be able to guarantee some minimum amount of buffering to each application.

This chapter has described the first architectural technique, two-case delivery, used to support the direct VNI. The next chapter describes the second architectural technique, virtual buffering, that maintains the illusion of unlimited buffer space while requiring only limited amounts of physical memory in practice.

1As yet another separate buffering issue, we could buffer at the sender as well at the receiver with nearly the same hardware. The required addition would be a sender-side divert mode bit which would causelaunchinstructions to trap.

The policy decisions of when to turn sender-side buffering on and off are unclear.

Chapter 5