The previous section presented the benefits of scheduled routing at a high level. This sec-tion examines the basic mechanisms of scheduled routing in sufficient detail to understand the key ideas of the thesis. Chapter 7.1 discusses the specifics of the NuMesh implementation of scheduled routing in somewhat more detail.
1.4.1 Scheduling Virtual Finite State Machines
The fundamental unit of scheduled routing is the virtual finite state machine, or VFSM. A VFSM handles moving a particular stream of data through a particular node. It includes some buffering for that particular stream of data as well as annotations (called the VFSM’s source and destination) that indicate where its data come from and go to. Figure 1-5 shows a simple VFSM for a stream of data moving from node one through node two’s router to node two’s processor. Linking together all the individual VFSM sources and destinations on each node yields an overall VFSM chain that moves the data from a stream’s original source to its final destination; in the given example, the stream’s source might be node zero, and its destination is node two.
If it was only necessary to move one stream of data through each node, a single VFSM could be used as a router. However, it is much more common to need to move several unrelated
VFSM VFSM
Processor
data
data buffer
Node 1 Node 2
Figure 1-5: A virtual state machine transferring data
streams of data through each processor. This can be accomplished by scheduling the VFSMs in some predefined sequence on a node, essentially time-slicing the router hardware into multiple virtual finite state machines (thus the ‘V’ in ‘VFSM’).
However, it is necessary to be able to differentiate two streams of data that just happen to pass between the same two nodes at some point. To do so, the VFSMs are scheduled globally, so that when a VFSM on one node does a write at a particular time, the VFSM that reads the same stream of data on the neighboring node is scheduled one clock cycle later. Figure 1-6 shows how VFSMs might be scheduled on three nodes (in a one-dimensional array), carrying three streams of data among them. Stream A, with bandwidth 0.5, carries data from nodes 0 to 2; streams B and C, with bandwidth 0.25, carry data from 0 to 1 and from 1 to 2, respectively.
Each word of data moves through the mesh at one cycle per node; the number of times a given VFSM appears in a schedule corresponds to its allotted bandwidth.
A B
C A B
C A
time
ANode 0 Node 1 Node 2
A
A
Figure 1-6: VFSM schedules on a small mesh
Another representation of how scheduled routing is shown in Figure 1-7, which demon-strates which cycles data is carried across the wires in the mesh; router nodes are shown as circles, with their attached processors in boxes. In the example, a data word might start on node 0 on the VFSM for stream A when it’s scheduled at time 0, get passed to node 1 at time 1, then passed to node 2 at time 2, where the last VFSM delivers data to the processor. A similar word leaving node 0 at time 1 must be associated with stream B, and will therefore be delivered by the VFSM for stream B on node 2 to the processor, instead of forwarded to node 3 as is true for stream A. Streams of data in a scheduled routing environment are thus bound tightly to a specific set of VFSMs on the nodes chosen to route them from their source to destination.
The nodes in the mesh are synchronized at boot time, and remain synchronized from then on. Each node runs its schedule repetitively, returning to the top of the schedule after running
N0 N1 N2
Figure 1-7: Data movement on a small mesh
the VFSMs scheduled last in the schedule. Thus, if the schedule is of length
k
, the 3rd timeslot in the schedule will be run at time3;
3+k;
3+2k
, and so on.The discussion so far has assumed that the physical hardware is capable of only a single communication action per cycle. Multiple physical FSMs can be included on each node that are each independently timesliced by VFSMs. These physical FSMs are called pipelines, using terminology that reflects the implementation of the current NuMesh hardware. Each pipeline is capable of scheduling an independent VFSM for each cycle. Thus, on a given cycle there may be a simultaneous transfer by one pipeline of data from the+
x
to the,x
directions, while another pipeline is transferring data from the processor to the +y
direction. Co-scheduled VFSMs must have non-conflicting sources and destinations to avoid conflicts.1.4.2 Flow Control in Scheduled Routing
So far, it has implicitly been assumed that whenever data is ready to be transferred it can always be accepted by the destination. However, if the stream’s destination processor is busy, and the source processor is attempting to write data into the mesh, the system must handle the potential for a backlog in the stream. This problem (known as the flow control problem) is handled by a synchronized exchange of control signals along with the data.
Whenever a valid data word can be written to a neighbor, the VFSM asserts the valid line to the remote reader and sends the data. When reading, a VFSM determines whether it has sufficient buffer space to accept another word of data, setting the accept line to the remote writer accordingly. After setting the accept line low, the reader will simply ignore any data
presented to it.
The difference from traditional online-routing schemes is that the accept line is driven by the reader at the same time as the valid line. Since the mesh is running schedules that are all offline-generated, a reader always knows when a writer is sending data, and it can generate an
‘accept’ during the same cycle as the data is written. This means that scheduled routing allows a single-cycle protocol for determining flow control.
When the writer VFSM discovers that the reader couldn’t accept the data word, it queues the word in its buffer. The next time the writer is scheduled, it will set the accept line low for its writer, and attempt to transfer the buffered word instead of reading a new one. Figure 1-8 shows one VFSM with its incoming and outgoing control and data lines, and summarizes what actions it takes depending on the state of those lines. These actions happen in synchrony across the entire mesh every cycle; this choreography—scheduling which VFSMs run on each node at each cycle—determines how data moves through the mesh.
valid data accept valid
data
accept VFSM
Transfer Result Action
read and write succeed transfer success read success, write fail word buffered
read fail node transfers buffer word if valid Figure 1-8: NuMesh transfer actions
1.4.3 Router Reprogramming
The router may be updated under program control. Each scheduled router is assumed to have a schedule memory which can hold multiple schedules (of which only one is active). Each location in the router in which a schedule can be placed has some index associated with it.
New schedules can be downloaded to the router by specifying the desired index and providing the schedule information. COP requires a router schedule memory large enough to hold at least two complete schedules, so it can load a new schedule into the router while the previous one continues to run.
Similarly, each router is assumed to have multiple VFSMs, of which only some are used in a given schedule. VFSMs can be used for the same operator in multiple schedules (as is discussed in Section 5.1); however, VFSMs can also be reused by changing the source and destination annotations associated with the VFSM, then using the VFSM in a new schedule.
An index is associated with each VFSM that allows it to be referenced both for scheduling and for reprogramming its annotations.
1.4.4 Other Characteristics of Scheduled Routing
Processor Interface. At each end of a stream of data, the router must deliver data to and from the processor itself. The interface between the two can be specified as just another
flow-controlled source or destination for the VFSMs. The architecture could require that the pro-cessor be synchronized to the mesh sufficiently to know that a data word in timeslot 5 belongs to a certain stream. However, to allow for a more general, asynchronous interface, multiple interface addresses are provided. The router can deliver data from stream A consistently to interface address 12; the processor can then read interface address 12 at its leisure to find data from stream A.
Scheduling Delays. Streams can also be delayed one or more cycles on a node. Normally, a VFSM is scheduled on a given cycle, reads the data from its specified source, then writes the data to the appropriate VFSM on the next node on the following cycle. However, global scheduling constraints may make it infeasible to schedule a stream on consecutive cycles in each node in its path. Accordingly, it may be necessary to delay a few cycles to find a suitable path. This can be done by allocating two VFSMs on a node to a stream. One reads from the neighbor node, then hands off to the other VFSM; the second VFSM, scheduled later, reads from the first one and then writes the data word to the destination node.
Data Forking. A data ‘fork’ allows a stream of data to be forked to two (or more) destina-tions within the router. This feature can be used to arrange a multicast stream that has multiple destinations. A data fork can be implemented by consecutively scheduling a set of VFSMs, the first of which is responsible for reading the data from the neighboring node, and each of which handles one of the destinations. Multiple data forks can be placed in a stream; this creates a distribution tree that can send the data to the destinations in an optimal manner.