HAL Id: jpa-00225843
https://hal.archives-ouvertes.fr/jpa-00225843
Submitted on 1 Jan 1986
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
POSSIBLE USE OF OCCAM AND TRANSPUTERS IN P.S.D. DATA TREATMENT
Shelby Wilson, R. Stansfield
To cite this version:
Shelby Wilson, R. Stansfield. POSSIBLE USE OF OCCAM AND TRANSPUTERS IN P.S.D. DATA TREATMENT. Journal de Physique Colloques, 1986, 47 (C5), pp.C5-193-C5-197.
�10.1051/jphyscol:1986526�. �jpa-00225843�
POSSIBLE USE OF OCCAM AND TRANSPUTERS IN P.S,D. DATA TREATMENT
S.A. W I L S O N * and R.F.D. STANSFIELD"
' ~ n s t i t u t Laue-Langevin, 156X, F-38042 Grenoble Cedex, France
" ~ e p a r t m e n t of Biochemistry, University of Edinburgh, GB-Edinburgh EH8 9XD, Scotland, Great-Britain
RBs&
-
L ' u t i l i s a t i o n du nouveau langage OCCAM pour l e a processus en p a r a l M l e e t l e microprocesseur asaocie, l e transputer, donne au prograaw=ur une liherte inhabituelle. I1 n ' e s t p l u s r e s t r e i n t par l'imposition d'un ordinateur donne e t son s y s t b de controle, e t peut erire l e prograame exactementcornme
il l e veut. En l e f a i s a n t , il oommence en meme temps h d e f i n i r l e "hardware" e t ses interconnections.Abstract
-
The use o f the new p a r a l l e l p r o g m g language OCCAU, and the c l o s e l y aaeociated transputer, gives an unprecedented freedom t o the progrmr. N o longer restricted by an 3mposed computer and operating system, he can write t h e program as he would like it t o be, and i n doing t h i s , h e begina t o define the required hardwate and its interconnections.Since most people w i l l probably not have heard o f either o U 3 ~ o r traneputers, a s h o r t introduction t o each of these may be useful.
The transputer is a 32-bit microprocessor produced by INhW3.5 Ltd. /1/, which is capable of performing 10 million instructions/second (10 MIPS), using a reduced i n s t r u c t i o n set. There are t w o features which d i s t i n g u i s h it from o t h e r
microprocessors. The present -el, t h e T414, h a s 2 k i l o b y t e s o f high speed w r y on the processor chip ( i n 1986 t h e T424 w i l l have 4 Kbytes), and a l s o the necessary c i r c u i t s t o c o n t r o l four p o r t s f o r e x t e r i o r rommunication. These p o r t s give two way communication, and are s p e c i f i c a l l y designed t o be connected t o the p o r t s o f o t h e r transputers. By eame time i n 1987, t h e r e should a l s o be a f l o a t i n g point processor on the c h i p allowing it t o perform 1 million f l o a t i n g point inetructione/sec. Ae w e l l as the ports, which can transmit data between t r a n s p u t e r s at a rate of about 1
l / 4 C4 bytes/sec. using d i r e c t memory access (CUA), t h e r e is a memory bus connection f o r external -ry, with a d a t a rate o f about 25 MBytes/sec.
The microcode on the c h i p includes a scheduler which, when a process can no longer continue, saves the fw necessary r e g i s t e r s and then a c t i v a t e s t h e next process which ie ready.
The traneputer wae designed t o be a building block, and one can understand t h a t assemblies o f them may e a s i l y be conetructeC1, each one with its own local mewry, and with high speed comamication between t h e elements o f the aaeiably. By using the interproceaeor ports, t h e use o f a commDn bus is avoided and hence the strangulation that always r e s u l t s f r o a ~ the i n e v i t a b l e overloading o f it. The atldition o f another transputer t o an aeaembly add another 10 MIPS t o t h e t o t a l processing power.
Article published online by EDP Sciences and available at http://dx.doi.org/10.1051/jphyscol:1986526
JOURNAL DE PHYSIQUE
Such assemblies of microprocessors could be very a t t r a c t i v e solutions t o many problems requiring l o t s o f processing prmer, but t h e t a s k of prograraning them might seem daunting. It is a t t h i s point t h a t one introduces OCCAM, a language designed f o r progranming parallel processing i n multiprocessor systems. It a l s o is a product of I-, based on work by Professor H o a r e of Oxford University, and is a reduced i n s t r u c t i o n set language. OCCAM and the transputer were designed together, and one of the f i r s t uses o f t h e language was t o define the design requirements of t h e transputer.
The language requires that any process o r variable, e t c . m u s t be c o r r e c t l y defined before use.
I n the language :-
CIIRNNELS are used f o r transmitting d a t a :- a R L N . I N ? DATA Take i n some d a t a aRLN.OUl' 1 DATA Send o u t scme data.
For comnunication to take place, both sender and receiver must be ready.
a
requires the i n s t r u c t i o n s following it t o be executed i n sequence.PAR requires t h e i n s t r u c t i o n s following t o be executed i n p a r a l l e l .
-
There i s no d e f a u l t option, one or other o f these must be s p e c i f i c a l l y stated f o r each aeparate u n i t of program.
ALT gives a choice between possible input processes. m e f i r s t t o be ready is
-
executed, along w i t h its associated i n s t r u c t i o n s .I F is a conditional, where the i n s t r u c t i o n s following are executed i f the
-
condition is TRUE. I f not, t h e a l t e r n a t i v e i n s t r u c t i o n s a r e executed.gives r e p e t i t i o n o f a process while a given condition is TRUE.
When a block o f code is extracted and given a name, use o f the name a f t e m a r d e causes i n s e r t i o n o f the block of code at that point. Channels, variables, etc. may be used as argument parameters.
The i n s t r u c t i o n s S a , PAR, and AKE may be multiplied up using FOR, as i n
Sm I
--
[fl f o r 161, when t h e i n s t r u c t i o n s following w i l l be repeated sixteen times with the index I taking the values frum @-15. Comaunication of values, variables, etc. behreen sequential processes is by appropriate parameters. Between parallel processes the correct channels a r e specified.A a e r i e s of p a r a l l e l processes, communicating by channels, a c t s as a pipeline, when the output from one became8 t h e input f o r the next. A simple example is a P i r e t In, P i r s t Out buffer, a more complicated one is when some processing is done, and the r e s u l t s handed on f o r f u r t h e r treatment i n the following process.
'Ihere is no formal operating system. The progr-r provides the necessary pmcesaes f o r d i s k operations, etc. I n t h i s way only the e s s e n t i a l processes are included, and it is f o r the proqr-r t o arrange these as he wishes.
This is a s u i t a b l e point f o r introducing PSD's. The +circle s i n g l e crystal diffractcmeter Dl9 at ILL, is equipped with a PSD o f 16 x 512 cells with t h e 512 cell dimension s e t v e r t i c a l l y . Tests were c a r r i e d out with t h e machine on one of the thermal guides where it was found t h a t , on average, a s u i t a b l e data c o l l e c t i o n time wae about 6 eeconds/frame, and t h e treatment time w a s about 3 seconds/frame, wing the associated VNC-11/750. This is c l e a r l y an acceptable r a t i o of timee, leaving a little leeway f o r possible decrease i n c o l l e c t i o n time o r increase i n treatrent time.
&newer, that was only the test position f o r the i n s t r u m n t , and it is now being i n s t a l l e d on the d i f i e d E l l be?m from t h e reactor. This should give about f o r t y times the f l u x of neutrons. Later, a f t e r ecme experience o f routine running of
hopeless, with the VAX unable t o cope with the t h e o r e t i c a l l y p o ~ s i b l e d a t a rates.
One can imagine some improvement i n t h i s s i t u a t i o n by using multiple VAX's, o r by replacing the VAX-11/750 w i t h a much more powerful computer, but these would be extremely exgensive ways of adding processing power.
The discussion which follovs r e f e r s t o a specific machine and program, b u t o f course, the approach may be w e d whenever it is necessary t o cope with high data rates.
One of the programs which can be used t o reduce t h e d a t a from D l 9 is PEKINT, referred t o by R. Starisfield /2/. Its method is t o f i n d e l l i p s o i d s which contain the peak d a t a i n t h e 3-dinensional a r r a y of count d a t a (on coordinates w, v,Y) and t o make up a l i b r a r y of the parameters of t h e e l l i p s o i d s s u i t a b l e f o r various values o f v and 7. These e l l i p s o i d s are then used t o f i n d the integrated i n t e n s i t y of the r e f l e c t i o n s
.
The program can be broken down i n t o f i v e d i s t i n c t processes, which are :- 1) Get a frame o f data. For each a c t i v e r e f l e c t i o n , define a window about t h e
calculated r e f l e c t i o n centre, and add t h i e 2-dimensional a r r a y t o the a r r a y w n t a i n i n g previous windows f o r t h i s r e f l e c t i o n , t h u s producing t h e 3-dimensional array mentioned above.
2 ) When a l l w u n t s of a p a r t i c u l a r r e f l e c t i o n have been collected, its a r r a y is given a f i r s t treatment which produces an approximate integrated i n t e n s i t y and a peak c e n t r e . This i n t e n s i t y is used t o decide whether t h e r e f l e c t i o n should be t r e a t e d as strong o r weak.
3 ) For strong r e f l e c t i o n s , t h e b e s t estimated values of e l l i p s o i d parameters at present i n the l i b r a r y are used as s t a r t i n g values. They are then modified t o give a best f i t t o the data. This gives t h e f i n a l i n t e g r ~ t e d i n t e n s i t y and peak position, and the e l l i p s o i d parameters are checked against the ones i n t h e l i b r a r y . I f they are b e t t e r than t h e previous ones, the new ones are used t o update the l i b r a r y .
4) Waak r e f l e c t i o n s are considered t o be t o o weak f o r t h e above procedure t o be reliable. I t is assumed therefore that they w i l l be found at t h e positions calculated from t h e UB matrix, am3 that s u i t a b l e e l l i p s o i d parameters f m the l i b r a r y w i l l be c o r r e c t . Sametimes it is necessary t o delay treatment of weak r e f l e c t i o n s u n t i l the l i b r a r y is s u f f i c i e n t l y determined.
5 ) Pinally, the r e s u l t s of t h e s e treatments are assembled and output t o d i s k and a p r i n t e r .
Looking at these processes, one can see that, although i n H)RIlUN they are written as a sequence of processes, from t h e point of view of OCCAn they w u l d form a pipeline, a set of p a r a l l e l processes. Once t h i s view is taken, one can e a s i l y imagine a s i x t h process, whose job is t o hand o u t best estimates of e l l i p s o i d parameters on =West, t a k e i n new values t o check them againet t h e present ones, and do its best t o f i l l i n the gape i n the l i b r a r y .
Thus, i n OCCAU. one would write, with t h e appropriate coxammication channels i n parentheses :
-
PAR
GET.FRIIlIE (DATA, WINDOW)
PIRST.PASS (WImoW, 3 r m N G . m , WEM.oUT)
L I B R Z W A N ( ~ . A s K , ~ . P A R , u E A K . A S K , W E A I C . P A R , NEW.PAR)
SrT(DNG (SPXNG.OWI', SPRWG-ASIC. STI(DNG.PAR, NEW.PAR, -.RESULT) W E M (WESIC.OZrr, WEAIC.ASlC, WEAIC.PAR, BEAK.RESULT)
RESULTS ( m G . R E S W , -.RESULT, OUPPWl'.DISIC, OWlPUI'.PfUW).
JOURNAL DE PHYSIQUE
U s i n g the OCCAM programming system which runs on t h e VAX under VMS, t h i s skeleton could be t h e basis f o r writing a program i n OCCAn, along with various support processes, which would run on t h e VAX.
'Ihie had been the intention, but f o r variow reasons it was not possible before t h e workshop. -r, had it been written, it is q u i t e l i k e l y t h a t it would have run more slowly than the Fortran version.
Where, then, is t h e advantage of OCCAM 7
In breaking down t h e data reduction i n t o these s i x processes, and i n writing them a8 above, one has already s t a r t e d t o define t h e hardware one would lFke, and its interc0MBCtione. Each process would have its awn processor with appropriate memory, and t h e interprocessor connections would follcm t h e channel descriptions.
A t this p i n t , one begins t o appreciate t h e difference between a linear program on a single processor, and p a r a l l e l processing on several processors. On a single processor, t h e existence of a program branch which i s l i t t l e wed means only t h a t the processor has wre time t o spend elsewhere i n t h e prcqram. With t h e p a r a l l e l multiprocessor situation, a l i t t l e used process might mean a l i t t l e used processor, with coneepuent waste of valuable processing power.
Thw the ability t o define the required hardware brings with it t h e need t o rethink the way t h a t programs a r e structured. A possible new version, also with six processors, m i g h t be r-
PAR
(gT.PRAne (DATA, IYINDOW) F1RST.PASS (WINDOW, PIRST.RUN)
LIBRARUN (FIRST.RUN, OWJ!.ONE, IN.ONE, O[lT.W, I N . W )
mxm
(om.=,a.m.
R E S ~ ~ ~ . O N E ) Renuce (ovr.W, I N . W , RESULTS.rn)(RESm.ONE, RESULTS.W, OUX'.DISK, 0CIT.PRINT)
In t h i s version, a l l reflection arrays are sent t o LIBRARIAN, with t h e r e s u l t s of the first i n t e n s i t y t e s t . Strong reflections a r e sent on t o whichever REDUCE is
free, along w i t h appmpziate e l l i p s o i d parameters, i f available. Improved
m r s may be calculated and s e n t back i f required, t o be wed f o r updating t h e l i b r a r y i f suitable. Weak reflection arrays would be stored i n t h e memory
associated w i t h LIBRARfRN's processor u n t i l s u i t a b l e parameters are available, and then sent on to the first f r e e ReDUCE processor.
Renuce can treat either weak o r strong reflections, and calculate new parameters E m s u i t a b l e strong reflections i f required.
c o l l e c t s and arranges t h e output f r a m t h e two REDOCE processors, and outputs appropriately t o disk and printer.
Plug-in boards are available which have a transputer and up t o 2 m a s of random
aECess lspory. This means that one could reasonably envisage in-memory storage of n n t a r i l y untreatable reflections, without recourse t o disk and disk f i l e s . Already i n the f i r s t three processors one could have 6 m e s of memory, wst of uh%ch would be available f o r tempor= buffering of frame arrays and reflection
-.
-r, one might be averwhelPed by t h e thought of writing each part of t h e pzograa, loading it i n t o the appropriate processor, and persuading the whole lot to run cormctly. *is introduces a further feature of t h e OCCAlI pmgranming system.
It has been wntioned above that an OOCAn version of PEKIt4T could be written and run on the VAX ( o r Meed on a single transputer with adequate mewry). It then only
f o r loading t h e processors, and so on. A t compile and asaembly time, account ia taken of all t h i s , so t h a t at run time t h e correct processes are loaded into t h e correct processors and set running, waiting only f o r t h e data t o atart t o flow.
Supposing mw that a l l rune correctly, but one discovers t h a t one procaes runa too slowly and could, with advantage, be duplicated o r divided i n t o two shorter
processes. This would involve only a comparatively minor change i n the program and t h e appropriate changes i n t h e hardwaxe and hardware description. M t e r compiling and assembling, t h e -am should run again, but f a s t e r . Thua it is quite easy t o adapt both program and hardware t o changing requirements.
V
-
WIPIIOCeSSING IN PRACTICESuch a concept may seem somewhat theoretical, and even Utopian, with visions of tens of transputers t i e d together, even with t h e nightmare of programming. EIcuever, a t least two applications of m u l t i transputer assemblies have been made and run successfully. M the University of Southampton, an array of seventeen waa wed to carry out calculations i n solid s t a t e physics /3/, and with the announcement by fWDS of t h e a r r i v a l on t h e market of t h e T414, there was a demonstration by a Bristol firm of an aseeaPbly of 128 tranaplters/4/. For the plrpo~es of the
d ~ p o n s t r a t i o n it was programmed t o give t h e user t h e a b i l i t y t o "travel" around and through a transputer chip, t h e "view" being shawn on t h e monitor as a real tima coiour display.
As explained e a r l i e r , t h e original purpose of t h e project leading t o t h i s paper waa t o rewrite t h e Fortran program PEKINT i n OCCAn and dem~nstrate it on the VAX-1l/750.
What has actually came out of the work is the m a l i e a t i o n of t h e independence rum given t o t h e prograrrmper, who has been freed fram t h e r e s t r i c t i o n of an imposed mmgiuter and its operating system. There is now a completely f r e e choice of program structure, and this structure defines t h e i n i t i a l hatdwace requirements. Ihe examples of t h e s i x processor system could give a t o t a l of 12 of -ry, and t h e processing parer would be 60 HIPS, and 6 MPIOPS, f o r a present price of about 524.000. plua card container, power suppliee, and peripherals.
x t
is rum reaeonable t o consider a transputer aeaembly d i r e c t l y connected t o t h e instrumant giving on-line data treatment, with only t h e r e s u l t s t o be tmnemLtted to a certtral computer f o r archiving and subsequent use.I n addition, t h e ease of adding further transputers brings t h e p o s s i b i l i t i e s of, f o r -le r-
1 ) Display of reflection shapes and resolution ellipsoids.
2 ) Continual verification of t h e UB matrix, with feedback t o t h e machine i f there is a significant change.
3 ) Continual structure refinement, a l l w i n g early detection of anomalies o r m e d p h e m ~ in t h e results.
4 ) Poseibly even a display aesieted structure determination, with production and display of Fourier maps, etc. f o r t h e more complicated structures.
such a *em should be b u i l t up steadily a s experience was gained and further d e v e t a n t a ehcmed thewelvem t o be desirable. It wMlld also be e a s i l y prog-le i f there were changes in t h e way t h e instrument was ucled.
/1/ IPIIPB YR), W t e f r i r r s , Isnrine Head, B r i e t o l , U.K.
/2/ R.P.D. Staiisfield, P.S.D. Workshop Proceedings.
/3/ Personal ccamurication.
/4/ E l e c t r o n i c e , October 7, 1985, pp. 43-45.
