5.1. Summary of Numerical Data
This section details the results of applications testing and includes a results analysis. The raw results obtained from the test cluster, including scripts of command line input/output can be found in Appendix E.
From the large amount of data and test results obtained in the laboratory, the most interesting are detailed in this section (applications level only) to show the various speedup indices of a cluster computer. Shown in Table 5-1 below, are the results of the rendering test cases as detailed in section 4.4.1.
No. of CPUs
Render Time
(seconds) Speedup Render Time (seconds)
Speedup over X-POV-Ray1
Speedup on
MPI-X-POV-Ray2
Render Time
(seconds) Speedup
1 129 1.0 25411 / 17862 1.0 1.0 279264 1.0
2 72 1.8 906 2.8 2.0
3 48 2.7 605 4.2 3.0
4 36 3.6 457 5.6 3.9
5 29 4.4 364 7.0 4.9
6 25 5.2 309 8.2 5.8
7 22 5.9 262 9.7 6.8
8 20 6.5 229 11.1 7.8 Failed
skyvase.pov poolballs.pov tulips.pov
Table 5-1 – Rendering Test Results
5.2. Results Analysis
With respect to Table 5-1 the following analysis of the numerical results is presented:
Skyvase
The skvase.pov test case is the benchmark for a cluster computer using parallel rendering.
The results show that a near ideal speedup was achieved.
From the POVBENCH benchmark data [Table 8-3] it can be seen that for a single node the test cluster achieved a POVmark of 114.73 which is considered above average for the hardware used (based on the POVmark data). This can be attributed to the use of structured design and implementation of Linux on each node as well as efficiencies included in the implemented Linux kernel version 2.4.7-10.
When rendered on all eight nodes the test cluster achieved a POVmark of 740.00. When comparing the cluster test results with the POVBENCH cluster data, the test cluster takes the 86th spot with a parallel rendering time of 20sec, out of the 236 parallel results submitted. This
Poolballs
Of all of the test-cases the Poolballs test results contained the most interesting trend.
The test case was run under two scenarios, as follows:
1. X-POV-Ray – Poolballs was first run on one node using the standard serial version of POV-Ray. This resulted in a best execution time of 2541 seconds. All further results were obtained using the parallel version as required for multiple CPU’s. It can be seen from the results that a super-linear speedup was achieved over the serial version.
2. MPI-X-POV-Ray – Whilst the first scenario was run many times, each time showing a super-linear speedup was achieved, it was decided to test the case when the Parallel version is run on one node using a master and slave process. This required the operating system to multitask between the two processes and it was expected that a similar result to the serial version would be achieved. However as shown in Table 5-1, the parallel version out-performed the serial version on one node. This changes the speedup results when compared to the parallel version, showing a close to ideal speedup.
Tulips
The tulips test case has a high degree of difficulty and was used to test the stability of the cluster. Additionally it was intended to provide a longer test case which could show the trend as many messages are sent and received over the network, increasing the load on the server, the communications medium, and each node.
Unfortunately, the test case took over 77 hours to render on one node and as such results were not obtained for each combination of processors. The test case was run on all eight nodes, however it was apparent that after five days that it had crashed the cluster, and due to time constraints further testing to get it running to completion was not possible. Hence, this test case has not provided any information on speedup index.
From the work that was done using the rendering test cases, in particular Tulips, it is theorized, that as the resources are heavily utilized the speedup index decreases to the lower bound Log2n.
Shown in Figure 5-1 below is a plot of the salient test results: skyvase, poolballs super linear speedup and the applicable theoretical expectations as derived in section 3.1.3.
0 2 4 6 8 10 12
0 2 4 6 8
Number of Processors
Speedup
Ideal Speedup Log2n
Amdahls f = 0.05 Skyvase Speedup PoolBalls Speedup
Figure 5-1 – Test Results & Theoretical Performance Comparison
It was noted from the raw test results produced by MPI-POV-Ray, that each node was not contributing an equal percentage of work. Whilst MPI-POV-Ray divides an image up equally, it does not evaluate the difficulty of each chunk. Each chunk is different and takes a different time to render. As compensation to this, MPI-POV-Ray distributes small chunks (also chunk size can be modified by the user to suit the particular architecture) such that the disparity is minimised. However, this disparity does lead to a decrease in the overall speedup index.
Whilst this is minimal for the rendering algorithm, as it can be considered symmetric, this is not the case for all algorithms, and applications.
[Refer to Appendix E – 8.5.2: Within each multi-CPU test case are PE (PE) Distribution Statistics. These statics detail the percentage of work that each node completed].
The following conclusions can be draw from these results:
• The performance of a Beowulf Cluster can be modelled by a pessimistic Lower bound Log2n for n
≤
8 and the upper bound, which is the ideal case n.• Amdahl’s law with f=0.05 models more closely the lower bound than Log2n however it is noted that both are highly pessimistic models when trended out to larger numbers of processors such as 64, 128 and 512.
• In some instances super-linear speedups can be achieved, however only with tree-search type applications or inherently parallel algorithms.
• Through the process of structured design, a high-performing, cost-effective Cluster Computer can be built with known performance bounds.
• Load Balancing for nodes is generally required. Should this feature not be available it is possible to manually balance work using the weighted harmonic mean principle [as