The numerical experiments have been carried out on a PC Intel Celeron with 2.7 GHz CPU and 512 MB RAM. The programs are coded with Compac Visual FORTRAN, Version 6.1. The unconstrained minimization tasks were carried out by means of a Quasi-Newton algorithm employing the BFGS updating formula from the Harwell Library, obtained in the site: (http://www.cse.scitech.ac.uk/nag/hsl/).
Solving the Minimum Sum of L1 Distances Clustering Problem by Hyperbolic. . . 43 Table 1 Results of AHSC-L1 applied to TSPLIB-1060 and TSPLIB-3038 Instance
TSPLIB 1060 TSPLIB 3038
q fAHSCL1Best Occur EMean Tcpu fAHSCL1Best Occur EMean Tcpu
2 0.386500E7 2 0.00 0.10 0.373171E7 2 0.88 0.76
3 0.313377E7 1 0.22 0.14 0.300708E7 1 1.33 0.76
4 0.258205E7 5 0.43 0.21 0.254499E7 1 0.67 0.79
5 0.231098E7 1 0.76 0.29 0.225571E7 1 1.28 0.95
6 0.213567E7 1 0.79 0.35 0.206006E7 1 1.12 0.95
7 0.196685E7 1 1.41 0.48 0.189650E7 1 1.34 1.06
8 0.183280E7 1 2.22 0.53 0.176810E7 1 1.14 1.13
9 0.168634E7 1 3.42 0.60 0.164559E7 1 2.13 1.21
10 0.155220E7 1 2.74 0.68 0.154550E7 1 1.97 1.28
Table 2 Results of AHSC-L1 applied to D15112 and Pla85900 Instance
D15112 Pla85900
q fAHSCL1Best Occur EMean Tcpu fAHSCL1Best Occur EMean Tcpu
2 0.822872E8 6 0.97 10.35 0.883378E10 2 0.00 242.74
3 0.655831E8 1 1.43 7.56 0.667961E10 1 0.21 166.48
4 0.567702E8 1 1.60 6.21 0.551287E10 2 0.06 129.30
5 0.511639E8 1 1.17 5.73 0.482328E10 1 1.43 112.95
6 0.462612E8 1 1.67 5.33 0.432972E10 1 2.47 103.02
7 0.425722E8 1 2.30 4.96 0.401388E10 1 1.87 98.25
8 0.398389E8 1 1.83 5.02 0.373878E10 1 3.47 92.14
9 0.376863E8 1 1.60 5.03 0.355741E10 1 2.40 82.33
10 0.354762E8 1 2.41 5.01 0.341472E10 1 1.77 87.41
In order to exhibit the distinct performance of the AHSC-L1 algorithm, Tables1 and 2 present the computational results of AHSC-L1 applied to four bench-mark problems, all from TSPLIB (Reinelt 1991; http://www.iwr.uni-heidelberg.
de/groups/comopt/software). Table 1 represent two instances frequently used as benchmark clustering problems. Table2left contains data of 15,112 German cities.
Table2right is the largest symmetric problem of TSPLIB.
The AHSC-L1 is a general framework that bears a broad number of implemen-tations. In the initialization steps the following choices were made for the reduction factors:1 D1=4; 2 D1=4and3 D 1=4:The specification of initial smoothing and perturbation parameters was automatically tuned to the problem data. So, the initialmaxfunction smoothing parameter (4) was specified by1 D =10where 2 is the variance of set of observation points: S D fs1; : : : ; smg: The initial perturbation parameter (3) was specified by1 D 41 and the Euclidian distance smoothing parameter by1D1=100:
All experiments where done using ten initial points. The adopted stopping criterion was the execution of the main step of the AHSC-L1 algorithm in a fixed number of six iterations.
44 A.E. Xavier et al.
Table 3 Speed-up of AHSC-L1 compared to HSC-L1 (the larger the better)
q TSP1060 TSP3038 D15112 Pla85900
2 2.60 1.33 0.94 0.82
3 3.79 1.84 1.43 0.95
4 3.24 3.08 1.95 1.30
5 4.41 3.16 3.17 1.76
6 5.23 4.69 4.60 2.58
7 4.96 5.84 5.94 3.80
8 6.94 7.09 8.44 4.44
9 6.30 7.97 10.34 6.78
10 7.66 12.07 13.26 8.75
The meaning of the columns Tables1 and2is as follows.q Dthe number of clusters.fAHSCL1Best D the best results of cost function using points obtained from AHSC-L1 method out of all the ten random initial points. Occur.Dnumber of times the same best result was obtained from all the tenl random initial points.
EMean D the average error of the ten solutions in relation to the best solution obtained (fAHSCL1Best). Finally,Tcpu D the average execution time per trial, in seconds.
The “A” of AHSC-L1 means “accelerated”, that is, the technique that partitions the set of observations into two non overlapping groups: “data in frontier” and
“data in gravitational regions”. The sample problems were solved using HSC-L1 and AHSC-L1 methods. Both algorithms obtain the same results with three decimal digits of precision. The Table3shows the speed-up produced by the acceleration technique. The meaning of the columns of Table3is as follows.qDthe number of clusters. Speed-up for TSPLIB-1060 Instance. Speed-up for TSPLIB-3038 Instance.
Speed-up for D15112 Instance. Speed-up for Pla85900 Instance.
The speed-up was calculated as the ratio between execution timesTHSCL1 and TAHSCL1, as shown in Eq. (28).
SpeedupD THSCL1
TAHSCL1
(28) The results in the Table3 show that in most cases the “accelerated” technique produces speed-up of the computation effort. In some cases, the speed-up is> 10.
In a few cases (e.g.qD 2, 85,900), the gains produced by the acceleration do not compensate the fixed costs introduced by the calculus of partition. In these cases the speed-up is less than one, that is, AHSC-L1 takes longer to run when compared to HSC-L1.
8 Conclusions
In this paper, a new method for the solution of the minimum sum of L1 Manhattan distances clustering problem is proposed, called AHSC-L1 (Accelerated Hyperbolic Smoothing Clustering – L1). It is a natural development of the original HSC
Solving the Minimum Sum of L1 Distances Clustering Problem by Hyperbolic. . . 45 method and its descendant AHSC-L2 method, linked to the minimum sum-of-squares clustering (MSSC) formulation, presented respectively by Xavier(2010) an by Xavier and Xavier(2011).
The special characteristics of L1 distance were taken into account to adapt inside the AHSC-L1 method from the AHSC-L2. The main idea proposed in this paper is the acceleration of the AHSC-L1 method by the partition of the set of observations into two non overlapping parts – gravitational and boundary. The classification of gravitational observations by each component, implemented by Eqs. (21)–(23), simplifies of the calculation of the objective function (26) and its gradient (27).
This classification produces a drastic simplification of computational tasks. The computational experiments confirm the speed-up, as shown in Table3.
The computational experiments presented in this paper were obtained by using a particular and simple set of criteria for all specifications. The AHSC-L1 algorithm is a general framework that can support different implementations.
We could not find in the literature any reference mentioning the solution of cluster L1 problem with instances of sizes similar to those presented in this paper.
So, our results represent a challenge for future works.
The most relevant computational task associated with the AHSC-L1 algorithm remains the determination of the zeros of the Eq. (9), for each observation in the boundary region, with the purpose of calculating the first part of the objective function. However, since these calculations are completely independent, they can be easily implemented using parallel computing techniques.
It must be observed that the AHSC-L1 algorithm, as presented here, is firmly linked to the MSDC-L1 problem formulation. Thus, each different problem formu-lation requires a specific methodology to be developed, in order to apply the partition into boundary and gravitational regions.
Finally, it must be remembered that the MSDC-L1 problem is a global optimiza-tion problem with several local minima, so both HSC-L1 and AHSC-L1 algorithms can only produce local minima. The obtained computational results exhibit a deep local minima property, which is well suited to the requirements of practical applications.
Acknowledgements The author would like to thank Cl´audio Joaquim Martag˜ao Gesteira of Federal University of Rio de Janeiro for the helpful review of the work and constructive comments.
References
Bradley, P. S., & Mangasarian, O. L. (1997) Clustering via concave minimization (Mathematical Programming Technical Report 96-03), May 1996. Advances in neural information processing systems (Vol. 9, pp. 368–374). Cambridge: MIT.
Hartigan, J. A. (1975) Clustering algorithms. New York: Wiley.
Reinelt, G. (1991). TSP-LIB – a traveling salesman library. ORSA Journal on Computing, 3, 376–384.
Sp¨ath, H. (1980). Cluster analysis algorithms for data reduction and classification. Upper Saddle River: Ellis Horwood.
46 A.E. Xavier et al.
Xavier, A. E. (1982). Penalizac¸ ˜ao hiperb´olica: um novo m´etodo para resoluc¸ ˜ao de problemas de otimizac¸ ˜ao. M.Sc. Thesis, COPPE – UFRJ, Rio de Janeiro.
Xavier, A. E. (2010). The hyperbolic smothing clustering method. Pattern Recognition, 43, 731–
737.
Xavier A. E., & Xavier, V.L. (2011). Solving the minimum sum-of-squares clustering problem by hyperbolic smoothing and partition into boundary and gravitational regions. Pattern Recognition, 44, 70–77.