GLOBAL OPTIMIZATION OF THE MAXIMUM K-CUT PROBLEM
VILMAR JEFTÉ RODRIGUES DE SOUSA
DÉPARTEMENT DE MATHÉMATIQUES ET DE GÉNIE INDUSTRIEL ÉCOLE POLYTECHNIQUE DE MONTRÉAL
THÈSE PRÉSENTÉE EN VUE DE L’OBTENTION DU DIPLÔME DE PHILOSOPHIÆ DOCTOR
(MATHÉMATIQUES DE L’INGÉNIEUR) MAI 2018
c
ÉCOLE POLYTECHNIQUE DE MONTRÉAL
Cette thèse intitulée :
GLOBAL OPTIMIZATION OF THE MAXIMUM K-CUT PROBLEM
présentée par : RODRIGUES DE SOUSA Vilmar Jefté en vue de l’obtention du diplôme de : Philosophiæ Doctor a été dûment acceptée par le jury d’examen constitué de :
Mme LAHRICHI Nadia, Ph. D., présidente
M. ANJOS Miguel F., Ph. D., membre et directeur de recherche
M. LE DIGABEL Sébastien, Ph. D., membre et codirecteur de recherche M. PERRIER Michel, Ph. D., membre
DEDICATION
To the Almighty God and to my family . . .
ACKNOWLEDGEMENTS
First and foremost I gratefully acknowledge the support and guidance of my supervisors Miguel F. Anjos and Sébastien Le Digabel. I’m infinitely thankful for your time, patience, ideas and corrections. Without you, the present study could not have been completed. I also want to thank Jacek Gondzio and all the members of my jury for their helpful insights.
To my family, muito obrigado for your unconditional love, support, and encouragement during the challenges. Especially, my wife, Elisama, thank you for being by my side throughout this process and for understanding my craziness.
To all my friends at GERAD: thank you, merci, obrigado, gracias, shukraan, Mamnoun... You’re a great group of friends; the tennis, volley, soccer, lunches, and practice of presentations made these three years pass faster and you kept me sane. I would also like to thank all the staff of GERAD and Polytechnique Montréal for their daily assistance.
RÉSUMÉ
Le problème de la k-coupe maximale (max-k-cut) est un problème de partitionnement de graphes qui est un des représentatifs de la classe des problèmes combinatoires N P-difficiles. Le max-k-cut peut être utilisé dans de nombreuses applications industrielles. L’objectif de ce problème est de partitionner l’ensemble des sommets en k parties de telle façon que le poids total des arrêtes coupées soit maximisé.
Les méthodes proposées dans la littérature pour résoudre le max-k-cut emploient, généralement, la programmation semidéfinie positive (SDP) associée. En comparaison avec les relaxations de la programmation linéaire (LP), les relaxations SDP sont plus fortes mais les temps de calcul sont plus élevés. Par conséquent, les méthodes basées sur la SDP ne peuvent pas résoudre de gros problèmes. Cette thèse introduit une méthode efficace de branchement et de résolution du problème max-k-cut en utilisant des relaxations SDP et LP renforcées.
Cette thèse présente trois approches pour améliorer les solutions du max-k-cut. La première ap-proche se concentre sur l’identification des classes d’inégalités les plus pertinentes des relaxations de max-k-cut. Cette approche consiste en une étude expérimentale de quatre classes d’inégalités de la littérature : clique, general clique, wheel et bicycle wheel. Afin d’inclure ces inégalités dans les formulations, nous utilisons un algorithme de plan coupant (CPA) pour ajouter seulement les inégalités les plus importantes . Ainsi, nous avons conçu plusieurs procédures de séparation pour trouver les violations. Les résultats suggèrent que les inégalités de wheel sont les plus fortes. De plus, l’inclusion de ces inégalités dans le max-k-cut peut améliorer la borne de la SDP de plus de 2%.
La deuxième approche introduit les contraintes basées sur formulation SDP pour renforcer la re-laxation LP. De plus, le CPA est amélioré en exploitant la technique de terminaison précoce d’une méthode de points intérieurs. Les résultats montrent que la relaxation LP avec les inégalités basées sur la SDP surpasse la relaxation SDP pour de nombreux cas, en particulier pour les instances avec un grand nombre de partitions (k ≥ 7).
La troisième approche étudie la méthode d’énumération implicite en se basant sur les résultats des dernières approches. On étudie quatre composantes de la méthode. Tout d’abord, nous pré-sentons quatre méthodes heuristiques pour trouver des solutions réalisables : l’heuristique itérative d’agrégation, l’heuristique d’opérateur multiple, la recherche à voisinages variables, et la procédure de recherche aléatoire adaptative gloutonne. La deuxième procédure analyse les stratégies dichoto-miques et polytodichoto-miques pour diviser un sous-problème. La troisième composante étudie cinq règles de branchement. Enfin, pour la sélection des nœuds de l’arbre de branchement, nous considérons
les stratégies suivantes : meilleur d’abord, profondeur d’abord, et largeur d’abord. Pour chaque stratégie, nous fournissons des tests pour différentes valeurs de k. Les résultats montrent que la méthode exacte proposée est capable de trouver de nombreuses solutions.
Chacune de ces trois approches a contribué à la conception d’une méthode efficace pour résoudre le problème du max-k-cut. De plus, les approches proposées peuvent être étendues pour résoudre des problèmes génériques d’optimisation en variables mixtes.
ABSTRACT
In graph theory, the maximum k-cut (max-k-cut) problem is a representative problem of the class of N P-hard combinatorial optimization problems. It arises in many industrial applications and the objective of this problem is to partition vertices of a given graph into at most k partitions such that the total weight of the cut is maximized.
The methods proposed in the literature to optimally solve the max-k-cut employ, usually, the associ-ated semidefinite programming (SDP) relaxation in a branch-and-bound framework. In comparison with the linear programming (LP) relaxation, the SDP relaxation is stronger but it suffers from high CPU times. Therefore, methods based on SDP cannot solve large problems. This thesis introduces an efficient branch-and-bound method to solve the max-k-cut problem by using tightened SDP and LP relaxations.
This thesis presents three approaches to improve the solutions of the problem. The first approach focuses on identifying relevant classes of inequalities to tighten the relaxations of the max-k-cut. This approach carries out an experimental study of four classes of inequalities from the literature: clique, general clique, wheel and bicycle wheel. In order to include these inequalities, we employ a cutting plane algorithm (CPA) to add only the most important inequalities in practice and we design several separation routines to find violations in a relaxed solution. Computational results suggest that the wheel inequalities are the strongest by far. Moreover, the inclusion of these inequalities in the max-k-cut improves the bound of the SDP formulation by more than 2%. The second approach introduces the SDP-based constraints to strengthen the LP relaxation. More-over, the CPA is improved by exploiting the early-termination technique of an interior-point method. Computational results show that the LP relaxation with the SDP-based inequalities outperforms the SDP relaxations for many instances, especially for a large number of partitions (k ≥ 7).
The third approach investigates the branch-and-bound method using both previous approaches. Four components of the branch-and-bound are considered. First, four heuristic methods are pre-sented to find a feasible solution: the iterative clustering heuristic, the multiple operator heuristic, the variable neighborhood search, and the greedy randomized adaptive search procedure. The sec-ond procedure analyzes the dichotomic and polytomic strategies to split a subproblem. The third feature studies five branching rules. Finally, for the node selection, we consider the following strategies: best-first search, depth-first search, and breadth-first search. For each component, we provide computational tests for different values of k. Computational results show that the proposed exact method is able to uncover many solutions.
Each one of these three approaches contributed to the design of an efficient method to solve the max-k-cut problem. Moreover, the proposed approaches can be extended to solve generic mix-integer SDP problems.
TABLE OF CONTENTS DEDICATION . . . iii ACKNOWLEDGEMENTS . . . iv RÉSUMÉ . . . v ABSTRACT . . . vii TABLE OF CONTENTS . . . ix
LIST OF TABLES . . . xii
LIST OF FIGURES . . . xiii
LIST OF SYMBOLS AND ABBREVIATIONS . . . xiv
CHAPTER 1 INTRODUCTION . . . 1
1.1 Background . . . 1
1.2 The maximum k-cut problem . . . 1
1.3 Objective and outline . . . 2
CHAPTER 2 CRITICAL LITERATURE REVIEW . . . 3
2.1 Applications . . . 3
2.2 Formulations of the max-k-cut . . . . 4
2.2.1 Integer programming formulations . . . 4
2.2.2 Linear programming relaxation . . . 6
2.2.3 Semidefinite programming relaxation . . . 7
2.2.4 SDP versus LP relaxations . . . 9
2.3 Strengthening max-k-cut formulations . . . . 9
2.3.1 Valid inequalities . . . 10
2.3.2 Cutting plane algorithm . . . 10
2.4 Methods to solve the max-k-cut problem . . . . 11
2.4.1 Approximations methods . . . 11
2.4.2 Heuristic methods . . . 12
2.5 Additional reviews . . . 13
CHAPTER 3 ORGANIZATION OF THE THESIS . . . 14
CHAPTER 4 ARTICLE 1: COMPUTATIONAL STUDY OF VALID INEQUALITIES FOR THE MAXIMUM K-CUT PROBLEM . . . 16
4.1 Introduction . . . 16
4.1.1 Problem formulation . . . 18
4.1.2 Semidefinite relaxation . . . 18
4.2 Formulation and separation of inequalities . . . 19
4.2.1 Triangle inequalities . . . 19
4.2.2 Clique inequalities . . . 20
4.2.3 General clique inequalities . . . 20
4.2.4 Wheel inequalities . . . 21
4.2.5 Bicycle wheel inequalities . . . 23
4.3 Cutting plane algorithm . . . 25
4.4 Computational tests . . . 26
4.4.1 Combinations of inequalities and test instances . . . 26
4.4.2 Comparison methodology . . . 28
4.4.3 Computational results . . . 30
4.4.4 Summary of the computational tests . . . 40
4.5 Discussion . . . 41
CHAPTER 5 ARTICLE 2: IMPROVING THE LINEAR RELAXATION OF MAXIMUM K-CUT WITH SEMIDEFINITE-BASED CONSTRAINTS . . . 43
5.1 Introduction . . . 43
5.1.1 Formulations . . . 44
5.2 SDP-based inequality . . . 46
5.2.1 Semi-infinite formulation of SDP . . . 46
5.2.2 Variable transformations . . . 47
5.2.3 SDP-based inequality formulation . . . 47
5.3 Cutting-plane algorithm . . . 47
5.3.1 Separation routines . . . 48
5.3.2 Dropping inequalities . . . 50
5.3.3 Solving the relaxations . . . 51
5.4 Computational tests . . . 52
5.4.2 Instances . . . 53
5.4.3 Comparison methodology . . . 54
5.4.4 Computational results . . . 55
5.4.5 Summary of computational tests . . . 60
5.5 Discussion . . . 60
CHAPTER 6 AN EXACT METHOD FOR THE MAXIMUM K-CUT PROBLEM . . . . 63
6.1 Introduction . . . 63
6.2 A generic branch-and-bound framework . . . 64
6.3 Bounding procedures in the branch-and-bound framework . . . 67
6.3.1 Computing upper bounds . . . 67
6.3.2 Lower bound . . . 71
6.4 Selection and branching strategies . . . 73
6.4.1 Splitting problem . . . 73
6.4.2 Branching Rules . . . 75
6.4.3 Node selection . . . 78
6.5 Computational environment and instances . . . 79
6.6 Computational results . . . 80
6.6.1 Comparison of the linear formulations in the branch-and-bound framework 80 6.6.2 Comparison of LP-EIG versus SDP . . . 81
6.6.3 Summary of the computational tests . . . 83
6.7 Discussion . . . 84
CHAPTER 7 GENERAL DISCUSSION . . . 85
CHAPTER 8 CONCLUSION AND RECOMMENDATIONS . . . 86
8.1 Advancement of knowledge . . . 86
8.2 Limits and constraints . . . 86
8.3 Recommendations . . . 87
LIST OF TABLES
Table 4.1 Ten combinations for study of valid inequalities. . . 27
Table 4.2 Summary of the best inequalities for each benchmark. . . 41
Table 5.1 Performance comparison for Biq Mac instances and k = 3. . . . 56
Table 5.2 Performance comparison for Biq Mac instances and k = 10. . . . 57
Table 5.3 Performance comparison for random instances and k = 3. . . . 57
Table 5.4 Performance comparison for random instances and k = 10. . . . 58
Table 5.5 Best method(s) for each type of problem. . . 60
Table 6.1 Results for cut-and-branch and branch-and-cut for k ∈ {3, 5}. . . . 70
Table 6.2 Results of four heuristic methods for finding feasible solutions. . . 73
Table 6.3 Results for dichotomic and k-chotomic strategies in the branch-and-bound framework for k ∈ {3, 5}. . . . 76
Table 6.4 Comparative results for five branching rules in the branch-and-bound frame-work. . . 78
Table 6.5 Comparative results of node selection strategies: BeFS, BrFS and DFS. . . 79
Table 6.6 Comparison of the linear formulations in the branch-and-bound framework. 81 Table 6.7 Results of the LP-EIG and SDP formulations in the branch-and-bound framework. . . 82
LIST OF FIGURES
Figure 1.1 Illustration of a max-k-cut problem in a graph for k = 3. . . . 2
Figure 4.1 Example of wheel with q = 6. . . . 22
Figure 4.2 Example of bicycle wheel with q = 5. . . . 24
Figure 4.3 Performance profiles for all instances. Ideal point is at (0.0, 100). . . 31
Figure 4.4 Data profiles for all instances with rmax = 0.0. Ideal point is at (0.0, 100). . 32
Figure 4.5 Performance versus CPU time for all instances. Ideal point is at (0.0, 0.0). . 33
Figure 4.6 Relevance of the inequalities for all instances. . . 34
Figure 4.7 Performance profiles for sparse instances. Ideal point is at (0.0, 100). . . . 35
Figure 4.8 Performance versus CPU time for sparse instances. Ideal point is at (0.0, 0.0). . . 36
Figure 4.9 Relevance of the inequalities for sparse instances. . . 37
Figure 4.10 Performance profiles for dense instances. Ideal point is at (0.0, 100). . . 38
Figure 4.11 Performance versus CPU time for dense instances. Ideal point is at (0.0, 0.0). 39 Figure 4.12 Relevance of the inequalities for dense instances. . . 40
Figure 5.1 Scheme of cutting plane algorithm (CPA). . . 49
Figure 5.2 Separation of Constraint (5.3) in the SDP formulation. . . 49
Figure 5.3 Study of early termination in interior point method (IPM). . . 53
Figure 5.4 Data profiles for instances with positive weights for various values of par-tition size k. . . . 59
Figure 5.5 Data profiles for instances with mixed weights for various values of parti-tion size k. . . . 59
Figure 5.6 Performance profiles for instances with positive weights for various values of partition size k. . . . 61
Figure 5.7 Performance profiles for instances with mixed weights for various values of partition size k. . . . 61
LIST OF SYMBOLS AND ABBREVIATIONS
G = (V, E) A graph
E Set of edges in a graph V Set of vertices in a graph n = |V | Number of vertices in set V |E| Number of edges in set E
R Real set
Sn Symmetric n × n Matrix S+
n Symmetric Positive Semidefinite n × n Matrix BeFS Best First Search
BrFS Breath First Search CPA Cutting Plane Algorithm CPU Central Processing Unit DC Dynamic Convexized Method DFS Depth First Search
GHz Gigahertz
GMSW Greedy Heuristic for Multiple Sizes of Wheel GPP Graph Partitioning Problem
GRASP Greedy Randomized Adaptive Search Procedure ICH Iterative Clustering Heuristic
IPM Interior Point Method
LP Linear Programming
LSIP Linear Semi-Infinite Programing max-k-cut Maximum k-Cut
MOH Multiple Operator Heuristic N P Nondeterministic Polynomial
PC Personal Computer
SDP Semidefinite Programming
SBC Semidefinite Branch-and-Cut algorithm SIP Semi-Infinite Programing
VLSI Very-Large-Scale-Integrated VNS Variable Neighborhood Search
CHAPTER 1 INTRODUCTION
The maximum k-cut (max-k-cut) problem is a graph partitioning problem that is a representative problem of the class of N P-complete combinatorial optimization problems. Although it has several industrial applications, there is not a lot of computational studies proposed in the literature and the existing exact methods are very limited. Therefore, the main objective of this thesis is to investigate and propose new mathematical optimization techniques to design an efficient method to solve the problem to optimality.
1.1 Background
Operations research is a scientific approach that applies advanced analytical methods to support the decision making process and to optimize systems. In order to deal with a problem, it is typically necessary to create an abstraction of the real-world situation. A mathematical model is used as an attempt to describe the system.
A wide variety of real-world problems can conveniently be described by a graph, for example, the famous traveling salesman problem [69] and the seminal problem of the Königsberg bridges [20]. In graph theory, a graph G = (V, E) is defined by a set of vertex V and a set of edges E.
Among all types of graphs, we consider the connected-weighted-simple graphs where V is a finite set and the edges are undirected with weight wij for all (i, j) ∈ E. Moreover, multiple edges or loops are disallowed.
Many operations can be performed on a graph, and one of them is a cut. In general, the graph is k-cut if the vertex set V is partitioned into at most k partitions in such a way that the intersection of two different partitions Pi ⊆ V and Pj ⊆ V is an empty set, Pi ∩ Pj = ∅ for i 6= j for each i, j ∈ {1, 2, . . . , k}. In addition, all vertices inside the k partitions should also be in the vertex set V = P1 ∪ P2· · · ∪ Pk. The value of k-cut is the sum of weights of all edges joining different partitions. Calculating the optimal cut is a graph partitioning problem.
1.2 The maximum k-cut problem
The objective of the max-k-cut problem is to partition the vertex set of a graph into at most k parti-tions such that the total weight of the k-cut is maximized. The problem arises in many applicaparti-tions, e.g., VLSI circuit design, wireless communication [54] and sports team scheduling [60]. Some of these applications are reviewed in Chapter 2. Figure 1.1 illustrates the max-k-cut for a graph with
|V | = 7, |E| = 11 and k = 3. 3 1 3 1 1 1 5 2 4 1 0.5 max-k-cut = 22 0.5 12 8 2
Figure 1.1 Illustration of a max-k-cut problem in a graph for k = 3.
For the max-k-cut, the special case with k = 2 is known as the max-cut problem. Due to the equivalence of this problem with unconstrained binary quadratic optimization, the max-cut has received most attention in the literature, see e.g. [51,74]. In our investigation, we focus on problems with k ≥ 3.
The max-k-cut is equivalent to the minimum k-partition problem [31]. In the minimum k-partition, the task is to minimize the total weight of the edges joining vertices in the same partition. The k-way equipartition problem is another related problem. In the k-way equipartition, we add some constraints that force the partitions to be the same (or almost) sizes.
In an unweighted graph, the max-k-cut can also be used to provide bounds for the chromatic num-ber. The smallest number of colors needed to color the vertices of a graph in such a way that two adjacent vertices (joined by an edge) cannot have the same color.
1.3 Objective and outline
The objective of this thesis is to design an efficient solver for the max-k-cut problem. Hence, we investigate the following three objectives in next chapters. First, computational study of some proposed facet-defining inequalities to identify the most relevant in practice. Second, investigate ways of strengthen even more the mathematical model of the max-k-cut. Third, design an exact method to solve the problem to optimality.
In the first chapter of this thesis, we present the state-of-art of max-k-cut with applications, formu-lations, and methods proposed in the literature. Chapter 3 details the organization of the research. The Chapters 4, 5, and 6 are the core of the thesis, where we present the two articles and one chap-ter about the branch-and-bound algorithm where we present an exact method to obtain the global optimal solution for the max-k-cut problem. Finally, we conclude our work in Chapter 8.
CHAPTER 2 CRITICAL LITERATURE REVIEW
This chapter presents the state-of-art of the max-k-cut problem. Section 2.1 presents the most im-portant applications. Section 2.2 review integer formulations and their relaxations. In Section 2.3, we present some valid inequalities that are proposed in the literature and Section 2.4 presents ap-proximations, heuristics and exact methods designed to solve the max-k-cut problem.
2.1 Applications
Several industrial optimization problems are formulated as max-k-cut, some of the most important are described below.
Statistical physics. In [6], two classical applications of max-k-cut are presented: the statistical physics and very-large-scale-integrated (VLSI) circuit design. The ground states or minimum en-ergy configuration of sping glasses are determined in statistical physics. The ground state can be found by minimizing the energy interaction between magnetic atoms in the sping glass. The en-ergy between two atoms depends on their orientations and their distance. A simplified version of 1-dimensional direction is studied in [6,52] where the problem is formulated as a max-cut problem.
(VLSI) circuit design. This problem arises in the phase of layer assignment in the construction of chip where wires belonging to different net may be crossed [6,8]. Additional cost and often failure of the board happens when two wires cross in the same layer. Therefore, it is desirable to find a layer assignment that minimizes cost and failure. In [6], the wires are considered as the vertices, the crossing as the edges, and the layers as the partitions of a max-k-cut problem. This application can be generalized to modular design where a system is divided into smaller parts called modules in order to maximize independence between modules [29].
Team Realignment. A very common problem that is formulated as a k-equipartition problem is the team scheduling. In [60, 72] the authors study the realignment problem for a football league where the objective is to minimize the sum of intra-divisional distance. In this application, the cities are the vertices of the graph, the distances between cities correspond to the weight of edges and the number of division are the k partitions. An interesting application related to team scheduling is proposed in [28] to assign doctors for victims in emergency situations.
Wireless communication. In the case of wireless communications, the wireless network designs are the hardest problems. Most investigations are related to the problem of frequency assignment where the aim is to minimize interferences between devices see e.g. [22,86]. However, in [54] the objective is to assign a group of exchange stations in such a way that as much traffic as possible can be routed inside these clusters.
Other applications. Many other cluster problems can be formulated as a max-k-cut problem such as parallel computing [44], floor planning [12], and others assignment problems like the ones studied in the capacitated version of max-k-cut [29]: the placement of television commercials and the placement of containers.
2.2 Formulations of the max-k-cut
The max-k-cut is known to be an N P-hard combinatorial optimization [71]. In order to compu-tationally solve the problem some integer programming formulations are proposed in the literature and their relaxations can be separated into two main groups: linear programming (LP) relaxation and semidefinite programming (SDP) relaxation.
2.2.1 Integer programming formulations
The max-k-cut has essentially three different integer formulations: One formulation with edge-only variables, another with both node-and-edge variables and a third with node-edge-only variables. We present all of these formulations.
Edge-only formulation
A 0-1 edge formulation is proposed in [10], where the integer variable xij for each i, j ∈ V is defined as:
xij =
0 if edge (i, j) is cut, 1 otherwise.
Hence, from an edge perspective, the max-k-cut is formulated as: (EDGE) max x X i,j∈V,i<j wij(1 − xij) (2.1) s.t. xih+ xhj − xij ≤ 1 ∀i, j, h ∈ V , (2.2) X i,j∈Q,i<j xij ≥ 1 ∀Q ⊆ V with |Q| = k + 1, (2.3) xij ∈ {0, 1} ∀i, j ∈ V . (2.4)
where Constraint (2.2) and (2.3) are called triangle and clique inequalities, respectively. Triangle Inequalities (2.2) correspond to the logical conditions that if the edges (i, h) and (h, j) are not cut then edge (i, j) cannot be cut. The Constraints (2.3) impose that at least one edge in a clique with k + 1 vertices cannot be cut.
Node-and-edge formulation
Also in [10], the authors present the node-and-edge formulation that has |V |k + |E| variables and constraints. For each v, i, j ∈ V , for each (i, j) ∈ E and p ∈ {1, . . . , k}. The binary variables are:
xij =
0 if edge (i, j) is cut,
1 otherwise. yvp = 1 if vertex v is in partition p, 0 otherwise.
Therefore, the node-and-edge integer formulations of max-k-cut is defined as: (No-Ed) max x X (i,j)∈E,i<j wij(1 − xij) (2.5) s.t. k X p=1 yvp = 1 ∀v ∈ V , (2.6) xij ≥ yip+ yjp− 1 ∀((i, j) ∈ E, p = 1, . . . , k), (2.7) xij ≤ yip− yjp+ 1 ∀((i, j) ∈ E, p = 1, . . . , k), (2.8) xij ≤ −yip+ yjp+ 1 ∀((i, j) ∈ E, p = 1, . . . , k), (2.9) xij ∈ {0, 1} ∀(i, j) ∈ E, (2.10) yvp∈ {0, 1} ∀(v ∈ V, p ∈ {1, . . . , k}). (2.11) where Inequalities (2.8)-(2.9) can be removed when all edges weight are non-negative.
Node-only formulation
The last formulation is based on (vertex) node variables. In [25], the authors mention that just allowing the variables to be xi ∈ {1, . . . , k} for each i ∈ V does not generate useful integer formulations. Instead, if xi is one of the k vectors a1, a2, . . . , akin Rk−1 such that the dot product of this vector is:
ai· aj = −1 k − 1
it provides a convenient quadratic formulation. In Lemma 4 of [25], it is proved that the value −1/(k − 1) gives the best angle separation for k vectors. Then, for xi ∈ {a1, . . . , ak} we have:
xixj =
−1
k−1 if xi 6= xj (i.e, vertices i and j are in different partitions), 1 if xi = xj.
Finally, we end up with the following quadratic formulation of the max-k-cut:
(N ODE) max x (k − 1) k X i,j∈V,i<j wij(1 − xixj) (2.12) s.t. xi ∈ {a1, a2, . . . , ak} ∀i ∈ V . (2.13)
2.2.2 Linear programming relaxation
Linear programs are the main field of operational research. In a linear programming formulation, all parameters of the model are known with certainty, all the variables are real, and all expressions (constraints and the objective function) are linear, i.e., a linear programming respects the following three assumptions: deterministic property, divisibility, and linearity [18].
The simplex algorithm [13] was, during many years, the unique method available to solve the LP. Although simplex is, in theory, a non-polynomial algorithm (it can make exponential steps to attain optimality), it is, in practice, very reliable and efficient. Nowadays, the polynomial interior point method (IPM) has shown to be a good alternative to the simplex method [34,58]. In fact, in [34,61] the authors show that the IPM enables the solution of many large-scale real-life problems and that IPM can exploit parallelism quite well.
Typically, a linear formulation is derivate from an integer programming formulation by relaxing the integrality constraint, i.e., in the LP we replace the constraint x ∈ {0, 1} by its relaxed form 0 ≤ x ≤ 1.
Edge-only relaxation. By relaxing the Integrality Constraint (2.4) in the EDGE formulation by
0 ≤ xij ≤ 1 ∀i, j ∈ V . (2.14)
we end up with the linear edge formulation of max-cut. This LP relaxation is used in the k-equipartition problems, see e.g. [60,72] and in [85] the authors use this relaxation to exploit sparsity in the max-k-cut problem. They show that it is not necessary to add dummy edges (of zero weight) if the graph is chordal.
We can notice that this formulation suffers from huge number inequalities: in a complete graph it can have 3|V |3triangle Inequalities (2.2) andk+1|V |clique Inequalities (2.3).
The edge-only formulation also suffers for not exploiting the structure of graphs like sparsity. In general, a chordal extension [39] of a sparse graph still adds a lot of dummy edge.
Node-and-edge relaxation. Replacing the Constraint (2.10) and (2.11) by their correspondent linear relaxations defines the linear node-and-edge formulation of the max-k-cut problem. This formulation is applied in the bounding procedure of [22] for a two-level graph partitioning prob-lem. In [22] the authors show that this relaxation is very weak and it suffers from symmetry. This formulation is further improved by the so-called representative formulations [2] where a represen-tative variable is added to break symmetry.
Both linear formulations presented above are investigated in Chapter 5 where we present some advantages and drawbacks. In summary, the LP formulation can rapidly be solved, but the bounds of LP are not strong [31], i.e., the linear solution is far from being an optimal solution.
2.2.3 Semidefinite programming relaxation
Semidefinite programming is one of the most active research areas in optimization [70]. In [73] the author points out that some N P-complete combinatorial problems can benefit from semidefi-nite optimization because it is solved in polynomial time (with fixed precision), and it can handle quadratic constraints in its model.
SDP refers to the problem of optimizing a linear function over the intersection of the cone of positive semidefinite (psd) matrices with an affine set. A matrix M is symmetric (M ∈ Sn) if MT = M and, in particular, M is a square matrix with dimension n. The matrix M ∈ Sn is psd (M ∈ Sn+, M 0) if:
It is known that M ∈ Sn+ if and only if all principal submatrix of M are also psd. Moreover, a diagonal element miifrom M is always among the elements of largest absolute value, i.e. for all i ∈ {1, . . . , n} the element mii = max{|mij| : j ∈ {1, . . . , n}}. Hence, if mii= 0 all elements in column i and row i of M ∈ S+
n are also 0. Others properties of psd are detailed in [40]. In SDP, the matrix variable X ∈ S+
n replaces the vector x ∈ Rn+of variables in the LP formulation,
i.e., the cone of the nonnegative orthant x ≥ 0 in the LP is replaced by the cone of semidefinite ma-trices X 0. The linear programming is a special case of SDP where all non-diagonal elements of X are zero. Another similarity with LP is that SDP is convex, and duality theory of LP generalizes naturally to SDP, in particular, the characterization of optimality is ensured if there exist feasible points in the interior of primal and dual formulations [41].
Since the studies of [3,66], the interior point method (IPM) is the most efficient method for solving semidefinite programming. The IPM iterates inside the SDP cone and it converges very fast by using Newton’s method. However, the computation of each iteration is often too high for practical applications with many constraints.
Others, less accurate methods, are proposed to solve the SDP faster. The spectral bundle method proposed in [43] reformulate the SDP as an eigenvalue optimization problem, in this way, the bundle method avoid the expensive calculation of Cholesky factorization by using a first order method that depends only on the calculation of eigenvalues and eigenvectors. However, for the spectral bundle, there is no polynomial bound on the number of arithmetic operations.
Another way of solving the SDP is by transforming SDP in a semi-infinite programming prob-lem [49,50]. In summary, this approach replaces the cone X 0 by Constraint (2.15). In Chapter 5 we study in detail this approach to formulate a new family of inequalities for the linear edge-only relaxation of the max-k-cut problem.
Node-only relaxation. The SDP relaxation is based on the quadratic node-only formulation of the max-k-cut (N ODE). To obtain a relaxation of the quadratic formulation we replace the con-straint (2.13) to:
xi ∈ Bn ∀i ∈ V . (2.16)
where Bn is the unit sphere in n dimensions. Thus, to avoid xi · xj = −1 we add the constraint xi· xj ≥ k−1−1 . Thereby, replacing xi· xj for Xij gives the SDP formulation proposed in [25]:
(SDP ) max X (k − 1) k X i,j∈V,i<j wij(1 − Xij) (2.17) s.t. Xii= 1 ∀i ∈ V , (2.18) Xij ≥ −1 k − 1 ∀i, j ∈ V, i < j, (2.19) X 0. (2.20)
Although this formulation has only a few constraints, the n(n−1)2 Constraints (2.19) can impact a lot the computing time of the SDP. For instance, in [40] the author indicates that it is more efficient to start only with Constraint (2.18) and to separate Xij ≥ −1
k−1 successively in a cutting plane algorithm.
2.2.4 SDP versus LP relaxations
Both SDP and LP formulations have their strength and weakness. We see that IPM and simplex methods are well suitable to provide fast solutions for large linear problems, but in [31] the authors affirm that the LP relaxations are weak and it could result in the enumeration of all the solutions in a branch-and-bound method.
In [19] the authors study the relation between the LP and SDP polytopes. They show that SDP relaxations of max-k-cut violates at most√2 − 1 of all triangle Constraint (2.2) and it violates at most 12 of all clique constraints, in comparison with a violation of 1 for the LP relaxation.
Moreover, in [4] the authors affirm that computationally speaking, the strength of the SDP relax-ations come with the cost of expensive running times.
2.3 Strengthening max-k-cut formulations
A relaxation can be tightened, without removing any feasible solution, by the addition of cutting planes (also called valid inequalities). In order to strength even more the LP and the SDP relaxations of the max-k-cut problem, some researchers have proposed some valid inequalities.
A cutting plane algorithm (CPA) is designed to solve a problem by, iteratively, adding only the most important inequalities. Therefore, this section presents some valid inequalities and introduces the CPA.
2.3.1 Valid inequalities
Typically, the triangle (2.2) and clique (2.3) inequalities of the edge-only formulation are also used to strengthen the SDP formulations. For example, in [81], it is shown that triangle inequalities are stronger than clique inequalities in the SDP relaxation.
In [9], the authors propose several valid and facet-defining inequalities for k-partition problem polytopes. Particularly, they use the edge-only linear relaxation to demonstrate the validity of the following inequalities:
• the General clique inequalities are generalizations of the Clique Inequalities 2.3, and they are valid if it has more than k vertices and it is facet-defining if the size of this inequality (number of vertices in it) is not a module of k,
• the wheel inequalities are facet-defining if the main cycle in the structure of this inequality is odd and if k ≥ 4, and
• the bicycle wheel inequalities are facet-defining for k ≥ 4 if the cycle is odd.
Several valid and facets-defining inequalities for the max-cut problem is studied in [16]. The results of [16] are generalized for the max-k-cut problem in [10] where some sufficient condition for hypermetric, cycle and anti-web inequalities are proposed.
In [21], a new family of inequality for the node-and-edge relaxation called projected clique in-equalities is obtained by the projection of edge-only formulation to node-and-edge formulation. Computational results show that the new inequalities are of practical use in the case of large sparse graphs.
In Chapter 4 we review in more detail some of these valid inequalities, especially the ones proposed in [9].
2.3.2 Cutting plane algorithm
A CPA is designed to find optimal solutions of different types of problems or to tighten bounds [69,
87]. The CPA is a standard technique applied to problems when the number of inequalities is too large to be explicitly represented in the model [36]. For the max-k-cut, the CPA is, usually, used to separate triangle and clique inequalities see e.g. [4,31].
In summary, the CPA starts with a basic relaxation. At each iteration, it solves the problem, then some separation routines search for violated cut planes (inequalities) and the most violated con-straints are added to the relaxation. Then, some unimportant concon-straints (with large slack variables)
can also be removed. Finally, if some cut planes were added a new iteration is started, otherwise, we stop the CPA.
Separation routines are exact or heuristic methods applied to search for violated inequalities in a relaxed solution. For the max-k-cut problem, in [4, 31] a greedy heuristic is applied to search for clique inequalities and an exact enumeration to search for triangle inequalities. In [15] the authors show that the separation of odd cycles can be done in polynomial time for wheel and bicycle wheel inequalities. For the projected inequalities of [21], the author proposes some exact and heuristic methods for the searching.
A variation of the CPA that uses IPM solvers is applied to the PDCGM solver [35] and in [58]. The technique employed is called the early-termination, where the IPM is stopped before it attains optimality to search for cut planes that are violated. Normally, the inequalities are stronger when the early-termination is applied [61].
A formal and more detailed description of CPA is provided in Chapters 4 and 5.
2.4 Methods to solve the max-k-cut problem
In this section, we present some approximations, heuristics and exact methods proposed in the literature to solve the max-k-cut problem.
2.4.1 Approximations methods
In [33] the famous 0.878-approximation algorithms for the max-cut problem is proposed. The main idea of this approximation is to solve an SDP formulation and use a randomized rounding heuristic to obtain a good solution. In this approximation, a random hyperplane that passes through the origin is chosen, and we partition a vertex vi ∈ V according to the side of the hyperplane that it falls.
In [25] and [47] the authors extend the approach of [33] to the max-k-cut. In [25], it is shown that there exist a sequence of constants αksuch that:
H(w(Pk)) ≥ αkw(Pk∗)
where Pk∗denotes the optimal solution, w(Pk∗) is the value of Pk∗, and H denotes the expected value of their approximations. In [25, Theorem 1], the authors show that the constants αksatisfies :
• α2 ≥ 0.878567, α3 ≥ 0.800217, α4 ≥ 0.850304, α5 ≥ 0.874243, α10 ≥ 0.926642, and α100 ≥ 0.990625.
For small values of k, the SDP approximation proposed in [14] gives better results than [25]. For example, if k = 3 they improved the bound to 0.832718 and for k = 4 to 0.857487. In [14] the rounding procedure of [25] is applied in a lifted matrix of the SDP solutions.
2.4.2 Heuristic methods
As opposed to exact and approximations method where the first guarantees optimality and the second a solution that cannot be worse than an α approximation, the heuristic and meta-heuristic methods attempt to yield good solutions with any guarantee. Typically, the heuristic is much faster than exact methods, and the heuristic solutions are normally better than the ones of approximation methods.
For the max-cut (k = 2) problem, many heuristic algorithms have been proposed (see e.g., [89,48]). For the max-k-cut, on the other hand, there are much fewer methods proposed. Therefore, we present all heuristic that we are aware of.
An iterative clustering heuristic (ICH) was proposed in [31] to finds feasible solutions from an SDP solution. Their computational results show that ICH provides better solutions than those obtained by [25]. The ICH works by building subgraphs based on the information from an SDP solution (X∗).
In [90] it is introduced a multi-start-type algorithm called dynamic convexized (DC) method where a local search algorithm is applied to a dynamically updated auxiliary function.
In [56] the authors present a multiple operator heuristic (MOH). The MOH is an iterative method that applies five search operators in three search phases. They show that their MOH provides better bounds in less time than DC method in 90% of their tests.
A more detailed review of some of these heuristic methods is presented in Chapter 6, where we also compare their efficiency.
2.4.3 Exact methods
Exact methods find optimal solutions for the max-k-cut problem. Most of the methods are based on the branch-and-bound algorithm. Both algorithms use a tree search strategy to implicitly enumerate all possible solutions. The difference between these methods is that the branch-and-cut uses a cutting plane algorithm in all nodes of the three search. We investigate in details the branch-and-bound framework in Chapter 5.
In [60, 59] a branch-and-cut algorithm based on edge-only relaxation is applied to the k-way equipartition problem to solve the realignment problem of a football league where k = 8 and |V | = 32. The branch-and-cut uses an LP formulation that is reinforced by triangle and clique inequalities. For problems with 100 to 500 vertices, they have found a percentage gap inferior to 2.5% for k = 4.
A branch-and-bound algorithm with cutting plane algorithm in the root node (called cut-and-branch algorithm) is applied in [22] for the two-level graph partitioning problem. The linear relaxation is based on the node-and-edge formulations of max-k-cut. The input graph is simplified by a preprocessing stage, and the LP relaxation is tightened by the triangle, clique, and general clique based inequalities. Computational results found the optimal solution for problems with |V | = 100 and k ∈ {2, 3, 4} for sparse graphs.
A branch-and-bound method based on the edge-only formulation of the max-k-cut is investigated in [85], where it is shown that a chordal graph can be formulated with only |E| variables instead of
n(n−1)
2 . In [85], sparse problems with |V | = 200 and k ∈ {3, 4} are solved in few seconds.
In [31] the semidefinite relaxation is used in a branch-and-cut algorithm called SBC. The SBC uses triangle and clique inequalities, and the SDP is solved by the spectral bundle method, more-over, the ICH heuristic is applied to get feasible solutions. The SBC algorithm computes optimal solutions for dense graphs with 60 vertices, and for sparse graphs with 100.
The BundleBC algorithm proposed in [4] is an improvement of the SBC method. BundleBC extends the ideas of the Biq Mac solver [74] to max-k-cut. The importance of separating clique inequalities with triangle inequalities is also investigated in BundleBC. Feasible solutions are computed via the approximation algorithm proposed in [25]. Computational results show that the use of the clique inequalities reduces the number of subproblems analyzed in the branch-and-bound tree, and the BundleBC is shown to be faster than SBC. The BundleBC is able to solve some dense instances with 70 nodes and sparse with 100, for k ∈ {3, 5, 7}.
2.5 Additional reviews
To close this chapter we refer to the reader the additional literature review on the cutting plane algorithm in Chapter 4. In Chapter 5, we review the semi-infinite formulations especially for the linear formulation of SDP. Finally, in Chapter 6, we review in detail all the components of the branch-and-bound method.
CHAPTER 3 ORGANIZATION OF THE THESIS
In this chapter, we present the organization of the research done as part of this thesis. The motiva-tions and the results initially hoped for are also explained.
In the previous chapter, we presented the state-of-art of the max-k-cut literature. We observe that some methods are proposed to exactly solve the max-k-cut. In particular, the SBC [31] and the BundleBC[4] solvers use the SDP formulations of max-k-cut to obtain upper bounds in a branch-and-cut framework. However, both methods are unable to solve instances with more than 70 ver-tices.
Several studies show that, on one hand, the SDP yields good bounds, but, on the other hand, it suffers from expensive running times [4,19]. Based on this, we aim to design a formulation that provides a trade-off between quality and time of solution, and we apply this formulation within an efficient branch-and-bound algorithm.
In Chapters 4 and 5, we report our articles [76] and [77]. In Chapter 6, we present an investigation of an exact method for the max-k-cut. The chapters are presented in a chronological way. Each part of the thesis focuses on improving solutions to the max-k-cut problem. We present the steps performed at each one of the three parts of this thesis.
The first article [76] was accepted to be published on Annals of Operations Research in March 2017, and is reported in Chapter 4. This article investigates some families of inequalities to strengthen the max-k-cut relaxation. These inequalities studied were proposed by [9] and they are known to be facet-defining. Specifically, we considered the five following classes of cutting planes: the Triangle, Clique, General clique, Wheel and Bicycle wheel inequalities. Due to the fact that each family of inequality can have a huge number of rows, we design a cutting plane algorithm (CPA) to add only the most important inequalities. Therefore, for each class, we study several separation methods. For example, for the wheel and bicycle wheel inequalities, we study four separation methods for finding violated inequalities in an iteration of CPA. In order to investigate the impact of each class, we create ten combination of methods that alternate the activation and deactivation of each family. Moreover, we study dense and sparse graphs, and we compare our combinations using different benchmarks. Hence, we intend to identify the most important classes of inequalities to be prioritized in the max-k-cut formulation.
The second article [77] submitted in April 2018 on EURO Journal on Computational Optimization is presented in Chapter 5. We proposed a family of inequalities to strengthen the LP relaxation based on the SDP formulation. These inequalities have infinity rows and we design an exact
sepa-ration routine based on eigenvalue of the relaxed solution. Moreover, we employ a modified CPA to find deeper cuts. In order to compare the SDP with the new LP formulation we test both methods for several instances.
Chapter 6 focuses on designing an efficient exact method to obtain global solutions of the max-k-cut problem. This chapter is the conclusion of our study where we include ideas of the two previous works [76,77] in a branch-and-bound algorithm. We investigate five of the most important components of the branch-and-bound method: upper-bound, feasibility, splitting, branching, and node selection. For each procedure, we computationally study several strategies. For example, in the branching rule procedure we study five different rules for selecting a variable in a node of the branch-and-bound tree. Moreover, we show the evolution of the LP formulations (from [76] to [77]) and we also compare both LP and SDP formulations in the branch-and-bound scheme for a variety of instances. This way, an efficient exact method is designed.
CHAPTER 4 ARTICLE 1: COMPUTATIONAL STUDY OF VALID INEQUALITIES FOR THE MAXIMUM K-CUT PROBLEM
Authors: Vilmar J. Rodrigues de Sousa, Miguel F. Anjos, Sébastien Le Digabel. Accepted for publication: Annals of Operations Research
Abstract We consider the maximum k-cut problem that consists in partitioning the vertex set of a graph into k subsets such that the sum of the weights of edges joining vertices in different subsets is maximized. We focus on identifying effective classes of inequalities to tighten the semidefinite programming relaxation. We carry out an experimental study of four classes of inequalities from the literature: clique, general clique, wheel and bicycle wheel. We considered 10 combinations of these classes and tested them on both dense and sparse instances for k ∈ {3, 4, 5, 7}. Our compu-tational results suggest that the bicycle wheel and wheel are the strongest inequalities for k = 3, and that for k ∈ {4, 5, 7} the wheel inequalities are the strongest by far. Furthermore, we observe an improvement in the performance for all choices of k when both bicycle wheel and wheel are used, at the cost of 72% more CPU time on average when compared with using only one of them.
Keywords. Maximum k-Cut, Graph Partitioning, Semidefinite Programming, Computational Study.
AMS subject classifications. 65K05, 90C22, 90C35.
4.1 Introduction
Graph partitioning problems (GPPs) are an important class of combinatorial optimization. There are various types of GPP based on the number of partitions allowed, the objective function, the types of edge weights, and the possible presence of additional constraints such as restrictions on the number of vertices allowed in each partition. This work considers the GPP known as the maximum k-cut (max-k-cut) problem: Given a connected graph G = (V, E) with edge weights wij for all (i, j) ∈ E, partition the vertex set V into at most k subsets so as to maximize the total weight of cut edges, i.e., the edges with end points in two different subsets. This problem is sometimes also called minimum k-partition problem [31]. The special case of max-k-cut with k = 2 is known as the max-cut problem and has received most attention in the literature, see e.g. [6,16,33,51,70,74].
Graph partitioning and max-k-cut problems have myriad applications as VLSI layout design [6], sports team scheduling [60], statistical physics [52], placement of television commercials [29], network planning [19], and floorplanning [12].
The unweighted version of max-k-cut is known to be N P-complete [71]. Hence, researchers have proposed heuristics [11, 56], approximation algorithms [11, 25], and exact methods [4, 31] for solving the max-k-cut. In particular the approximation method proposed in [25] extends the famous semidefinite programming (SDP) results for max-cut in [33] to max-k-cut. The authors of [14] subsequently improved the approximation guarantees for small values of k.
Ghaddar et al. [31] propose the SBC algorithm. It is a branch-and-cut algorithm for the minimum k-partition problem based on the SDP relaxation of max-k-cut. They found experimentally that for k > 2 the SDP relaxation yields much stronger bounds than the LP relaxation, for both sparse and dense instances. This algorithm was improved in [4] which introduced the bundleBC algorithm to solve max-k-cut for large instances by combining the approach of [31] with the design principles of Biq Mac [74]. The results in [4] confirm that, computationally speaking, the SDP relaxations often yield stronger bounds than LP relaxations.
The matrix lifting SDP relaxation of the GPP, and the use the triangle and clique inequalities to strengthen it, were studied in [81, 83]. In particular [81] shows that this relaxation is equivalent to the relaxation in [25] and is as competitive as any other known SDP bound for the GPP. The results also show that the triangle inequalities are stronger than the clique (independent set) inequalities. Using a similar approach as [81], the authors in [84] propose several new bounds for max-k-cut from the maximum eigenvalue of the Laplacian matrix of G. Moreover, [84] shows that certain perturbations in the diagonal of the Laplacian matrix can lead to stronger bounds. In [67] the author proposes a more robust bound than the one proposed in [84] by using the smallest eigenvalue of the adjacency matrix of G.
In [21] the authors project the polytopes associated with the formulation with only edge variables into a suitable subspace to obtain the polytopes of the formulation that has both node and edge variables. They then derive new valid inequalities and a new semidefinite programming relaxation for the max-k-cut problem. The authors suggest that the new inequalities may be of practical use only in the case of large sparse graphs.
The objective of this work is to carry out a computational study of valid inequalities that strengthen the SDP relaxation for the max-k-cut problem, and to identify the strongest ones in practice. Valid and facet-inducing inequalities for k-partition were studied in [9,10]. We focus our analysis on the following inequalities: Triangle, Clique, General clique, Wheel and Bicycle wheel. To the best of our knowledge, no research has specifically focused on comparing the strength of the range of
inequalities considered here.
This paper is organized as follows. Sections 5.1.1 and 4.1.2 review the SDP formulation and relax-ation of the max-k-cut problem. Section 5.2 presents the classes of inequalities and discusses the separation methods used. Section 4.3 describes the cutting plane algorithm used, and Section 5.4 presents and discusses the test results. The conclusions from this study are provided in Section 6.7.
4.1.1 Problem formulation
We assume without loss of generality that the graph G = (V, E) is a complete graph because missing edges can be added with a corresponding weight of zero. Let k ≥ 2 and define the matrix variable X = (Xij), i, j ∈ V , as: Xij = −1
k−1 if vertices i and j are in different partitions of the k-cut of G, 1 otherwise.
The max-k-cut problem on G can be expressed as:
max X X i,j∈V,i<j wij (k − 1)(1 − Xij) k (4.1) s.t. Xii= 1 ∀i ∈ V , (4.2) Xij ∈ ( −1 k − 1, 1 ) ∀i, j ∈ V, i < j, (4.3) X 0. (4.4)
We refer to this formulation as (MkP). This formulation was first proposed by [25]. As mentioned in [25, Lemma 4], the value −1/(k − 1) gives the best angle separation for k vectors.
4.1.2 Semidefinite relaxation
Replacing Constraint (4.3) by k−1−1 ≤ Xij ≤ 1 defines the SDP relaxation. However, the constraint Xij ≤ 1 can be removed since it is enforced implicitly by the constraints Xii = 1 and X 0.
Therefore the SDP relaxation, denoted (RMkP), is: max X X i,j∈V,i<j wij (k − 1)(1 − Xij) k (4.5) s.t. Xii= 1 ∀i ∈ V , (4.6) Xij ≥ −1 k − 1 ∀i, j ∈ V, i < j, (4.7) X 0. (4.8)
4.2 Formulation and separation of inequalities
The SDP relaxation (RMkP) can be tightened by adding valid inequalities, i.e., cutting planes that are satisfied for all positive semidefinite matrices that are feasible for problem (MkP) but are violated by solution X0 of the (current) SDP relaxation. In this section we present the classes of inequalities from [9,10] that we consider adding to tighten the SDP relaxation. For each class, we describe its properties, its formulation in terms of the SDP variable X, and the separation routine used.
The first two classes of inequalities (Triangle and Clique) were used in the SBC branch-and-cut method [31] and in the matrix lifting approach [81,83]. We are unaware of the other classes having been computationally tested or used.
4.2.1 Triangle inequalities
Triangle inequalities are based on the observation that if vertices i and h are in the same partition, and vertices h and j in the same partition, then vertices i and j necessarily have to be in the same partition. The resulting 3|V |3 triangle inequalities are:
Xih+ Xhj − Xij ≤ 1 ∀i, j, h ∈ V . (4.9)
Separation oftriangle inequalities
As pointed out in [4], the enumeration of all triangle inequalities is computationally inexpensive, even for large instances. Therefore, a complete enumeration is done to find triangles in the graph G that violate (4.9). The tolerance used to detect a violation is 10−5.
4.2.2 Clique inequalities The |V | k + 1
clique inequalities ensure that for every subset of k + 1 vertices, at least two of the
vertices belong to the same partition. We express them as:
X
i,j∈Q,i<j
Xij ≥ − k
2 ∀Q ⊆ V with |Q| = k + 1. (4.10)
Separation ofclique inequalities
The exact separation of clique is N P-hard in general, and the complete enumeration is intractable even for small values of k [31]. We use the heuristic described in [31] to find a violated clique inequality for a solution X0of the SDP relaxation. This heuristic is described here as Algorithm 1, and returns up to |V | clique inequalities. We use a tolerance ε = 10−5.
for all j ∈ V do C = {j};
while |C| ≤ k do
Select u ∈ V \C such that u ∈
( arg min v X i∈C Xi,v0 | v ∈ V \C ) ; C = C ∪ {u}; end if X i,z,∈C,i<z Xiz0 < −k 2 − ε then Q = C;
Construct an inequality of type (4.10); end
end
Algorithm 1: Heuristic to find violated clique inequalities.
4.2.3 General clique inequalities
The General clique inequalities in [9] are a generalization of the Clique inequality (4.10).
Let Q = (V (Q), E(Q)) be a clique of size p := tk + q where t ≥ 1 and 0 ≤ q < k, i.e., t is the largest integer such that tk ≤ p. In terms of the SDP variable X, the general clique inequalities have the form:
X i,j∈Q Xij ≥ k k − 1 " z + w k − 1 k − 1 !# ∀Q ⊆ V, |Q| = p. (4.11)
where z = 12t(t − 1)(k − q) + 12t(t + 1)q and w = p(p−1)2 . It is proved in [9] that (4.11) is valid for the polytope of the max-k-cut problem if p ≥ k, i.e., t ≥ 1. Furthermore, it is facet-defining if and only if 1 ≤ q ≤ k − 1, t ≥ 1, i.e., p is not an integer multiple of k.
Separation ofgeneral clique inequalities
Because the clique inequalities are included in general clique, it is clear that enumeration is also impracticable for general clique. Thus, we propose a heuristic to find the most violated general clique inequalities for X0.
Our heuristic is a small modification of the one used to separate the clique inequalities (Algo-rithm 1). Specifically:
• The argument in the WHILE condition becomes |C| ≤ p − 1. • The inequality in the IF condition is replaced by
w(j) < k k − 1 " z + w k − 1 k − 1 !# − ε ! .
The parameter p may vary with the number of partitions k. After preliminary tests using p ∈ {5, 7, 8, 10, 11, 13} for k = 3, p ∈ {6, 7, 9, 11, 13, 14} for k = 4, p ∈ {7, 9, 11, 13, 16, 18} for k = 5, and p ∈ {9, 11, 13, 16, 18, 20} for k = 7, we settled on with the following choices: p = 5 for k = 3; p = 6 for k = 4; p = 7 for k = 5; and p = 11 for k = 7.
4.2.4 Wheel inequalities
In [9], the q-wheel is defined as Wq = (Vq, Eq), where • Vq = {v, vi|i = 1, 2, ..., q}
• Eq = ¯E ∪ Ê, where ¯E = {(v, vi), i = 1, 2, ..., q} and Ê = {(vi, vi+1), i = 1, 2, ..., q}.
where v ∈ V , and all indices greater than q are modulo q. For q = 6, W6is illustrated in Figure 4.1.
Let Wq be a subgraph of G for q ≥ 3. In terms of the SDP variable X, the wheel inequality is defined as: q X i=1 Xv,vi− q X i=1 Xvi,vi+1 ≤ 1 2q k k − 1. (4.12)
v1 v2 v3 v4 v5 v6 v
Figure 4.1 Example of wheel with q = 6.
The idea of this inequality is that vertex v can be in the same partition as vertices viand vi+1if edge e = (vi, vi+1) is not cut. It is proved in [9] that (4.12) is a valid inequality for max-k-cut, and that it defines a facet if q is odd and k ≥ 4.
Separation ofwheel inequalities
By Inequality (4.12), a q-wheel is violated by X0 if: q X i=1 Xv,v0 i− q X i=1 Xv0i,vi+1− 1 2q k k − 1 ! > 0. (4.13)
For each vertex v ∈ V , we wish to maximize the left-hand side of Inequality (4.13) to find the most violated wheel inequality.
In [15] it is shown that the odd wheel inequalities can be separated in polynomial-time. We also considered four heuristics, for a total of five methods tested:
• Exact algorithm [15].
• The GRASP meta-heuristic [23]. • A genetic algorithm (GA) from [80].
• The K-step greedy lookahead [78] with K=2.
• Our own greedy heuristic for multiple sizes of wheel (GMSW), described in Algorithm 2. The GMSW heuristic achieved, on average, the best ratio between CPU time and performance for all tests. For instance, the performance ratio (see Section 5.4.3) was less than 0.03% higher than the exact method. In addition, its CPU time was in average 45% less. In GSMW, the parameter
for all j ∈ V do i = 1; vi = j; W = {vi}; while i ≤ NbIte do q = |W |; z = 0; if q ≥ 3 then Select v ∈ V \W s.t.: z = maxvPqi=1Xv,v0 i − Pq i=1Xv0i,vi+1− b1 2qc k k−1 ; end if z > 0 then
Build Inequality (4.12) with W and v; j = j + 1;
i = NbIte + 1 ; else
Select u ∈ V \W such that u ∈ {arg min v X 0 vq,v+ X 0 v,v1 | v ∈ V \W }; i = i + 1; vi = {u}; W = W ∪ {vi}; end end end
Algorithm 2: The GMSW heuristic to find violated wheel inequalities.
NbIte ≤ |V | − 1 is the maximum size allowed for the wheel. We obtained the best results using NbIte = 2k when 2k ≤ |V | − 1.
4.2.5 Bicycle wheel inequalities
The fourth class of inequalities corresponds to a q-bicycle wheel BWq = (Vq, Eq) subgraph of G, where:
Vq = {u1, u2} ∪ {vi, i = 1, 2, ..., q}, Eq = E1∪ E2∪ {¯e} ,
E1 = {(ui, vj), i = 1, 2, j = 1..., q}, E2 = {(vj, vj+1), j = 1..., q} , ¯e = {(u1, u2)},
v1 v2 v3 v4 v5 u1 u2 edges ∈ E1 edges ∈ E2 edge ¯e
Figure 4.2 Example of bicycle wheel with q = 5.
For BWq with q ≥ 3, the corresponding bicycle wheel inequality is defined as:
2 X i=1 q X j=1 Xui,vj− q X j=1 Xvj,vj+1− Xu1,u2 ≤ k k − 1 2b1 2qc − q + 1 + q − 1. (4.14)
where v ∈ V and all indices greater than q are modulo q. It is proved in [9] that the bicycle wheel inequality is a valid inequality to (MSkP), and for k ≥ 4 it induces a facet if q is odd and BWq is an induced subgraph of G.
The idea behind bicycle wheel inequalities is that at least one edge e ∈ E1 must be cut for each
edge from ¯e or E2that is cut.
Separation ofbicycle wheel inequalities
In [15] is pointed out that the separation for the odd bicycle wheel inequalities (4.14) can also be done in polynomial time, by adapting the approach in [30].
As we did for wheel separation, we tested also some heuristic method to find violated inequalities of the form (4.14) for a given X0.
The basic idea of those heuristics is:
• Search for a cycle that minimizes the sum
j≤q X j=1 Xvj,vj+1 .
• Find the best points u1and u2that maximize
Pi=1,2,j≤q
j,i=1 Xui,vj
and (−Xu1,u2).
For finding good cycles in the first step of this heuristic, we tested the same four heuristics men-tioned in Section 4.2.4. Our tests showed that the GA from [80] with q = 3 obtained the best results in 95% of the cases. Moreover, this heuristic had the same performance as the exact method for all tests, and its CPU time was 95% lower.
The GA has the following features: • Size of population is 100.
• Half of the initial population was constructed with the GRASP and other half was randomly selected.
• In crossover process the parents with higher fitness are favored. • Offspring genes can accept some slight mutations.
We keep the best cycle Cw found in the final population after 100 iterations of the GA. Using this cycle, the main heuristic then checks all the combinations of u1, u2 ∈ V \Cw to find the one that maximizes the right-hand side of (4.14).
4.3 Cutting plane algorithm
This section presents the cutting plane algorithm (CPA) used for our computational tests. This algorithm is similar to the one described in [31]. The basic idea is to start by solving the SDP problem (RMkP), then check for violated inequalities, add them to the SDP problem, and resolve to obtain a better upper bound on the global optimal value of the max-k-cut instance.
The pseudo-code of the CPA is given in Algorithm 3. We use the following notation: i is the iterations counter, Fi is the set of inequalities in iteration i, Xi is the optimal solution of the SDP relaxation at iteration i, and S(Xi) is the set of valid inequalities for (MkP) that are violated by Xi.
Initializations: i = 0, F0 = ∅. repeat
Xi = Solve (RMkP) with Fi;
S(Xi) = set of violated inequalities for Xi; if Constraint (4.3) is satisfied then
STOP; /* Xi is also optimal for (MkP) */
end
if S(Xi) == ∅ then
STOP; /* no violated inequalities */
else
Fi+1 = Fi∪ S(Xi); i = i + 1 ;
end until STOP;
The set S(Xi) of inequalities is obtained by the separation routines of triangle, clique, general clique, wheel and bicycle wheel inequalities described in Section 5.2. We say that a class of inequalities is active if its separation routine is applied to find violations in the relaxed solution at each iteration of the CPA.
The size of the set S(Xi) is controlled by the parameter NbIneq: |S(Xi)| ≤ NbIneq at each iteration of the CPA. After some tests with NbIneq ∈ {100, 300, 500, 1000, ∞}, the CPA with NbIneq = 300 was found to be the least expensive in terms of CPU time and CPA iterations for a variety of graphs and choices of k. This limitation in the CPA imposes that just the most violated inequalities of the set of activated families are added at each iteration.
4.4 Computational tests
The SDP relaxations of the max-k-cut were solved using the mosek solver [5] on a linux PC with two Intel(R) Xeon(R) 3.07 GHz processors.
Tests were performed for each value of k ∈ {4, 3, 5, 7} on 147 test instances. Our instances were taken from the Biq Mac Library [88] or generated using rudy [75].
Section 4.4.1 introduces the combinations of inequalities tested, and the instances used. Sec-tion 5.4.3 explains the benchmarking methodology used for analyzing the different algorithms, and Section 5.4.4 reports the computational results. Finally, a summary of our results is given in Section 4.4.4.
4.4.1 Combinations of inequalities and test instances
This section defines the ten combinations of active inequalities developed to analyze the perfor-mance of the four classes of inequalities, and the instances used in the tests.
Combinations of inequalities
This section presents ten combinations of classes of inequalities. Each combination differ from each other by the classes that are activated or deactivated when running the CPA described in Section 4.3.
The triangle inequalities have been extensively studied in the literature and their efficiency has been confirmed by various researchers [81,74,4]. For this reason, we activated them in all combinations. From the sixteen possible combinations of the remaining four classes of inequalities (clique, wheel, bicycle wheel and general clique), we used the ten combinations defined in Table 4.1
