HAL Id: hal-02944011
https://hal.archives-ouvertes.fr/hal-02944011
Submitted on 21 Sep 2020
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Distributed under a Creative CommonsAttribution - NonCommercial - ShareAlike| 4.0 International License
SEGOMOE: Super Efficient Global Optimization with Mixture of Experts
Rémy Priem, Nathalie Bartoli, Youssef Diouane, Thierry Lefebvre, Sylvain Dubreuil, Michel Salaün, Joseph Morlier
To cite this version:
Rémy Priem, Nathalie Bartoli, Youssef Diouane, Thierry Lefebvre, Sylvain Dubreuil, et al.. SEGO- MOE: Super Efficient Global Optimization with Mixture of Experts. Workshop CIMI Optimization
& Learning, Sep 2018, Toulouse, France. 2018, �10.13140/RG.2.2.14377.01120�. �hal-02944011�
SEGOMOE: Super Efficient Global Optimization with Mixture of Experts
Rémy Priem – Nathalie Bartoli (ONERA) – Youssef Diouane (ISAE-Supaero) – Thierry Lefebvre (ONERA) – Sylvain Dubreuil (ONERA) – Michel Salaün (ISAE-Supaero) – Joseph Morlier (ISAE-Supaero)
Contact : Rémy Priem – remy.priem@isae-supaero.fr / remy.priem@onera.fr
Bibliography
[1] D. R. Jones, et al., ‘Efficient global optimization of expensive black- box functions’, Journal of Global optimization, vol. 13, no. 4, pp. 455–
492, 1998.
[2] M. A. Bouhlel, et al., ‘Efficient global optimization for high- dimensional constrained problems by using the Kriging models combined with the partial least squares method’, Engineering Optimization, pp. 1–16, 2018.
[3] N. Bartoli et al., ‘An adaptive optimization strategy based on mixture of experts for wing aerodynamic design optimization’, in 18th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, 2017, p. 4433.
Context
New Aircraftconcepts No semi-empiric
models
Multidisciplinary Optimization
Expensive to evaluate black
box models
Surrogate Models
Surrogate model
Constraint criteria Success SEGO-M [6] 3/10 SEGO-U (𝜏 = 3) 10/10 SEGO-U (𝜏 log behavior) 10/10
Constraint Handling max 𝐼𝐶 𝑥
𝑥∈ℝ𝑑
𝑠. 𝑡.
𝑐 𝑥 ≥ 0
Add the uncertainty information on the constraints surrogate
models
𝑼 𝑥 = 𝑐 𝑥 + 𝜏𝜎 (𝑥),
𝐸𝐹 𝑥 = 𝑐 𝑥 Φ 𝜎 𝑥𝑐 𝑥 + 𝜎 𝑥 ϕ 𝜎𝑐 𝑥 𝑥 − 𝜏, 𝐴𝐸𝐹 𝑥 = 𝛽𝑐 𝑥 + 1 − 𝛽 𝐸𝐹 𝑥 − 𝜏,
New constraints handling criteria 𝐶𝐻𝐶
New optimization sub-problem
Find most probable areas and search in
uncertain areas
𝛽 ∈ 0,1 ϕ = pdf of 𝑁(0,1)
Φ = cdf of 𝑁(0,1)
𝑴 𝑥 = 𝑐 𝑥
max 𝐼𝐶 𝑥
𝑥∈ℝ𝑑
𝑠. 𝑡.
𝐶𝐻𝐶 𝑥 ≥ 0
𝜏 ∈ 0,3
Super Efficient Global Optimization with Mixture of Experts
Compute the Observation
Train the GP based models
max 𝐼𝐶 𝑥
𝑥∈ℝ𝑑
𝑠. 𝑡.
𝑐 𝑥 ≥ 0 Evaluate the
points Optimization
problem
𝑥∈ℝmin𝑑𝑓 𝑥 𝑠. 𝑡.
𝑐 𝑥 ≥ 0
IDEA: Use the surrogate models to perform the optimization
→ Identify the best points for the
optimization and learning of the model via an iterative enrichment process GP based models
prediction and error
SEGOMOE [3]
One enrichment step
Infill criteria 𝐼𝐶 that allows to choose the new point to
evaluate (EI, etc.) [1]
Take the constraints into account
Mixture of Experts
• Gaussian mixture model clustering
• Each point 𝑥 of the domain belong to each cluster 𝑘 with a probability 𝛼𝑘 𝑥
• One Gaussian Process (GP) model 𝑦 𝑘 𝑥 , 𝑠 𝑘 𝑥 trained by cluster
• KPLS & KPLS+K: GP mixed with PLS for large scale model [2]
• 𝑦 𝑥 = 𝛼𝑘 𝑘 𝑥 𝑦 𝑘 𝑥 Prediction of y
• 𝑠 2 𝑥 = 𝑘𝛼𝑘2 𝑥 𝑠 𝑘2 𝑥 Error estimation
Observations Clusters probabilities Cluster borders
Recombination
Training Clustering
Find and train expert models on changing behavior areas [3]
Optimizer Success SNOPT 5/15 (33%)
SEGO-M 18/18 (100%)
Results
Modified Branin function (2 DV, 1 C)
𝑓∗ = 12,005 Results on engineering test cases ADODG6 (17DV, 1C [3])
Aerodynamic wing shape constrained optimization
1 Global & 5 Local minima
local
Global
