Abstract

The ensemble of surrogate models is increasingly implemented in practice for its more flexibility and resilience than individual models. The optimization-based two-layer pointwise ensemble of surrogate models (OTL-PEM) is proposed in this paper as a novel pointwise ensemble of surrogate models. The framework of two-layer surrogate models is defined in the OTL-PEM, with data-surrogate models having several types of individual surrogate models fitting the given dataset. In contrast, the weight-surrogate models are modeled based on the cross-validation errors aiming to fit the pointwise weights of several individual surrogate models. To avoid the negative influence of the poor individual surrogate models, the model selection problem is transformed into several optimization problems, which can be solved easily using a sophisticated optimization algorithm to eliminate the globally poor surrogate models. In addition, the optimization space is extracted to reduce the prediction instability caused by weight-surrogate model extrapolation. More than 40 test functions are used to select the appropriate hyperparameters of the OTL-PEM and evaluate the OTL-PEM’s performance. The results show that the OTL-PEM can provide more accurate and robust approximation performance than individual surrogate models and other ensembles of surrogate models.

References

1.
Han
,
Z. H.
, and
Görtz
,
S.
,
2012
, “
Hierarchical Kriging Model for Variable-Fidelity Surrogate Modeling
,”
AIAA J.
,
50
(
9
), pp.
1885
1896
.
2.
Qian
,
J.
,
Yi
,
J.
,
Cheng
,
Y.
,
Liu
,
J.
, and
Zhou
,
Q.
,
2020
, “
A Sequential Constraints Updating Approach for Kriging Surrogate Model-Assisted Engineering Optimization Design Problem
,”
Eng. Comput.
,
36
(
3
), pp.
993
1009
.
3.
Zhu
,
Z.
, and
Du
,
X.
,
2016
, “
Reliability Analysis with Monte Carlo Simulation and Dependent Kriging Predictions
,”
ASME J. Mech. Des.
,
138
(
12
), p.
121403
.
4.
Chen
,
Z.
,
Peng
,
S.
,
Li
,
X.
,
Qiu
,
H.
,
Xiong
,
H.
,
Gao
,
L.
, and
Li
,
P.
,
2015
, “
An Important Boundary Sampling Method for Reliability-Based Design Optimization Using Kriging Model
,”
Struct. Multidiscipl. Optim.
,
52
(
1
), pp.
55
70
.
5.
Zhang
,
D.
,
Han
,
X.
,
Jiang
,
C.
,
Liu
,
J.
, and
Li
,
Q.
,
2017
, “
Time-dependent Reliability Analysis Through Response Surface Method
,”
ASME J. Mech. Des.
,
139
(
4
), p.
041404
.
6.
Hariri-Ardebili
,
M. A.
,
2018
, “
MCS-based Response Surface Metamodels and Optimal Design of Experiments for Gravity Dams
,”
Struct. Infrastruct. Eng.
,
14
(
12
), pp.
1641
1663
.
7.
Liu
,
X.
,
Liu
,
X.
,
Zhou
,
Z.
, and
Hu
,
L.
,
2021
, “
An Efficient Multi-Objective Optimization Method Based on the Adaptive Approximation Model of the Radial Basis Function
,”
Struct. Multidiscipl. Optim.
,
63
(
3
), pp.
1385
1403
.
8.
Li
,
K.
,
Liu
,
Y.
,
Wang
,
S.
, and
Song
,
X.
,
2021
, “
Multi-Fidelity Data Fusion Based on Gradient-Enhanced Surrogate Modelling Method
,”
ASME J. Mech. Des.
,
143
(
12
), p.
121704
.
9.
Clarke
,
S. M.
,
Griebsch
,
J. H.
, and
Simpson
,
T. W.
,
2004
, “
Analysis of Support Vector Regression for Approximation of Complex Engineering Analyses
,”
ASME J. Mech. Des.
,
127
(
6
), pp.
1077
1087
.
10.
Shi
,
M.
,
Lv
,
L.
,
Sun
,
W.
, and
Song
,
X.
,
2020
, “
A Multi-Fidelity Surrogate Model Based on Support Vector Regression
,”
Struct. Multidiscipl. Optim.
,
61
(
6
), pp.
2363
2375
.
11.
Hao
,
P.
,
Wang
,
B.
, and
Li
,
G.
,
2012
, “
Surrogate-based Optimum Design for Stiffened Shells with Adaptive Sampling
,”
AIAA J.
,
50
(
11
), pp.
2389
2407
.
12.
Bhattacharjee
,
K. S.
,
Singh
,
H. K.
, and
Ray
,
T.
,
2018
, “
Multiple Surrogate-Assisted Many-Objective Optimization for Computationally Expensive Engineering Design
,”
ASME J. Mech. Des.
,
140
(
5
), p.
051403
.
13.
Bichon
,
B. J.
,
Eldred
,
M. S.
,
Mahadevan
,
S.
, and
McFarland
,
J. M.
,
2013
, “
Efficient Global Surrogate Modeling for Reliability-Based Design Optimization
,”
ASME J. Mech. Des.
,
135
(
1
), p. 011009.
14.
Shi
,
R.
,
Long
,
T.
,
Baoyin
,
H.
,
Ye
,
N.
, and
Wei
,
Z.
,
2020
, “
Adaptive Kriging-Assisted Optimization of low-Thrust Many-Revolution Transfers to Geostationary Earth Orbit
,”
Eng. Optim.
,
53
(
12
), pp.
2040
2055
.
15.
Jin
,
R.
,
Du
,
X.
, and
Chen
,
W.
,
2003
, “
The use of Metamodeling Techniques for Optimization Under Uncertainty
,”
Struct. Multidiscipl. Optim.
,
25
(
2
), pp.
99
116
.
16.
Lim
,
D.
,
Ong
,
Y. S.
,
Jin
,
Y.
, and
Sendhoff
,
B.
,
2007
, “
A Study on Metamodeling Techniques, Ensembles, and Multi-Surrogates in Evolutionary Computation
,”
Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation
,
Boston, MA
,
July 9–13
, pp.
1288
1295
.
17.
Cheng
,
K.
, and
Lu
,
Z.
,
2020
, “
Structural Reliability Analysis Based on Ensemble Learning of Surrogate Models
,”
Struct. Saf.
,
83
, p.
101905
.
18.
Liu
,
H.
,
Xu
,
S.
,
Wang
,
X.
,
Meng
,
J.
, and
Yang
,
S.
,
2016
, “
Optimal Weighted Pointwise Ensemble of Radial Basis Functions with Different Basis Functions
,”
AIAA J.
,
54
(
10
), pp.
3117
3133
.
19.
Zerpa
,
L. E.
,
Queipo
,
N. V.
,
Pintos
,
S.
, and
Salager
,
J. L.
,
2005
, “
An Optimization Methodology of Alkaline–Surfactant–Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates
,”
J. Pet. Sci. Eng.
,
47
(
3–4
), pp.
197
208
.
20.
Bishop
,
C. M.
,
1995
,
Neural Networks for Pattern Recognition
,
Oxford University Press
,
Oxford
.
21.
Goel
,
T.
,
Haftka
,
R. T.
,
Shyy
,
W.
, and
Queipo
,
N. V.
,
2007
, “
Ensemble of Surrogates
,”
Struct. Multidiscipl. Optim.
,
33
(
3
), pp.
199
216
.
22.
Acar
,
E.
, and
Rais-Rohani
,
M.
,
2009
, “
Ensemble of Metamodels with Optimized Weight Factors
,”
Struct. Multidiscipl. Optim.
,
37
(
3
), pp.
279
294
.
23.
Ferreira
,
W. G.
, and
Serpa
,
A. L.
,
2016
, “
Ensemble of Metamodels: the Augmented Least Squares Approach
,”
Struct. Multidiscipl. Optim.
,
53
(
5
), pp.
1019
1046
.
24.
Viana
,
F. A.
,
Haftka
,
R. T.
, and
Steffen
,
V.
,
2009
, “
Multiple Surrogates: how Cross-Validation Errors can Help us to Obtain the Best Predictor
,”
Struct. Multidiscipl. Optim.
,
39
(
4
), pp.
439
457
.
25.
Strömberg
,
N.
,
2021
, “
Comparison of Optimal Linear, Affine and Convex Combinations of Metamodels
,”
Eng. Optim.
,
53
(
4
), pp.
702
718
.
26.
Abdallah
,
I.
,
Tatsis
,
K.
,
Mylonas
,
C.
,
Ntertimanis
,
V.
, and
Chatzi
,
E.
,
2020
, “
A Method to Reduce Model Uncertainty by Fusing the Output From Multiple Stochastic Simulators
,”
J. Phys.: Conf. Ser.
,
1618
(
4
), p.
042019
.
27.
Acar
,
E.
,
2010
, “
Various Approaches for Constructing an Ensemble of Metamodels Using Local Measures
,”
Struct. Multidiscipl. Optim.
,
42
(
6
), pp.
879
896
.
28.
Lee
,
Y.
, and
Choi
,
D. H.
,
2014
, “
Pointwise Ensemble of Meta-Models Using v Nearest Points Cross-Validation
,”
Struct. Multidiscipl. Optim.
,
50
(
3
), pp.
383
394
.
29.
Zhang
,
J.
,
Chowdhury
,
S.
, and
Messac
,
A.
,
2012
, “
An Adaptive Hybrid Surrogate Model
,”
Struct. Multidiscipl. Optim.
,
46
(
2
), pp.
223
238
.
30.
Song
,
X.
,
Lv
,
L.
,
Li
,
J.
,
Sun
,
W.
, and
Zhang
,
J.
,
2018
, “
An Advanced and Robust Ensemble Surrogate Model: Extended Adaptive Hybrid Functions
,”
ASME J. Mech. Des.
,
140
(
4
), p.
041402
.
31.
Kaminsky
,
A. L.
,
Wang
,
Y.
, and
Pant
,
K.
,
2021
, “
An Efficient Batch K-Fold Cross-Validation Voronoi Adaptive Sampling Technique for Global Surrogate Modeling
,”
ASME J. Mech. Des.
,
143
(
1
), p.
011706
.
32.
Arlot
,
S.
, and
Celisse
,
A.
,
2010
, “
A Survey of Cross-Validation Procedures for Model Selection
,”
Stat. Surv.
,
4
, pp.
40
79
.
33.
Katoch
,
S.
,
Chauhan
,
S. S.
, and
Kumar
,
V.
,
2021
, “
A Review on Genetic Algorithm: Past, Present, and Future
,”
Multimed. Tools. Appl.
,
80
(
5
), pp.
8091
8126
.
34.
Hamamoto
,
A. H.
,
Carvalho
,
L. F.
,
Sampaio
,
L. D. H.
,
Abrão
,
T.
, and
Proença Jr
,
M. L.
,
2018
, “
Network Anomaly Detection System Using Genetic Algorithm and Fuzzy Logic
,”
Expert Syst. Appl.
,
92
, pp.
390
402
.
35.
Jamil
,
M.
, and
Yang
,
X. S.
,
2013
, “
A Literature Survey of Benchmark Functions for Global Optimisation Problems
,”
Int. J. Math. Model. Numer. Optim.
,
4
(
2
), pp.
150
194
.
36.
Park
,
J. S.
,
1994
, “
Optimal Latin-Hypercube Designs for Computer Experiments
,”
J. Stat. Plan. Inference
,
39
(
1
), pp.
95
111
.
37.
Loeppky
,
J. L.
,
Sacks
,
J.
, and
Welch
,
W. J.
,
2009
, “
Choosing the Sample Size of a Computer Experiment: A Practical Guide
,”
Technimetrics
,
51
(
4
), pp.
366
376
.
38.
Song
,
X. G.
,
Wang
,
L.
, and
Park
,
Y. C.
,
2010
, “
Transient Analysis of a Spring-Loaded Pressure Safety Valve Using Computational Fluid Dynamics (CFD)
,”
ASME J. Pressure Vessel Technol.
,
132
(
5
), p.
054501
.
You do not currently have access to this content.