Surrogate Model Algorithms for Global Optimization Problems
This talk focuses on algorithms developed for solving computationally expensive black-box global optimization problems. These problems are encountered in application areas such as, for example, structural optimization, carbon sequestration, watershed management, and climate model research where time consuming simulations have to be run in order to obtain objective (and constraint) function values. The algorithms discussed here exploit information from a repeatedly updated surrogate model (also known as response surface model or metamodel), which is a computationally inexpensive approximation of the true objective function and helps to decide at which points in the variable domain the next computationally expensive function evaluation should be done. Hence, compared to optimization algorithms that rely on numerical differentiation or evolutionary strategies, fewer expensive simulations are required to find (near) optimal solutions and the optimization time is significantly lower.
We discuss algorithms for optimization problems with continuous and mixed-integer variables. Within the scope of continuous optimization, various choices of surrogate models have been examined and radial basis functions as well as ensembles of radial basis functions and cubic polynomial regression models proved to be the best. A comparison to NOMAD and PSWARM, which are derivative-free algorithms, showed that the surrogate model algorithms find good solutions more efficiently.
A surrogate model algorithm for mixed-integer problems has been developed and compared in numerical experiments to a genetic algorithm, branch and bound, and NOMAD. The surrogate model algorithm was shown to perform significantly better than the alternative approaches for 10 out of 16 synthetic test problems and for application problems arising in structural optimization and optimal reliability design. Hence, algorithms using surrogate models are a promising option when optimizing computationally expensive black-box problems with integer constraints.