Constrained Bayesian Optimization with Barrier Functions; Impact of heteroscedastic noise on the search and identification phases

Abstract

In recent years, the integration of simulation and optimization techniques has transformed the landscape of problem-solving across various fields, such as engineering and operations management. The combination of advanced modeling methods and increasingly accessible computational power has paved the way for simulation-based optimization approaches, enabling decision-makers to explore alternatives while studying complex systems. This thesis dives deeper into Bayesian optimization, one of the widely used methods to solve expensive simulation optimization problems, focusing on the challenges posed by the presence of heteroscedastic noise in constrained problems. More specifically, the thesis focuses on the impact of heteroscedastic noise on the search and identification phases of constrained Bayesian optimization. Experimental results reveal that employing a surrogate model that naturally handles this type of noise to model the objective and constraint functions, and consequently leveraging the provided information in the acquisition function, significantly changes the sequence of points sampled by the optimization algorithm. Evaluating the system in the most informative locations yields a better estimation of the unknown function behavior. This yields final solutions that are closer to the true optimum of the problem, both in the solution space and in the objective space. Our research also revealed that current identification approaches are shortsighted in identifying the estimated optimal solution, even when the algorithm has actually sampled the true global optimum during the search. This is caused by “misclassification” and “misidentification” errors. Consequently, we propose strategies to minimize the risk of such errors. To minimize the risk of misclassification, we propose a method to improve the accuracy of the estimated probability of feasibility, which is used to distinguish feasible solutions from infeasible ones. To address the misidentification error, we adapt the screen to the best procedure from the literature to account for the remaining uncertainty on the predictor value (which reflects the expected performance of a given solution). This approach allows us to identify the subset of solutions that is statistically non-inferior, and that guarantees the inclusion of the true best solution among the candidate points with a user-defined confidence level. Depending on the magnitude of the remaining uncertainty, the size of the returned subset can vary. To further reduce the subset size (if desired), another acquisition function is proposed. We conclude by analyzing the trade-off between predictor value and corresponding predictor uncertainty, to facilitate the final choice of the best estimated solution. The approaches proposed in this thesis are combined into a single constrained Bayesian optimization algorithm, and tested on a real-world engineering design optimization problem. The resulting algorithm proves to outperform the well-known NOMAD blackbox optimization software, providing solutions that have a lower risk of being infeasible, and yielding more accurate estimations of the objective value with drastically lower computational cost.