Tree-GP: A Scalable Bayesian Global Numerical Optimization algorithm

Abstract

This paper presents the Tree-GP algorithm: a scalable Bayesian global numerical optimization algorithm. The algorithm focuses on optimizing evaluation functions that are very expensive to evaluate. It models the search space using a mixture model of Gaussian process regression models. This model is then used to find new evaluation points, using our new CMPVR acquisition criteria function that combines both the mean and variance of the predictions made by the model. Conventional Gaussian process based Bayesian optimization algorithms often do not scale well in the total amount of function evaluations. Tree-GP resolves this issue by using a mixture model of Gaussian process regression models stored in a vantage-point tree. This makes the algorithm almost linear in the total amount of function evaluations.