Balancing exploration and exploitation by utilizing the model's mean (prediction) and uncertainty (variance)
Bayesian Optimization Process
- Choose a Surrogate model for modeling the true function and define its prior over the space of objective functions to model our black-box function.
- Bayes Update: obtain or update Surrogate Posterior using Bayes’ rule by incorporating set of observations into the surrogate model
- Use Acquisition functions , which depend on the surrogate posterior, to determine next sample evaluation point
- Add newly sampled data to the set of observations and goto step #2 till convergence or budget elapses
