Seonglae ChoLikelihood Weighting
Not a mathematical proposal, just a real approximation
The stochasticity trick of Monte Carlo method (Importance sampling)
Assume you have a difficult integral to compute
In short, Monte Carlo methods enable us to estimate any integral by random sampling. InBayesian Statistics, Evidence is also form of integral so it becomes tractable.
Self-normalized importance sampling
Small setback: for the particular case where our function to integrate is the posterior , we can only evaluate up to a constant.
When we define unnormalized weights, which can be used to approximate the Evidence.
Use the unnormalized importance weights with the same sampled value from , to estimate both the numerator and the denominator.
And then, we could normalize unnormalized importance weight into , then
After that we can compute Posterior with Delta function for point mass
Annealed Importance Sampling (AIS)
Instead of using a fixed proposal distribution, samples are drawn through a process of gradually changing distributions, transitioning to the target distribution through intermediate distributions MCMC
해당 trajectory 발생 확률만큼만 Off-policy data 고려할 수 있도록 수학적으로 사용해서 데이터 더 많이 사용하고 importance만큼 고려
