VAE Training

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2023 Jun 1 2:13
Editor
Edited
Edited
2024 Dec 15 14:34
Refs
Refs
notion image
is for reconstruction
is for regularization to make the approximate posterior close to the prior
 

1. Maximize first term

The first term of ELBO has no analytic solution due to the integral

Decoder Part

We can approximate the expectation with
Monte Carlo Method
(Approximating the expectation by the sample mean)
notion image
we can approximate the derivative w.r.t.
notion image

Encoder Part

we cannot approximate the derivative w.r.t. in this case, because the distribution is replaced by its samples. So we use re-parametrization trick (key trick to train VAE)

Re-parametrization Trick

Some random variables can be represented as a function of another variable.
Any normal distribution can be explained by the standard normal distribution. That is, we can also take a sample of normal distribution using the sample from the standard normal distribution.
notion image
notion image
 

2. Maximize second term

Minimize KL divergence
 
 
 
 

Recommendations