AutoEncoder

Autoencoding Model

Neural Network structure which map to the latent space with

Latent Variable. The latent feature should capture most information about data. Then, the data can be almost perfectly reconstructed from its latent feature (reconstructable).

Linear AEs define rotations and linear combinations of dimensions of x. It assumes that there is a latent space, 𝑍, which contains enough structure to sufficiently represent. If MSE of AEs are 0, it would not be useful. Useful in the sense that we want to find a transformation where we observe the most information in the least dimensions possible.

Reconstruction (
Loss Function design)

Guide the latent space based on the desired result. Latent dimension should be as informative of the downstream task as possible.

Reconstruction Error is Loss that the difference between decoded data and original data. It holds significance in the fact that it was able to make breakthroughs in text embedding by restoring the original data without labeling through Loss.

Posterior can be approximated by using encoder network. The decoder of AE can be used for data generation.

Kernel PCA and Isomap require to explicitly describe the distance among data points reduction

We can learn a kernel mapping function from the data itself so we don’t need label. That is why we call them AutoEncoder.

Encoder and Decoder can be parametric functions like neural network including convolution.

Decoder 마지막 activation layer sms reconstruction 위해 없는게 더 성능좋기도 하다

Auto Encoders

LSTM Autoencoder

VAE

Stacked AutoEncoder

Denoising AutoEncoder

Variational AutoEncoder

Masked Autoencoder