Gaussian Mixture Model, Mixture of Gaussian, MoG
The Gaussian Mixture distribution is a linear superposition of Gaussians. Mixture models can be used to build complex distribution and to cluster data.
Each data point is assumed to be generated from a specific Gaussian distribution, where z variables are Latent Variable Multinomial Distribution. is called as mixture weight.
- Mixture weight is an prior probability that component be chosen
- Latent variable is a variable that indicates the selected component as one-hot encoding
- Responsibility is a posterior that was generated by component
we introduce K-dimensional binary random variable z in which only one element zk is equal to 1 and the others are all 0.
Therefore we can induce first equation easily using Marginalization
Responsibility
is responsibility of component for
Likelihood
Log Likelihood
EM Algorithm
E Step
M Step: Solutions