LDA is an algorithm that improves upon LSA and is suitable for Topic model.
Let's assume we have a number of topics, each defined as distributions over words. A document is generated through the following process: First, we choose a distribution over the topics. Then, for each word position, we select a topic assignment and choose a word from that corresponding topic.
Method
For each of the K topics, draw a multinomial distribution βk from a Dirichlet distribution with parameter η which controls the mean shape and sparsity of β
For each of the D documents, draw w a multinomial distribution θj from a Dirichlet distribution with parameter α which controls the mean shape and sparsity of θ
For each word position Dji in a document Dj
Select a latent topic zji from the multinomial distribution θj
Choose the observation wji from the multinomial distribution βzji
θ and β has Vparameters where V is the size of the vocabulary across all D documents.
In each step, replace the value of one of the variables by a value drawn from the distribution of that variable conditioned on the values of the remaining variables
Repeat until convergence
Estimate the probability of assigning wji to each topic, conditioned on the topic assignments (zj,−i) of all other words wj,−i(notation indicating the exclusion of wji)
p(zji=kzj,−i,w,α,η)∝Probability that document j choose topic k P(k∣dj)∑k′=1Knj,k′,−i+αk′nj,k,−i+αk⋅∝Probability that topic k generates word wjiP(wji∣k)∑ν=1Vmk,ν,−i+ηνmk,wji,−i+ηwji
From the above conditional distribution, sample a topic and set it as the new topic assignment zji of zji
