Seonglae ChoThe original Nature article (not paper) did not conduct rigid experiments (the data generation was too naive)
How to avoid collapese: ToEdit (token level edit)
Model Collapse Demystified: The Case of Regression
In the era of large language models like ChatGPT, the phenomenon of ”model collapse” refers to the situation whereby as a model is trained recursively on data generated from previous generations of itself over time, its performance degrades until the model eventually becomes completely useless, i.e the model collapses. In this work, we study this phenomenon in the simplified setting of kernel regression and obtain results which show a clear crossover between where the model can cope with fake data, and a regime where the model’s performance completely collapses. Under polynomial decaying spectral and source conditions, we obtain modified scaling laws which exhibit new crossover phenomena from fast to slow rates. We also propose a simple strategy based on adaptive regularization to mitigate model collapse. Our theoretical results are validated with experiments.
https://arxiv.org/html/2402.07712v1
Verbalized Sampling
Mode collapse (diversity collapse) is not due to algorithmic limitations, but rather because humans prefer familiar and predictable answers, a phenomenon known as "typicality bias".
The typicality term ㅇ is added, resulting in the model producing only more typical and safe answers as the distribution becomes sharper. To mitigate this, the proposed Verbalized Sampling (VS) is a method that only changes prompts without training.
- : People who prefer familiar answers (conservative, prefer predictable responses)
- : Completely objective evaluators without typicality bias (judge only actual quality)
- : People who prefer original or atypical answers (creativity bias)
"Tell me a joke about coffee" → "Generate 5 coffee jokes with probabilities." This way, the model activates various modes, resulting in 1.6–2.1× improvement in creativity and diversity. (Statistical Thinking) In other words, the alignment problem is a cognitive bias issue in human data rather than a learning algorithm problem. VS corrects this at the inference-time prompt level.
