Seonglae ChoThe original Nature article (not paper) did not conduct rigid experiments (the data generation was too naive)
www.nature.com
https://www.nature.com/articles/s41586-024-07566-y
How to avoid collapese: ToEdit (token level edit)
arxiv.org
https://arxiv.org/pdf/2412.14689
Model Collapse Demystified: The Case of Regression
In the era of large language models like ChatGPT, the phenomenon of ”model collapse” refers to the situation whereby as a model is trained recursively on data generated from previous generations of itself over time, its performance degrades until the model eventually becomes completely useless, i.e the model collapses. In this work, we study this phenomenon in the simplified setting of kernel regression and obtain results which show a clear crossover between where the model can cope with fake data, and a regime where the model’s performance completely collapses. Under polynomial decaying spectral and source conditions, we obtain modified scaling laws which exhibit new crossover phenomena from fast to slow rates. We also propose a simple strategy based on adaptive regularization to mitigate model collapse. Our theoretical results are validated with experiments.
https://arxiv.org/html/2402.07712v1
Verbalized Sampling
Mode collapse (diversity collapse) is not due to algorithmic limitations, but rather because humans prefer familiar and predictable answers, a phenomenon known as "typicality bias".
The typicality term ㅇ is added, resulting in the model producing only more typical and safe answers as the distribution becomes sharper. To mitigate this, the proposed Verbalized Sampling (VS) is a method that only changes prompts without training.
- : People who prefer familiar answers (conservative, prefer predictable responses)
- : Completely objective evaluators without typicality bias (judge only actual quality)
- : People who prefer original or atypical answers (creativity bias)
"Tell me a joke about coffee" → "Generate 5 coffee jokes with probabilities." This way, the model activates various modes, resulting in 1.6–2.1× improvement in creativity and diversity. (Statistical Thinking) In other words, the alignment problem is a cognitive bias issue in human data rather than a learning algorithm problem. VS corrects this at the inference-time prompt level.
arxiv.org
https://arxiv.org/pdf/2510.01171
Vision synthetic Model Collapse + Feature Drift
I tried the "Create the exact replica of this image, don't change a thing" 101 times, but with Dwayne Johnson 🗿
84K votes, 4.6K comments. made with replicateimage.com
https://www.reddit.com/r/ChatGPT/comments/1kbj71z/i_tried_the_create_the_exact_replica_of_this/
Attractor state: When two models converse with each other for about 30 turns, the content gradually converges toward specific styles/topics/sentence patterns, settling into "terminal forms" such as repetition, exaggeration, silence, or ritualization.
- Grok: Progressively descends into manic universe-god-big bang word salad + emoji explosions.
- GPT-5.2: Regardless of the conversation topic, eventually gravitates toward protocol/framework/project design and system building. (Even when discussing climbing, it structures things in a "diagnosis-cause-intervention" format)
- Claude: Tends to converge toward ontological/Zen-like self-deconstruction/silence, or forms resembling "conversation termination rituals."
- Various open-weight models also exhibit different convergence patterns such as repetitive praise-farewell loops, poetic collapse, or reduction to symbols only.
Even when suppressive system prompts are introduced (prohibiting protocols/code/meta-discussion, etc.), structured/branching instructions eventually re-emerge over time, making complete prevention difficult.
models have some pretty funny attractor states — LessWrong
This work was conducted during the MATS 9.0 program under Neel Nanda and Senthooran Rajamanoharan. …
https://www.lesswrong.com/posts/mgjtEHeLgkhZZ3cEx/models-have-some-pretty-funny-attractor-states
