The Reversal Curse

Creator

Creator

Seonglae Cho

Created

Created

2024 Jan 14 11:36

Editor

Editor

Seonglae Cho

Edited

Edited

2026 Apr 9 13:37

Refs

Refs

Implicit Reasoning

Fundamental limitation of Causal LM

Failed to generalize A→B then ⇒ B → A

The Reversal Curse: LLMs trained on "A is B" fail to learn...

We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to...

https://arxiv.org/abs/2309.12288

Reverse Training to Nurse the Reversal Curse

Train by duplicating every word, reversing the training strings, and keeping certain substrings (e.g., entir) unchanged. (

Reverse Training to Nurse the Reversal Curse

HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on.

https://arxiv.org/html/2403.13799v1

Recommendations

////////