Seonglae ChoAn obvious alternative to unlearning is to not learn at all (RAG)
- Exact unlearning (distributionally identical) Differential privacy
- Empirical unlearning (unlearning via fine-tuning)
Machine Unlearning Notion
Machine Unlearning Methods
Open problems
Unlearning offers an incomplete solution for impermissible knowledge removal in LLMs with strong In-context learning capability
LLMs can approximate unlearning when prompted with instructions and demonstrations. (Trivial Prompting)