Machine Unlearning

Creator

Creator

Seonglae Cho

Created

Created

2024 May 6 12:56

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Oct 24 23:11

Refs

Refs

An obvious alternative to unlearning is to not learn at all (
RAG)

Exact unlearning (distributionally identical)
Differential privacy

Empirical unlearning (unlearning via fine-tuning)

Machine Unlearning Notion

Latent Knowledge

Unlearning evaluation methods

https://arxiv.org/pdf/2402.16835

Machine Unlearning in 2024

As our ML models today become larger and their (pre-)training sets grow to inscrutable sizes, people are increasingly interested in the concept of machine unlearning to edit away undesired things like private data, stale knowledge, copyrighted materials, toxic/unsafe content, dangerous capabilities, and misinformation, without retraining models from scratch.

Machine Unlearning in 2024

https://ai.stanford.edu/~kzliu/blog/unlearning

NeurIPS 2023 Machine Unlearning Challenge

Website for the NeurIPS 2023 Machine Unlearning Challenge.

https://unlearning-challenge.github.io/

Unlearning Benchmarks

TOFU: A Task of Fictitious Unlearning for LLMs

TOFU: A Task of Fictitious Unlearning for LLMs

https://locuslab.github.io/tofu/

Measuring and Reducing Malicious Use With Unlearning

https://www.wmdp.ai/

LLMs can approximate unlearning when prompted with instructions and demonstrations. (Trivial Prompting)

https://arxiv.org/pdf/2310.07579

Recommendations

//////