Seonglae ChoCatastrophic interference
새로운 정보를 학습할 때 이전에 학습한 정보를 갑자기 급격하게 잊어버리는 경향
Cross Entropy 기반의 loss를 사용하여 딥뉴럴넷 학습은 은 현재 데이터에 대한 분포를 추정하는 것이므로, 이전 데이터에 대한 분포는 고려하지 못한채 학습을 하는 것
scaling으로 해결 there is a capacity.
Catastrophic interference
Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the network approach and connectionist approach to cognitive science. With these networks, human capabilities such as memory and learning can be modeled using computer simulations.
https://en.wikipedia.org/wiki/Catastrophic_interference
