Seonglae Choinference level computing으로 fast weight 구현하는게 핵심
Currently, there are only two time scales, we need temporary weight changing. We can’t do it now since there are batch processing for production which is more efficient.
- Vector context scale
- Weight learning scale
Neuroplasticity 는 새로운 모델 구조를 필요로 하고 Exploding gradient 같은 테크닉을 통해 AI Feature Dimensionality를 빠르게 높은 Sample efficiency로 변화시킬 수 있어야 한다. 즉 pre-training computational path와 Fast Weight 용 path가 달라야