Fast Weight

Way to combine Training time compute &

Test-time Compute

• Temporally applying
  LoRA is a good candidate

• Forward Forward Algorithm without a whole memory holding is also supportive

Geoffrey Hinton: Using Fast Weights to Store Temporary Memories

Speaker: Geoffrey Hinton, U Toronto

https://www.youtube.com/watch?v=GLmptInTNSw