MFR

Creator
Creator
Seonglae Cho
Created
Created
2025 Jan 29 11:18
Editor
Edited
Edited
2025 Mar 8 12:30
Refs
Refs

MFR (Mutual Feature Regularization)

Train two SAEs in parallel, then perform additional training to increase decoder weight similarity. Multimodal Mutual Feature Collaboration (MMCS) reduces
SAE Dead Feature
. Auxiliary penalty MFS was added to loss function.
AuxiliaryPenalty=ij(1MMCS(Wi,Wj))Auxiliary Penalty = \sum_i\sum_j(1-MMCS(W^i,W^j))MMCS(Wi,Wj)=1WiimaxCosSim(wi,wj)MMCS(W^i,W^j) = \frac{1}{|W^i|}\sum_i \max{CosSim(w_i, w_j)}
 
 
 
 

Recommendations